Intelligent manufacturing scheduling method and device, electronic equipment and medium

Abstract

The application discloses an intelligent manufacturing scheduling method and device, electronic equipment and medium. Through the application of the technical solution, the total cost including product processing cost, overtime cost of employees, product delay delivery cost, and product inventory cost is taken as the optimization target. And combined with the deep reinforcement learning algorithm, the scheduling plan corresponding to the order information is output at the lowest cost. Thus, on the one hand, the production manufacturer can automatically give a scheduling scheme within a period of time in real time. On the other hand, the scheduling scheme is not accurate due to the problem that the employee production cost in the production process is not considered in the related art.

Description

Technical Field

[0001] This application relates to product planning and generation technology, and in particular to a smart manufacturing scheduling method, apparatus, electronic equipment, and medium. Background Technology

[0002] With the rapid development of new-generation information technologies such as the Industrial Internet, big data, and artificial intelligence, the theory of intelligent manufacturing is also constantly advancing. Currently, intelligent manufacturing has become one of the core technologies for enhancing the overall competitiveness of the manufacturing industry.

[0003] In the smart manufacturing industry, a key aspect of related technologies, order planning and scheduling is crucial. For instance, by integrating materials, product production processes, machines, and employees into a smart scheduling system, customer orders within a given period can be used as input, and various optimization algorithms can be employed to generate the optimal scheduling plan for that period.

[0004] However, the existing production process based on product planning does not take into account the production costs of employees during the production process, which leads to inaccurate production scheduling plans and affects the progress of business processing. Summary of the Invention

[0005] This application provides a smart manufacturing scheduling method, apparatus, electronic device, and medium. It addresses the shortcomings of existing technologies where scheduling schemes are inaccurate because they do not consider employee production costs during the production process.

[0006] According to one aspect of the embodiments of this application, a smart manufacturing scheduling method is provided, wherein:

[0007] Obtain basic manufacturer information and order information within the target time period. The basic manufacturer information includes employee production information that reflects employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs.

[0008] Based on the manufacturer's basic information and the order information, multiple state feature vectors are obtained, where each state feature vector is used to reflect the production status of the production machine or the production status of the product.

[0009] The multiple feature vectors are used as input to the reinforcement learning model so that the reinforcement learning model outputs a target scheduling plan that matches the multiple feature vectors. The target scheduling plan is used to reflect the range of target products to be produced and the corresponding target production machines required to process the order information at the lowest cost.

[0010] Based on the target scheduling plan, production is scheduled and manufactured according to the order information;

[0011] The target scheduling plan includes one of the following:

[0012] The following options are available: Select the product range with the minimum delay time and the corresponding production machine scheduling plan; Select the product range with the minimum ratio of remaining delivery time to remaining production time and the corresponding production machine scheduling plan; Select the product range with the maximum delay time and the corresponding production machine scheduling plan; Randomly select the product range and the corresponding production machine scheduling plan; Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

[0013] Optionally, in another embodiment based on the method described above in this application, obtaining multiple state feature vectors based on the manufacturer's basic information and the order information includes:

[0014] The manufacturer's basic information and the order information are input into the intelligent manufacturing scheduling system to obtain multiple state feature vectors that match the manufacturer's basic information and the order information;

[0015] The manufacturer's basic information includes:

[0016] Production machine processing time, employee attendance information, number of production machines on different production lines, production costs and employee overtime costs on different production lines, and product cost information.

[0017] Optionally, in another embodiment based on the method described above in this application, the state feature vector includes:

[0018] The vectors are: an average utilization rate vector reflecting the resource utilization rate of the production machines; a utilization rate variance vector reflecting the load balancing status of each production machine; a vector of total process completion rate and vectors of different product completion rates reflecting the processing progress of the order information; and a vector of estimated delay rate and vector of actual delay rate reflecting the delayed progress of the order information.

[0019] Optionally, in another embodiment based on the method described above, after scheduling production based on the target scheduling plan for the order information, the method further includes:

[0020] Based on the target product range and corresponding target production machine reflected in the target scheduling plan, determine the production time required for the target production machine to produce the target product quantity.

[0021] Based on the start processing time of the target production machine and the production duration, calculate the overtime cost corresponding to the target scheduling plan; and obtain the product delay cost included in the order information.

[0022] Based on the relationship between the overtime cost and the product delay cost, the reinforcement learning model is optimized.

[0023] Optionally, in another embodiment based on the method described above, optimizing the reinforcement learning model based on the relationship between the overtime cost and the product delay cost includes:

[0024] If the overtime cost is detected to be greater than the product delay cost, the incentive function corresponding to the target scheduling plan will be marked as negative.

[0025] The reinforcement learning model is optimized using the activation function marked with a negative value.

[0026] Optionally, in another embodiment based on the method described above, after scheduling production based on the target scheduling plan for the order information, the method further includes:

[0027] Select a first number of samples from the data sample pool;

[0028] Using the mean squared error loss function and the sample set, the reinforcement learning model is trained by backpropagation, and the parameters of the reinforcement learning model are updated until it is determined that the reinforcement learning model has been trained.

[0029] Optionally, in another embodiment based on the method described above in this application, the step of training the reinforcement learning model through backpropagation includes:

[0030] The average utilization rate vector, which reflects the resource utilization rate of the production machine, and the estimated delay rate vector and the actual delay rate vector, which reflect the delay progress of the order information, are used as reward functions.

[0031] The reinforcement learning model is trained using the reward function via backpropagation.

[0032] According to another aspect of the embodiments of this application, a smart manufacturing scheduling device is provided, wherein:

[0033] The acquisition module is configured to acquire basic manufacturer information and order information within a target time period. The basic manufacturer information includes employee production information reflecting employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs.

[0034] The conversion module is configured to obtain multiple state feature vectors based on the manufacturer's basic information and the order information, wherein each state feature vector is used to reflect the production status of the production machine or the production status of the product.

[0035] The output module is configured to take the plurality of feature vectors as input to the reinforcement learning model, so that the reinforcement learning model outputs a target scheduling plan that matches the plurality of feature vectors. The target scheduling plan is used to reflect the range of target products to be produced and the corresponding target production machines required to process the order information at the lowest cost.

[0036] The processing module is configured to schedule production based on the target scheduling plan and the order information.

[0037] The target scheduling plan includes one of the following:

[0038] The following options are available: Select the product range with the minimum delay time and the corresponding production machine scheduling plan; Select the product range with the minimum ratio of remaining delivery time to remaining production time and the corresponding production machine scheduling plan; Select the product range with the maximum delay time and the corresponding production machine scheduling plan; Randomly select the product range and the corresponding production machine scheduling plan; Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

[0039] According to another aspect of the embodiments of this application, an electronic device is provided, comprising:

[0040] Memory, used to store executable instructions; and

[0041] A display is used in conjunction with the memory to execute the executable instructions to perform the operation of any of the above-described intelligent manufacturing scheduling methods.

[0042] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided for storing computer-readable instructions, which, when executed, perform the operations of any of the above-described intelligent manufacturing scheduling methods.

[0043] This application obtains basic manufacturer information reflecting employee overtime costs, as well as order information including order quantity, product delivery date, and product delay costs. Based on the manufacturer information and order information, multiple state feature vectors are obtained. These feature vectors are used as input to a reinforcement learning model, enabling the model to output a target scheduling plan that matches the feature vectors. The target scheduling plan reflects the target product quantity and corresponding target production machine required to process the order information at the lowest cost. Based on the target scheduling plan, production is scheduled for the order information. By applying the technical solution of this application, the total cost, including product processing costs, employee overtime costs, product delay delivery costs, and product inventory costs, can be optimized as the target. Combined with a deep reinforcement learning algorithm, it outputs the scheduling plan corresponding to processing order information at the lowest cost to the user. This allows for real-time automatic provision of production scheduling plans for manufacturers over a period of time. Furthermore, it avoids the shortcomings of related technologies where the production scheduling plan does not consider the employee production costs during the production process, resulting in inaccurate scheduling plans.

[0044] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0045] The accompanying drawings, which form part of this specification, illustrate embodiments of this application and, together with the description, serve to explain the principles of this application.

[0046] This application can be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0047] Figure 1 A schematic diagram of an intelligent manufacturing scheduling method provided in an embodiment of this application is shown;

[0048] Figure 2 A schematic diagram of the training process for reinforcement learning model parameters provided in an embodiment of this application is shown;

[0049] Figure 3 A schematic diagram of the network architecture of a reinforcement learning model provided in an embodiment of this application is shown;

[0050] Figure 4 A flowchart illustrating the practical application of a reinforcement learning model provided in an embodiment of this application is shown.

[0051] Figure 5 This invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0052] Figure 6 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0053] Figure 7 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0054] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0055] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0056] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this application or its application or use.

[0057] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0058] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0059] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0060] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0061] The following is combined with Figures 1-4 This application describes a method for intelligent manufacturing scheduling according to exemplary embodiments thereof. It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario.

[0062] This application also proposes a smart manufacturing scheduling method, apparatus, electronic equipment, and medium.

[0063] Figure 1 A schematic flowchart illustrating an intelligent manufacturing scheduling method according to an embodiment of this application is shown. Figure 1 As shown, the method includes:

[0064] S101, Obtain basic manufacturer information and order information within the target time period. The basic manufacturer information includes employee production information to reflect employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs.

[0065] S102, based on the manufacturer's basic information and the order information, multiple state feature vectors are obtained, wherein each state feature vector is used to reflect the production status of the production machine or the production status of the product.

[0066] S103, multiple feature vectors are used as input to the reinforcement learning model so that the reinforcement learning model outputs a target scheduling plan that matches the multiple feature vectors. The target scheduling plan is used to reflect the number of target products to be produced and the corresponding target production machines required to process order information at the lowest cost.

[0067] S104, based on target scheduling planning, schedules production based on order information.

[0068] In related technologies, current generative planning algorithms tend to solve product planning and design problems within a short period (such as within a day), with the optimization goal of maximizing machine utilization or minimizing product delays, and heuristic algorithms such as exact algorithms or genetic algorithms are used to solve the problem.

[0069] However, the current algorithm has certain technical limitations. First, solving planning problems within short cycles requires pre-determining the daily quantity of products to be processed, which necessitates additional personnel in actual production and reduces the level of automation. Furthermore, in the manufacturing of large industrial products, there are many cases with long production cycles, such as transformers and machine tools, where the production cycle of some processes can even exceed one day. Existing algorithms struggle to handle the scheduling problems of such large industrial products.

[0070] In addition, many current studies have neglected the constraint of employee attendance time, failed to consider the impact of employees' commuting time on the product processing sequence, and failed to consider the overtime costs that may be incurred when employees perform processing procedures outside of working hours.

[0071] Furthermore, while operations research algorithms offer high accuracy and guarantee optimal solutions, their extremely long computation time, often requiring several hours, is unacceptable in practical production. Genetic algorithms, on the other hand, are among the most commonly used optimization algorithms, offering shorter runtimes and ensuring solution quality, making them widely applicable in planning and scheduling algorithms. However, their efficiency is unsatisfactory when solving large-scale planning problems, and the solution quality declines with increasing scale. Additionally, while heuristic planning methods based on priority scheduling offer fast solution speeds, their generalization performance is poor; rules that perform well in one scenario may not guarantee the same results in others, as no single rule is optimal in all situations.

[0072] Finally, considering only machine utilization and delays in terms of optimization objectives is insufficient. In actual production, delays lead to costs associated with postponed delivery, but completing processing ahead of schedule also incurs inventory costs, and half-finished products remaining on the production line result in semi-finished inventory costs. These costs are of great concern to managers.

[0073] To address the aforementioned problems, this application provides an intelligent manufacturing scheduling method. The idea is to take the total cost, including product processing cost, employee overtime cost, product delayed delivery cost, and product inventory cost, as the optimization target, and combine it with a deep reinforcement learning algorithm to output the scheduling plan corresponding to processing order information at the lowest cost to the user.

[0074] like Figure 2 As shown below, the intelligent manufacturing scheduling method proposed in this application will be explained in detail:

[0075] Step 1: Collect basic information about the manufacturers, including their production information, product information, and employee production information.

[0076] The manufacturer's production information and manufacturer's product information include:

[0077] (1) The processing time of each process of each production task on different production machines.

[0078] (2) The attendance time of employees on different production lines.

[0079] (3) The capacity of different production lines is reflected in the number of production machines on different production lines.

[0080] (4) Normal production costs and employee overtime costs on different production lines.

[0081] (5) Inventory costs of different types of products and semi-finished product inventory costs.

[0082] Understandably, since the above data is usually relatively stable in actual production and does not change frequently, this application embodiment integrates the above data as parameters into the algorithm.

[0083] Step 2: Obtain order information.

[0084] The order information can be order information within a target time period (e.g., one day or one week), and includes the order quantity, product delivery date, and product delay costs.

[0085] Step 3: Based on the manufacturer's basic information and order information, obtain multiple state feature vectors, where each state feature vector is used to reflect the production status of the production machine or the production status of the product.

[0086] Furthermore, this application can obtain multiple state feature vectors that match the manufacturer's basic information and order information by inputting the order information, including the order quantity, product delivery date, and product delay cost, as well as the pre-collected manufacturer production information and manufacturer product information, into the intelligent manufacturing scheduling system.

[0087] It should be noted that the following explains the various parameters subsequently proposed in this application:

[0088]

[0089]

[0090] Based on the above parameters, the state feature vector proposed in this application is explained below:

[0091] (1) Average utilization vector, used to reflect the utilization efficiency of resources of each production machine in a manufacturer. Denoteed as Uave, where,

[0092] U m The utilization rate of production machine m.

[0093] in,

[0094] (2) The utilization variance vector, which represents the degree of load balance among different production machines in the current system, is denoted as U. std .

[0095] in,

[0096] (3) Total process completion rate, which describes the current progress of order planning. A value close to 0 indicates that planning has just begun, and a value close to 1 indicates that planning is about to end. It is denoted as CRT.

[0097] in,

[0098] (4) The average completion rate of each product, used to describe the completion rate of the current plan, is denoted as CRJ. ave .in, Among them CRJ i This represents the process completion rate of product i. Furthermore,

[0099] (5) The variance of the completion rate of different products can describe the degree of uniformity in the completion of different products. The larger the variance, the greater the difference in the degree of completion of different products. It is denoted as CRJ. std .

[0100] in,

[0101] (6) Estimated delay rate of the product. Estimated delay means that although there is no delay in the current plan, the current time plus the remaining processing time will exceed the delivery date, that is, a delay will occur in the future. The estimated delay rate is denoted as Te.

[0102] in, N left N represents the number of products that have not yet been processed. te This indicates the number of products that may experience delivery delays.

[0103] (7) Actual delay rate of the product. Actual delay means that the product has been delayed at the current time. Let Ta be the actual delay rate.

[0104] Step 4: Obtain the preset reinforcement learning model.

[0105] In this embodiment, the reinforcement learning model adopts the DQN structure. DQN is one of the most common algorithms in reinforcement learning and has been widely used in many fields.

[0106] In one approach, this application can employ the Double DQN algorithm to implement the output of the target scheduling plan. The Double DQN algorithm optimizes upon DQN by using two identical neural networks to address the correlation between data samples and network training, thus avoiding overestimation caused by the Q-value excessively converging towards the potential optimization target during training.

[0107] In one embodiment, the reinforcement learning model in this application has five hidden layers between the input and output layers, each with 30 nodes and an activation function of tanh. The neural structure of the Q-network is as follows: Figure 3 As shown.

[0108] Step 5: Use multiple feature vectors as input to the reinforcement learning model so that the output of the reinforcement learning model matches the target scheduling plan of the multiple feature vectors.

[0109] Furthermore, this application embodiment utilizes a reinforcement learning model to output the target scheduling plan with the lowest cost based on different state feature vectors. That is, the target scheduling plan determines the range of target products to be produced and the corresponding target production machines for processing order information at the lowest cost.

[0110] Specifically, the target scheduling plan is one of multiple scheduling plans. This application can use multiple feature vectors as input to a reinforcement learning model and select a target scheduling plan that matches the feature vectors from multiple scheduling plans.

[0111] The scheduling plans in this application include the following:

[0112] (1): Select the product range with the minimum delay time and the corresponding production machine scheduling plan.

[0113] This means selecting the product with the shortest average delay relaxation time and the production machine that finishes processing earliest. For example, we can denote the actual delayed product set D and the unfinished product set U. If D is not empty, select the product with the longest actual delay time; if D is empty, select the set S of products that have already undergone processing from the unfinished set U.

[0114] This strategy prioritizes processing products that have already undergone processing, avoiding excessively long time intervals between different processes of the same product, thereby reducing semi-finished product inventory costs. From the product set S, the product with the shortest average delay slack time, i.e., the one most likely to experience a delay, is selected. The average delay slack time of a product is equal to the remaining processing time divided by the remaining number of processes.

[0115] Additionally, when selecting production machines, choose the one that finishes processing earliest. This requires considering the machines' start and end times. By weighing overtime costs against delay costs, determine when to begin processing the product on each machine. In one approach, if overtime costs exceed the cost of delay, it's not worthwhile to work overtime, and the product should be processed after the next workday. If overtime costs are lower than delay costs, the delay costs are too high, and starting processing immediately is more cost-effective.

[0116] (2) Select the product range and corresponding production machine scheduling plan with the minimum ratio of remaining delivery time to remaining production time.

[0117] In other words, select the product with the smallest ratio of remaining delivery time to remaining processing time, plus the production machine that finishes processing earliest. For example, we can denote the delay set D and the unfinished product set U. If D is not empty, select the product with the largest estimated delay time; if D is empty, select the set S of products that have already been processed from the unfinished set U, and then select the product with the smallest ratio of relaxation time to remaining processing time from S.

[0118] Understandably, a smaller ratio indicates a greater difficulty in delivering the product on time. If S is empty, select from U. When choosing a production machine, choose the one that finishes processing earliest.

[0119] (3) Select the product range with the maximum delay time and the corresponding production machine scheduling plan.

[0120] In other words, select the product with the longest delay time and the production machine with the lowest utilization rate or the highest workload. Specifically, for all unfinished products, calculate their possible delay time and select the product with the longest delay time, prioritizing products that have already undergone some processing steps. When selecting production machines, choose the production machine with the lowest utilization rate with a 50% probability, and choose the production machine with the highest workload with a certain probability (e.g., 50%).

[0121] (4) Select the product range with the maximum delay time and the corresponding production machine scheduling plan.

[0122] In other words, while selecting the product with the longest delay, it's also necessary to choose the production machine that finishes processing earliest. Specifically, for all unfinished products, calculate their potential delay time and select the product with the longest delay, prioritizing those products that have already undergone some processing steps. When selecting production machines, choose the one that finishes processing earliest.

[0123] (5) Randomly select the product range and the corresponding production machine scheduling plan.

[0124] That is, randomly select a ready product. When selecting a production machine, choose the machine that finishes processing earliest.

[0125] (6) Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

[0126] In other words, select the product with the greatest estimated delay and the production machine that finishes processing earliest. The estimated delay is represented by the inverse product of the estimated delay time and the product completion rate; the higher the value, the more severe the delay.

[0127] For example, let D be the delay set and U be the set of unfinished products. If D is not empty, select the product with the largest estimated delay. If D is empty, select the set S of products that have already undergone processing from the unfinished set U, and then select the product with the smallest product of remaining delivery time and product completion rate from S. If S is empty, then select from U. When selecting production machines, choose the production machine that completes processing earliest.

[0128] As can be seen from the above, this application uses order information acquired over a period of time, and based on the order quantity, product delivery date, and product delay costs contained in the order information, as well as the manufacturer's current production and product information, to convert the order information into a corresponding production status feature vector of the production machine processing the order, or a production status feature vector of the product. By combining this with a reinforcement learning model, a scheduling plan corresponding to this status feature vector is automatically output, enabling subsequent planning and processing of the order information based on this scheduling plan.

[0129] In this application, basic information about the manufacturers that reflects employee production information to reflect overtime costs, as well as order information including order quantity, product delivery date, and product delay costs, can be obtained. Based on the basic information about the manufacturers and the order information, multiple state feature vectors are obtained. These multiple feature vectors are used as input to a reinforcement learning model so that the reinforcement learning model outputs a target scheduling plan that matches the multiple feature vectors. The target scheduling plan is used to reflect the target product quantity and corresponding target production machine required to process the order information at the lowest cost. Based on the target scheduling plan, production is scheduled for the order information.

[0130] By applying the technical solution of this application, the total cost, including product processing costs, employee overtime costs, product delayed delivery costs, and product inventory costs, can be optimized as the target. Combined with a deep reinforcement learning algorithm, it outputs the scheduling plan corresponding to processing order information at the lowest cost to the user. This allows for real-time automatic provision of production scheduling plans for manufacturers over a period of time. Furthermore, it avoids the shortcomings of related technologies where the production scheduling plan is inaccurate due to the failure to consider employee production costs during the production process.

[0131] Optionally, in another embodiment based on the method described above in this application, multiple state feature vectors are obtained based on the manufacturer's basic information and the order information, including:

[0132] The manufacturer's basic information and the order information are input into the intelligent manufacturing scheduling system to obtain multiple state feature vectors that match the manufacturer's basic information and the order information;

[0133] The manufacturer's basic information includes:

[0134] Production machine processing time, employee attendance information, number of production machines on different production lines, production costs and employee overtime costs on different production lines, and product cost information.

[0135] Optionally, in another embodiment based on the method described above in this application, the state feature vector includes:

[0136] The vectors are: an average utilization rate vector reflecting the resource utilization rate of the production machines; a utilization rate variance vector reflecting the load balancing status of each production machine; a vector of total process completion rate and vectors of different product completion rates reflecting the processing progress of the order information; and a vector of estimated delay rate and vector of actual delay rate reflecting the delayed progress of the order information.

[0137] Optionally, in another embodiment based on the method described above, after scheduling production based on the target scheduling plan for the order information, the method further includes:

[0138] Based on the target product range and corresponding target production machine reflected in the target scheduling plan, determine the production time required for the target production machine to produce the target product quantity.

[0139] Based on the start processing time of the target production machine and the production duration, calculate the overtime cost corresponding to the target scheduling plan; and obtain the product delay cost included in the order information.

[0140] Based on the relationship between the overtime cost and the product delay cost, the reinforcement learning model is optimized.

[0141] Optionally, in another embodiment based on the method described above, optimizing the reinforcement learning model based on the relationship between the overtime cost and the product delay cost includes:

[0142] If the overtime cost is detected to be greater than the product delay cost, the incentive function corresponding to the target scheduling plan will be marked as negative.

[0143] The reinforcement learning model is optimized using the activation function marked with a negative value.

[0144] In one embodiment of this application, after obtaining the target scheduling plan output by the reinforcement learning model, the overtime cost and the corresponding delay cost of processing the target product and the corresponding target production machine reflected in the target scheduling plan can be compared, and the model can be optimized based on the comparison results.

[0145] Specifically, if the overtime cost required to execute the target scheduling plan is large (i.e., greater than the delay cost), it indicates that the target scheduling plan incurs high costs. Therefore, this embodiment of the application can mark the output of the reinforcement learning model (i.e., the target scheduling plan) as a negative activation function to prevent the reinforcement learning model from outputting the target scheduling plan for the order information again. This reduces delay costs, overtime costs, production machine start-up costs, and semi-finished product inventory costs, thereby ensuring that the output target of the reinforcement model is optimal.

[0146] Optionally, in another embodiment based on the method described above, after scheduling production based on the target scheduling plan for the order information, the method further includes:

[0147] Select a first number of samples from the data sample pool;

[0148] Using the mean squared error loss function and the sample set, the reinforcement learning model is trained by backpropagation, and the parameters of the reinforcement learning model are updated until it is determined that the reinforcement learning model has been trained.

[0149] In each scheduling execution, the system state is updated and a reward is obtained by executing the scheduling plan provided by the model, resulting in a quadruple (s, a, r, s_) consisting of the original system state s, action a, reward r, and the updated system state s_. This quadruple is stored in a data sample pool. This ensures that after each subsequent scheduling execution, a certain number of samples are selected from the data sample pool, and the mean squared error loss function (y_) is applied. j -Q(v j ,a j )) 2 The parameters of the network Q are updated by backpropagation of gradients to the reinforcement learning model, and the learning continues until the training termination condition is met.

[0150] Furthermore, after obtaining the trained reinforcement learning model, it can be stored. This allows for the subsequent execution of the intelligent manufacturing scheduling method proposed in this application using the trained reinforcement learning model. As an example, the execution steps can be as follows: Figure 4 As shown.

[0151] Optionally, in another embodiment based on the method described above in this application, training the reinforcement learning model using backpropagation includes:

[0152] The average utilization rate vector, which reflects the resource utilization rate of production machines, and the estimated delay rate vector and actual delay rate vector, which reflect the delay progress of order information, are used as reward functions.

[0153] The reinforcement learning model is trained using backpropagation with a reward function.

[0154] In one embodiment of this application, the reinforcement learning model can be further optimized and trained by designing a reward function. The reward function includes three parts: the actual latency rate of the product, the estimated latency rate, and the average utilization rate of the production machine.

[0155] Understandably, using these three feature vectors as the reward function allows for the most compact process arrangement, thus shortening the total processing time. A more compact processing flow effectively reduces delay costs, overtime costs, production machine uptime costs, and semi-finished product inventory costs, thereby ensuring the optimal overall objective.

[0156] In one alternative approach, to further optimize inventory costs, embodiments of this application may perform an operation to determine whether the start time of each process step on different production machines can be postponed, working backwards from the end. Based on the determination result, it is then determined whether postponing this operation can reduce inventory costs without affecting other costs (such as overtime costs and delay costs). If so, the start time of the process is postponed; otherwise, no adjustment is made.

[0157] By applying the technical solution of this application, the total cost, including product processing costs, employee overtime costs, product delayed delivery costs, and product inventory costs, can be optimized as the target. Combined with a deep reinforcement learning algorithm, it outputs the scheduling plan corresponding to processing order information at the lowest cost to the user. This allows for real-time automatic provision of production scheduling plans for manufacturers over a period of time. Furthermore, it avoids the shortcomings of related technologies where the production scheduling plan is inaccurate due to the failure to consider employee production costs during the production process.

[0158] Understandably, the intelligent manufacturing scheduling method proposed in this application incorporates constraints related to employee attendance costs into the reinforcement learning model. It also uses the total cost, including product processing costs, employee overtime costs, product delay delivery costs, product inventory costs, and semi-finished product inventory costs, as the optimization objective. This results in a more accurate output of the scheduling plan.

[0159] Furthermore, since the action set of the reinforcement learning algorithm proposed in this application is represented in the form of rules, it includes both rules for selecting products and rules for selecting production machines. In the rules for selecting products, products are divided into three categories: delayed products, unfinished products, and semi-finished products. When selecting products, priority is given to delayed products to minimize delay costs; secondly, selection is made from semi-finished products to reduce semi-finished product inventory costs; only when both of these product sets are empty is a product selected from unfinished products.

[0160] Secondly, since the reinforcement learning scheduling environment proposed in this application also uses the above comparison method to determine the start processing time of a product on a certain production machine, in order to select the start processing time that minimizes the total cost.

[0161] Finally, this application proposes a heuristic optimization method to reduce product inventory costs and semi-finished product inventory costs in a reinforcement learning scheduling environment. For processing sequences on different production machines, a judgment operation is performed on each process from back to front to determine whether the start time of the process can be postponed, and whether this postponement can reduce inventory costs without affecting other costs (such as overtime costs and delay costs). If it can, the start time of the process is postponed; otherwise, no adjustment is made.

[0162] Optionally, in another embodiment of this application, such as Figure 5 As shown, this application also provides an intelligent manufacturing scheduling device. It includes:

[0163] The acquisition module 201 is configured to acquire basic manufacturer information and order information within a target time period. The basic manufacturer information includes employee production information reflecting employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs.

[0164] The conversion module 202 is configured to obtain multiple state feature vectors based on the manufacturer's basic information and the order information, wherein each state feature vector is used to reflect the production status of the production machine or the production status of the product.

[0165] The output module 203 is configured to take the plurality of feature vectors as input to the reinforcement learning model, so that the reinforcement learning model outputs a target scheduling plan that matches the plurality of feature vectors. The target scheduling plan is used to reflect the range of target products to be produced and the corresponding target production machines required to process the order information at the lowest cost.

[0166] Processing module 204 is configured to schedule production based on the target scheduling plan for the order information;

[0167] The target scheduling plan includes one of the following:

[0168] The following options are available: Select the product range with the minimum delay time and the corresponding production machine scheduling plan; Select the product range with the minimum ratio of remaining delivery time to remaining production time and the corresponding production machine scheduling plan; Select the product range with the maximum delay time and the corresponding production machine scheduling plan; Randomly select the product range and the corresponding production machine scheduling plan; Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

[0169] By applying the technical solution of this application, the total cost, including product processing costs, employee overtime costs, product delayed delivery costs, and product inventory costs, can be optimized as the target. Combined with a deep reinforcement learning algorithm, it outputs the scheduling plan corresponding to processing order information at the lowest cost to the user. This allows for real-time automatic provision of production scheduling plans for manufacturers over a period of time. Furthermore, it avoids the shortcomings of related technologies where the production scheduling plan is inaccurate due to the failure to consider employee production costs during the production process.

[0170] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0171] The manufacturer's basic information and the order information are input into the intelligent manufacturing scheduling system to obtain multiple state feature vectors that match the manufacturer's basic information and the order information;

[0172] The manufacturer's basic information includes:

[0173] Production machine processing time, employee attendance information, number of production machines on different production lines, production costs and employee overtime costs on different production lines, and product cost information.

[0174] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0175] The vectors are: an average utilization rate vector reflecting the resource utilization rate of the production machines; a utilization rate variance vector reflecting the load balancing status of each production machine; a vector of total process completion rate and vectors of different product completion rates reflecting the processing progress of the order information; and a vector of estimated delay rate and vector of actual delay rate reflecting the delayed progress of the order information.

[0176] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0177] Based on the target product range and corresponding target production machine reflected in the target scheduling plan, determine the production time required for the target production machine to produce the target product quantity.

[0178] Based on the start processing time of the target production machine and the production duration, calculate the overtime cost corresponding to the target scheduling plan; and obtain the product delay cost included in the order information.

[0179] Based on the relationship between the overtime cost and the product delay cost, the reinforcement learning model is optimized.

[0180] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0181] If the overtime cost is detected to be greater than the product delay cost, the incentive function corresponding to the target scheduling plan will be marked as negative.

[0182] The reinforcement learning model is optimized using the activation function marked with a negative value.

[0183] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0184] Select a first number of samples from the data sample pool;

[0185] Using the mean squared error loss function and the sample set, the reinforcement learning model is trained by backpropagation, and the parameters of the reinforcement learning model are updated until it is determined that the reinforcement learning model has been trained.

[0186] In another embodiment of this application, the conversion module 202 is configured to perform the following steps:

[0187] The average utilization rate vector, which reflects the resource utilization rate of the production machine, and the estimated delay rate vector and the actual delay rate vector, which reflect the delay progress of the order information, are used as reward functions.

[0188] The reinforcement learning model is trained using the reward function via backpropagation.

[0189] This application also provides an electronic device for executing the above-described intelligent manufacturing scheduling method. Please refer to... Figure 6 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 6 As shown, the electronic device 3 includes: a processor 300, a memory 301, a bus 302, and a communication interface 303. The processor 300, the communication interface 303, and the memory 301 are connected through the bus 302. The memory 301 stores a computer program that can run on the processor 300. When the processor 300 runs the computer program, it executes the intelligent manufacturing scheduling method provided in any of the foregoing embodiments of this application.

[0190] The memory 301 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 303 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0191] Bus 302 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs. After receiving an execution instruction, the processor 300 executes the program. The data recognition method disclosed in any of the foregoing embodiments of this application can be applied to the processor 300, or implemented by the processor 300.

[0192] Processor 300 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 300 or through software instructions. The processor 300 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an On-Premises Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0193] In one embodiment, the processor 300 can also be a graphics processing unit (GPU). It can implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 301, and the processor 300 reads the information in memory 301 and, in conjunction with its hardware, completes the steps of the above methods.

[0194] The electronic device provided in this application embodiment and the intelligent manufacturing scheduling method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0195] This application also provides a computer-readable storage medium corresponding to the intelligent manufacturing scheduling method provided in the foregoing embodiments. Please refer to... Figure 7 The computer-readable storage medium shown is an optical disc 40, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the intelligent manufacturing scheduling method provided in any of the foregoing embodiments.

[0196] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0197] The computer-readable storage medium provided in the above embodiments of this application and the data identification method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0198] It should be noted that:

[0199] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0200] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0201] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0202] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A smart manufacturing scheduling method, characterized in that, in: Obtain basic manufacturer information and order information within the target time period. The basic manufacturer information includes employee production information reflecting employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs. The basic manufacturer information includes: production machine processing time, employee attendance information, number of production machines on different production lines, production costs and employee overtime costs on different production lines, and product cost information. Based on the manufacturer's basic information and the order information, multiple state feature vectors are obtained, each of which reflects the production status of the production machine or the production status of the product. The state feature vectors include: an average utilization rate vector reflecting the resource utilization rate of the production machine; a utilization rate variance vector reflecting the load balancing status of each production machine; a total process completion rate vector and a vector reflecting the completion rate of different products for the processing progress of the order information; and an estimated delay rate vector and an actual delay rate vector reflecting the delayed progress of the order information. The multiple feature vectors are used as input to the reinforcement learning model so that the reinforcement learning model outputs a target scheduling plan that matches the multiple feature vectors. The target scheduling plan is used to reflect the range of target products to be produced and the corresponding target production machines required to process the order information at the lowest cost. Based on the target scheduling plan, production is scheduled and manufactured according to the order information; The target scheduling plan includes one of the following: The following options are available: Select the product range with the minimum delay time and the corresponding production machine scheduling plan; Select the product range with the minimum ratio of remaining delivery time to remaining production time and the corresponding production machine scheduling plan; Select the product range with the maximum delay time and the corresponding production machine scheduling plan; Randomly select the product range and the corresponding production machine scheduling plan; Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

2. The method as described in claim 1, characterized in that, Based on the manufacturer's basic information and the order information, multiple state feature vectors are obtained, including: The manufacturer's basic information and the order information are input into the intelligent manufacturing scheduling system to obtain multiple state feature vectors that match the manufacturer's basic information and the order information.

3. The method as described in claim 1, characterized in that, After scheduling and manufacturing the order information based on the target scheduling plan, the process further includes: Based on the target product range and corresponding target production machine reflected in the target scheduling plan, determine the production time required for the target production machine to produce the target product quantity. Based on the start processing time of the target production machine and the production duration, calculate the overtime cost corresponding to the target scheduling plan; and obtain the product delay cost included in the order information. Based on the relationship between the overtime cost and the product delay cost, the reinforcement learning model is optimized.

4. The method as described in claim 3, characterized in that, The optimization of the reinforcement learning model based on the relationship between the overtime cost and the product delay cost includes: If the overtime cost is detected to be greater than the product delay cost, the incentive function corresponding to the target scheduling plan will be marked as negative. The reinforcement learning model is optimized using the activation function marked with a negative value.

5. The method as described in claim 1, characterized in that, After scheduling and manufacturing the order information based on the target scheduling plan, the process further includes: Select a first number of samples from the data sample pool; Using the mean squared error loss function and the sample set, the reinforcement learning model is trained by backpropagation, and the parameters of the reinforcement learning model are updated until it is determined that the reinforcement learning model has been trained.

6. The method as described in claim 5, characterized in that, The backpropagation training of the reinforcement learning model includes: The average utilization rate vector, which reflects the resource utilization rate of the production machine, and the estimated delay rate vector and the actual delay rate vector, which reflect the delay progress of the order information, are used as reward functions. The reinforcement learning model is trained using the reward function via backpropagation.

7. A smart manufacturing scheduling device, characterized in that, in: The acquisition module is configured to acquire basic manufacturer information and order information within a target time period. The basic manufacturer information includes employee production information reflecting employee overtime costs, and the order information includes order quantity, product delivery date, and product delay costs. The basic manufacturer information includes: production machine processing time, employee attendance information, number of production machines on different production lines, production costs and employee overtime costs on different production lines, and product cost information. The conversion module is configured to obtain multiple state feature vectors based on the manufacturer's basic information and the order information, wherein each state feature vector reflects the production status of the production machine or the production status of the product; wherein the state feature vector includes: an average utilization rate vector reflecting the resource utilization rate of the production machine, a utilization rate variance vector reflecting the load balancing status of each production machine, a total process completion rate vector and a vector of different product completion rates reflecting the processing progress of the order information, and an estimated delay rate vector and an actual delay rate vector reflecting the delayed progress of the order information. The output module is configured to take the plurality of feature vectors as input to a reinforcement learning model, so that the reinforcement learning model outputs a target scheduling plan that matches the plurality of feature vectors. The target scheduling plan is used to reflect the range of target products to be produced and the corresponding target production machines required to process the order information at the lowest cost. The processing module is configured to schedule production based on the target scheduling plan and the order information. The target scheduling plan includes one of the following: The following options are available: Select the product range with the minimum delay time and the corresponding production machine scheduling plan; Select the product range with the minimum ratio of remaining delivery time to remaining production time and the corresponding production machine scheduling plan; Select the product range with the maximum delay time and the corresponding production machine scheduling plan; Randomly select the product range and the corresponding production machine scheduling plan; Select the product range with the maximum estimated delay time and the corresponding production machine scheduling plan.

8. An electronic device, characterized in that, include: Memory, used to store executable instructions; as well as, A processor, configured to execute the executable instructions with the memory to perform the operation of any of the intelligent manufacturing scheduling methods described in claims 1-6.

9. A computer-readable storage medium for storing computer-readable instructions, characterized in that, When the instruction is executed, it performs the operation of any of the intelligent manufacturing scheduling methods described in claims 1-6.

Images