A cloud-edge production scheduling and regulation method based on GRN-RL in a high-frequency disturbance environment

Abstract

The present application relates to the cloud manufacturing production regulation technical field, especially to a kind of cloud edge production scheduling regulation method based on GRN-RL under high-frequency disturbance environment, monitoring and regulation are carried out through edge, and the value optimization of edge model is carried out using cloud.The specific configuration optimization in production process and process monitoring are carried out by local production area edge task gene regulation network model by such processing.And when the preset update condition is triggered, the optimization of relevant parameters is carried out by cloud.Using the method, even in high-frequency disturbance environment, the optimization configuration of equipment resource can be realized in production execution process in time and effectively, and high-quality dynamic scheduling regulation service is provided for cloud manufacturing task in workshop site.And the effect of scheduling regulation is also improved, which is conducive to the landing application of cloud manufacturing mode in the vast number of discrete manufacturing enterprises workshop bottom.

Description

TECHNICAL FIELD

[0001] The present application relates to the cloud manufacturing production regulation technical field, and particularly relates to a cloud-edge production scheduling regulation method based on GRN-RL in a high-frequency disturbance environment. BACKGROUND

[0002] Cloud manufacturing is a new concept developed on the basis of the concept of "manufacturing as a service" and the idea of cloud computing. It uses the idea of cloud computing to achieve high sharing of manufacturing resources and reduce waste of manufacturing resources. In the field, ensuring the reasonable scheduling of cloud manufacturing tasks is the key to the smooth execution of cloud manufacturing services.

[0003] However, the cloud manufacturing platform is far away from the field, and the resources of different fields belong to different owners. These resources are geographically dispersed, and if the cloud manufacturing platform is directly used to schedule and manage the field tasks, the time efficiency will be poor and the communication energy consumption will be high. At the same time, production disturbances occur frequently in the field, and the production system attributes are in a state of change at any time. The cloud manufacturing task scheduling scheme needs to be adjusted several times in combination with the changes in the production environment to make the production execution performance optimal. In summary, in a high-frequency disturbance environment, the existing cloud manufacturing technology cannot realize real-time sensing of the underlying production process information, and cannot effectively optimize the allocation of equipment resources in the production execution process.

[0004] Therefore, in a high-frequency disturbance environment, how to effectively optimize the allocation of equipment resources in the production execution process and provide high-quality dynamic scheduling and regulation services for cloud manufacturing tasks in the field has become a problem to be solved. SUMMARY

[0005] In view of the above problems of the prior art, the present application provides a cloud-edge production scheduling regulation method based on GRN-RL in a high-frequency disturbance environment, which can effectively optimize the allocation of equipment resources in the production execution process and provide high-quality dynamic scheduling and regulation services for cloud manufacturing tasks in the field.

[0006] In order to solve the above technical problems, the present application adopts the following technical scheme:

[0007] A cloud-edge production scheduling regulation method based on GRN-RL in a high-frequency disturbance environment, comprising the following steps:

[0008] S1, a task gene regulation network model for monitoring and regulating production is constructed at the edge of the production area; the task gene regulation network model includes a trigger factor and an operation mechanism, the trigger factor is used for monitoring the production process, and the operation mechanism is used for sorting the sequence of each task and selecting the processing equipment for each task;

[0009] S2, a regulation parameter optimization model is constructed on a cloud platform, for optimizing regulation parameters of a task gene regulation network model;

[0010] S3, the parameter optimization model is trained according to historical data, to obtain initial regulation parameters of the task gene regulation network model;

[0011] S4, based on the initial regulation parameters, the production tasks in the production area are sorted through the operation mechanism of the task gene regulation network model, to obtain an initial processing sequence, and production processing is performed according to the initial processing sequence;

[0012] S5, in a rolling window manner, processing information in a rolling window of a preset size is collected, and real-time trigger factors are obtained by processing and analyzing the processing information through the task gene regulation network model, and it is judged whether the values of the trigger factors meet the regulation conditions; if yes, go to S6; if there is an executed task reaching the bottom of the rolling window, the rolling window will slide down and go to S6; if the current rolling window includes all unprocessed tasks, and all tasks in the current rolling window have been completed, go to S8;

[0013] S6, real-time production information is sent to the cloud platform, and after obtaining real-time regulation optimization parameters through the regulation parameter optimization model of the cloud platform, the regulation optimization parameters are sent to the task gene regulation network model; the production information includes device processing parameters and task processing information;

[0014] S7, combined with the received regulation optimization parameters, the unprocessed tasks in the rolling window are re-sequenced through the operation mechanism of the task gene regulation network model, and production processing is performed according to the re-sequenced processing sequence; and then return to S5;

[0015] S8, end.

[0016] Preferably, the regulation equation of the trigger factor of the task gene regulation network model is:

[0017]

[0018] In the formula, parameter δ r represents a task r execution process monitoring operator, for reflecting the execution progress of task r; represents a device i processing task r execution quality prediction operator, for predicting the processing quality fluctuation of device i; dev r represents the deviation degree of the actual processing time of task r from the static expected value; represents the quality early warning prediction value of device i processing task r; κ is a direct trigger operator, for triggering the regulation condition after being assigned a preset maximum value when an acute disturbance event occurs, the acute disturbance event including urgent insertion and order urgency.

[0019] Preferably, the deviation dev is calculated as follows: r

[0020]

[0021] wherein T r represents the static processing time of task r, D r represents the production time limit of task r, represents the expected processing time of the jth process of task r, represents the actual processing time of the jth process of task r, and J represents the current process number.

[0022] Preferably, the early warning prediction value is calculated as follows:

[0023]

[0024] wherein represents the maximum expected quality loss of device i during the execution of task r, and W represents the quality loss warning value.

[0025] Preferably, in S5, the triggering condition is:

[0026]

[0027] wherein R represents the number of task types, and S and A are triggering thresholds, the values of S and A being obtained by a parameter optimization model of a cloud platform, and the initial control parameters in S3 including the values of S and A.

[0028] Preferably, the operation mechanism of the task gene regulation network model includes task sequence expression control and resource selection expression control; the task sequence expression control is used for sorting the sequence of each task, and the resource selection expression control is used for selecting a processing device for each task.

[0029] Preferably, the content of the task sequence expression control includes:

[0030] calculating the average processing time of the task in each round

[0031]

[0032] wherein tur represents the current round, k tur·n represents the process number to which task n progresses in the turth round;

[0033] is a decision variable, being 1 if the k tur n th process of task n can be processed on device m, and 0 otherwise; ​

[0034] indicates the kth process of task n tur n the processing time of the kth process of task n on equipment m

[0035] the completion time of task n before the current round

[0036]

[0037] wherein the first term is the processing time of all processes of task n before the current round; y nk m is a decision variable, 1 if the kth process of task n is processed on equipment m, otherwise 0; t nkm indicates the processing time of the kth process of task n on equipment m; the second term indicates the total adjustment time required for equipment switching to process tasks, and the second term of the equation is 0 if tur=1;

[0038] and calculate the difference between the delivery period and the total time consumed after the processing of the current round for indicating the urgency of the task:

[0039]

[0040] wherein T n indicates the delivery period of task n

[0041] are normalized to obtain corresponding

[0042] Calculate the task order gene regulation equation again:

[0043]

[0044] wherein λ1, λ2, and λ3 are order expression weights, the initial values and subsequent optimized values of λ1, λ2, and λ3 are obtained through a parameter optimization model of a cloud platform, the initial regulation parameters in S3 include the initial values of λ1, λ2, and λ3, and the regulation optimization parameters in S6 include the latest optimized values of λ1, λ2, and λ3;

[0045] After each round, the order gene regulation expression value of the next process task of all tasks to be processed in the current round is calculated, and the task sequence set obtained by arranging in ascending order is taken as the task scheduling regulation order of the current round:

[0046]

[0047] Preferably, the content of the resource selection expression control includes: ​​​

[0048] Calculate the processing time of task n in the turth round when selecting device m. Processing energy consumption and equipment load

[0049] After that, and Normalization is performed to obtain the corresponding and

[0050] Then, the gene resource selection expression regulation equation for the task is calculated to obtain the corresponding gene resource selection expression value.

[0051]

[0052] In the formula, ω1, ω2, and ω3 are the weights of resource selection indicators, which are calculated using the subjective analytic hierarchy process.

[0053] After calculating the gene resource expression values ​​of all available devices in task n at round tur, the device with the smallest gene expression value is selected as the final device.

[0054]

[0055] In the formula, Refers to task n The set of available equipment for each process step;

[0056] Once all tasks in this round have completed gene expression, a task gene network is generated based on the order of gene expression and resource selection information. The start and end times of the current task on the equipment are calculated based on the processing completion time of the previous task, the task adjustment time, and the average repair time of faulty equipment. Production tasks are then sequentially placed into the rolling window.

[0057] Preferably, the process of obtaining the trigger thresholds S and A through the parameter optimization model includes:

[0058] a. Obtain task and equipment processing information, and train the equipment quality prediction curve;

[0059] b. Randomly set a set of control operation parameters, initialize threshold S = 0, A = 0, and set the increment step of thresholds S and A. and the upper limit of the threshold S max A max ;

[0060] c. Start by performing n production simulations with a fixed threshold S, and record the average number of overdue tasks.

[0061] d、 Start the production simulation n times to determine the threshold value A, and record the average task quality loss prediction value;

[0062] e, if it is the first record, go to step f, g; if S max go to step f, otherwise go to step h; if A max go to step g, otherwise go to step k;

[0063] f, repeat step c;

[0064] g, repeat step d;

[0065] h, aggregate data, performance ranking, and preferred threshold value S;

[0066] f, aggregate data, performance ranking, and preferred threshold value A.

[0067] Preferably, the parameter optimization model is trained by reinforcement learning to obtain the order expression weights λ1, λ2 and λ3 of the task order gene;

[0068] The behavior in the reinforcement learning includes the following types:

[0069]

[0070] Wherein, the rule T is: in order to avoid the processing time of subsequent tasks being too long, the priority of the task order with shorter processing time is higher;

[0071] The rule K is: in order to avoid the idle of the equipment, the priority of the task order which is processed earlier is higher;

[0072] The rule S is: the closer the task is to the delivery period, the higher the order priority is.

[0073] Compared with the prior art, the present application has the following beneficial effects:

[0074] 1. The prior art is a "cloud + terminal" operation mode, that is, unified management and control is performed through the cloud, which is difficult to realize real-time perception of bottom layer production process information, and cannot realize timely and effective optimization of device resources in the production execution process.

[0075] The present application creatively proposes a new type of cloud edge scheduling and control method, which monitors and controls through the edge, and at the same time, uses the cloud to optimize the value of the edge model. Through such processing, the specific configuration optimization and process monitoring in the production process are implemented by the local production area edge task gene control network model. And when the preset update condition is triggered, the cloud is used to optimize the related parameters.

[0076] In this way, first, the computing power of the cloud can be fully utilized, and when an exception occurs, such as a disturbance in the production process, the optimization update of the relevant parameters can be quickly completed, ensuring the effectiveness of the task control in the specific production; second, since the cloud only optimizes the relevant parameters, and the detection and control in the production process are executed by the local task gene control network model at the edge of the production area, the timeliness of the specific control can be ensured; third, using the method, only when the relevant parameters need to be optimized, the local and cloud communication will be performed in the production process, and since the cloud only optimizes the relevant parameters, the data volume of a single communication is also very small, and such a way can also effectively reduce the communication energy consumption.

[0077] In summary, using the method, even in a high-frequency disturbance environment, the optimized allocation of equipment resources in the production execution process can be achieved in a timely and effective manner, providing high-quality dynamic scheduling and control services for the cloud manufacturing tasks in the production area. And it also improves the scheduling and control effect, which is conducive to the application of the cloud manufacturing mode in the bottom layer of the discrete manufacturing enterprise workshop.

[0078] 2、In the specific production process, the method collects the processing information in the rolling window of a predetermined size, and determines whether the relevant parameters need to be updated, and when reordering is needed (the trigger factor meets the condition, or the rolling window slides down), only the updated parameters are updated, and the tasks in the rolling window that have not been processed are reordered. Such an updating mechanism can ensure the speed and flexibility of the entire reordering process.

[0079] 3、In a high-frequency disturbance environment, the production quality of cloud manufacturing tasks has uncertainty, and the application adopts a hybrid method of real-time monitoring and prediction based on historical data to design the control trigger factor. Thus, the timeliness and effectiveness of the control are taken into account.

[0080] 4、The trigger condition in the application can ensure that when reordering is needed (the trigger factor meets the condition, or the rolling window slides down), it can be identified in a timely and accurate manner.

[0081] 5、The application combines the gene expression of the task when reordering the task, which takes into account the specific circumstances of the task from multiple aspects, thereby ensuring the reliability of subsequent adjustment.

[0082] 6、In the parameter optimization of the application, the behavior type of reinforcement learning considers the separate compliance, mutual compliance, and simultaneous compliance of the original rules T, K, and S, which can ensure the effectiveness of the obtained order expression weight. BRIEF DESCRIPTION OF DRAWINGS

[0083] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings, in which:

[0084] Figure 1 A schematic diagram of the regulation framework in the embodiment;

[0085] Figure 2 A flowchart in the embodiment;

[0086] Figure 3 A schematic diagram of the process quality loss cycle prediction process of the equipment in the embodiment;

[0087] Figure 4 A schematic diagram of the production scheduling regulation mechanism based on GRN in the embodiment. DETAILED DESCRIPTION

[0088] The application will be further described in detail below through specific embodiments:

[0089] Embodiment:

[0090] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings, in which:

[0091] As shown in Figure 1 , the production area performs production according to the initial scheduling scheme, and the production process information is collected to the edge of the production area for monitoring. When serious production performance index deviation or quality warning occurs, the production execution process quality monitoring index will exceed the threshold value. At this time, the cloud manufacturing platform will calculate the regulation parameters in combination with the information and task data of the equipment in the production area, and the calculated model parameters will be issued to the edge of the production area for regulation of the original scheduling scheme. Figure 1 The dashed box in the middle represents a rolling window.

[0092] As shown in Figure 2 , the embodiment discloses a cloud-edge production scheduling regulation method based on GRN-RL in a high-frequency disturbance environment, which comprises the following steps:

[0093] S1, a task gene regulation network model for monitoring and regulating production is constructed at the edge of the production area; the task gene regulation network model comprises a trigger factor and an operation mechanism. The trigger factor is used for monitoring the production process, and the operation mechanism is used for sorting the sequence of each task and selecting the processing equipment for each task.

[0094] S2, a regulation parameter optimization model is constructed on the cloud platform, which is used for optimizing the regulation parameters of the task gene regulation network model.

[0095] S3, the parameter optimization model is trained according to historical data to obtain the initial regulation parameters of the task gene regulation network model.

[0096] In particular implementation, the regulation equation of the trigger factor of the task gene regulation network model is as follows:

[0097]

[0098] In the formula, the parameter δ r represents the execution process monitoring operator of the task r, and is used to reflect the execution progress of the task r; represents the execution quality prediction operator of the device i processing the task r, and is used to predict the machining quality fluctuation of the device i; dev r represents the deviation degree of the actual machining duration of the task r from the static expected value; represents the quality early warning prediction value of the device i processing the task r; κ is a direct trigger operator, which is used to trigger the regulation condition after being assigned with a preset maximum value when an acute disturbance event occurs, and the acute disturbance event includes urgent insertion and order urgency.

[0099] The calculation formula of the deviation degree dev r is as follows:

[0100]

[0101] In the formula, T r represents the static machining time of the task r, D r represents the production time limit of the task r, represents the expected machining time of the jth process of the task r, is the actual machining time of the jth process of the task r, and J is the current execution process number. The deviation degree dev r thus designed can effectively reflect the influence of the deviation between the actual and expected performance indicators of the task r execution process on the task overdue.

[0102] The calculation formula of the early warning prediction value κ is as follows:

[0103]

[0104] In the formula, is the maximum expected quality loss of the device i during the execution of the task r, and W is the quality loss warning value.

[0105] The machining quality loss of the device is evaluated every time a production cycle is experienced, until the sample quantity is sufficient. The variable weight fuzzy comprehensive evaluation method is used to evaluate the influence of the production disturbance on the product quality, and the neural network is combined with the historical data to predict the machining quality fluctuation of the device. The degree to which the fluctuation curve exceeds the warning part.

[0106] The device execution process quality loss cycle prediction process is as follows: Figure 3 ​As shown, firstly, a data sample library is established, the evaluation index system and evaluation level are determined, the weight of the first index and the second index is determined, and the first dimension index evaluation matrix and the second dimension index evaluation matrix are established, the fuzzy comprehensive evaluation of the production process quality loss is carried out, and the evaluation result is stored in the data sample library. When the data sample is sufficient, the neural network is trained to obtain the execution process quality loss prediction curve. Then, according to the prediction result and the tolerance threshold, it is determined whether the production scheduling is to be carried out.

[0107] In this embodiment, the evaluation system is composed of common equipment disturbance indexes in the production area, as shown in Table 1.

[0108] Table 1 Production area disturbance index system

[0109]

[0110] The operation mechanism of the task gene regulation network model includes task sequence expression control and resource selection expression control; the task sequence expression control is used for sequencing the order of each task, and the resource selection expression control is used for selecting the processing equipment for each task.

[0111] The content of the task sequence expression control includes:

[0112] Calculate the average processing time of the task in each round

[0113]

[0114] In the formula, tur represents the current round, k tur·n represents the process number to which the task n progresses in the tur round;

[0115] is a decision variable, and is 1 if the k tur n th process of the task n can be processed on the equipment m, and is 0 otherwise;

[0116] represents the processing time of the k tur·n th process of the task n on the equipment m;

[0117] Calculate the completion time of the task before the current round

[0118]

[0119] In the formula, the first term is the processing time consumption of all processes of the task n before the current round; y nkm is a decision variable, and is 1 if the k nkmTkn,m,k represents the processing time of the kth process of task n on equipment m; the second term refers to the total adjustment time required for equipment switching to process the task, and if tur=1, the second term of the equation is 0;

[0120] and the total time difference between the delivery period and the total time consumed after the processing of the round is calculated for representing the urgency of the task:

[0121]

[0122] wherein T n represents the delivery period of task n;

[0123]

[0124] the task order gene regulation equation is calculated again:

[0125]

[0126] wherein λ1, λ2, and λ3 are order expression weights, the initial values and subsequent optimized values of λ1, λ2, and λ3 are obtained through a parameter optimization model of a cloud platform, and the initial regulation parameters in S3 include the initial values of λ1, λ2, and λ3.

[0127] After each round, the order gene regulation expression values of the next process tasks of all the tasks to be processed in the current round are calculated, and the task sequence set obtained by arranging in ascending order is taken as the task scheduling regulation order of the current round:

[0128]

[0129] The content of resource selection expression control includes:

[0130] the processing man-hours of task n on equipment m in the turth round are calculated processing energy consumption and equipment load

[0131] then, the normalization is performed on and to obtain the corresponding and

[0132] the task gene resource selection expression regulation equation is calculated again to obtain the corresponding gene resource selection expression value

[0133] ​​​​

[0134] In the formula, ω1, ω2, ω3 are resource selection index weights, which are calculated by subjective analytic hierarchy process;

[0135] After calculating the genetic resource selection expression value of task n in all selectable devices in the tur round, the device with the minimum gene is selected as the final selected device

[0136]

[0137] In the formula, The available device set of the process of task n in the tur round;

[0138] When the genetic expression of the task in the current round is completed, a task genetic network is generated according to the order information and resource selection information of the task genetic expression, and the start and end times of the processing of the current task on the device are calculated according to the previous task processing time, task adjustment time and average repair time of the fault device, and the production tasks are sequentially arranged in the rolling window.

[0139] S4, based on the initial control parameters, the production tasks in the production area are sorted through the operation mechanism of the task genetic regulation network model, and an initial processing sequence is obtained, and production processing is performed according to the initial processing sequence;

[0140] S5, through the rolling window, the processing information in the rolling window of a predetermined size is collected, and the real-time trigger factor is obtained by processing and analyzing the task genetic regulation network model, and it is judged whether the value of the trigger factor meets the control condition; if it meets, go to S6; if there is an executed task reaching the bottom of the rolling window, the rolling window will slide down and go to S6; if the current rolling window includes all unprocessed tasks, and all tasks in the current rolling window have been completed, go to S8.

[0141] The trigger condition is:

[0142]

[0143] In the formula, R represents the number of task types; S and A are trigger thresholds, and the values of S and A are obtained by a parameter optimization model of a cloud platform. The initial control parameters in S3 include the values of S and A.

[0144] In specific implementation, the process of obtaining the trigger thresholds S and A by the parameter optimization model includes:

[0145] a. Obtain task and device processing information, and train a device quality prediction curve;

[0146] b. Randomly set a group of control operation parameters, initialize the thresholds S=0 and A=0, and set the increase step of the thresholds S and A ​ and upper threshold S max , A max ;

[0147] c、 Start production simulation n times with threshold S, record the average number of overdue tasks;

[0148] d、 Start production simulation n times with threshold A, record the average task quality loss prediction value;

[0149] e, if it is the first record, go to step f, g; if S max go to step f, otherwise go to step h; if A max go to step g, otherwise go to step k;

[0150] f, repeat step c;

[0151] g, repeat step d;

[0152] h, aggregate data, sort performance, and select threshold S;

[0153] f, aggregate data, sort performance, and select threshold A.

[0154] S6, send real-time production information to the cloud platform, obtain real-time control optimization parameters through the control parameter optimization model of the cloud platform, and send the control optimization parameters to the task gene control network model; the production information includes equipment processing parameters and task processing information. The control optimization parameters include the latest optimization values of λ1, λ2 and λ3.

[0155] In specific implementation, the parameter optimization model obtains the order expression weights λ1, λ2 and λ3 of the task order gene through reinforcement learning training;

[0156] The behaviors in the reinforcement learning include the following types:

[0157]

[0158] Among them, the rule T is: in order to avoid the waiting time of subsequent tasks being too long, the priority of the task order with shorter processing time is higher;

[0159] The rule K is: in order to avoid equipment idling, the priority of the task order that is completed earlier is higher;

[0160] The rule S is: the closer the task is to the delivery period, the higher the order priority is.

[0161] Reinforcement learning follows a Markov decision process. The model consists of system decision time T, system state S, reward R, and behavioral policy A. The system makes decisions when scheduling is activated and when the scroll window slides down, until all tasks are scheduled. The system's decision time set is T = {t1, t2, ..., t...} n System status refers to the scheduling progress of each task. Let's assume... The task of type 1 has been scheduled to the j1st process. If there are a total of M types of production tasks, then the jth system state... Since each decision results in a system state transition, the number of system states is always one more than the number of decision points, S = {S1, S2, ..., S...} n ,S n+1}, state S n+1 This refers to the completion of scheduling all production tasks. The reward R refers to the incentive received by the agent after performing an action; R influences the convergence direction of reinforcement learning. The performance of the scheduling scheme is evaluated in terms of task relaxation Y and total energy consumption E. Then, state S... j The formula for calculating the return is as follows:

[0162]

[0163] The system is in state S j The difference between the processing time limit of task m and the current execution time of the task. The total energy consumption during the execution process is calculated as follows: if no tasks are overdue, then... All values ​​must be positive, and the reward must also be positive. Otherwise, if the task expires, the reward will be penalized with a value 10 times greater than the positive reward, and the round will end.

[0164] Behavioral strategy A refers to the system in state S j The set of actions that can be taken at this time The regulatory behavior refers to determining the initial rule weights λ1, λ2, and λ3 to generate specific scheduling rules. Each time an action is executed, the system undergoes a state transition and receives a reward R. i This continues until all production tasks are scheduled and a complete strategy trajectory is obtained.

[0165] π={S1,A1(S1),S2,R1,A2(S2),S3,R2…,S t ,A(S t ),S t+1 ,R t …,S n+1 ,R n};

[0166] State-value function V under policy π π(S) (s e S) and state-action value function Q π (s,a) (s e S, a e A(s)) is calculated by

[0167]

[0168]

[0169] where P(s,a,s') is the probability of transitioning from state s to state s' by taking action a, and g is the discount rate of return.

[0170] Take the four attitudes of emphasis, consideration, simultaneous consideration and abandonment, and use the method of equal weight to establish the order weight combination to generate the behavior set. The behavior set containing 10 scheduling rules distributed uniformly in the direction of the initial rule can be formed as shown in Table 5. In fact, more behavior rules can be expanded by more weight combinations.

[0171] In this embodiment, Q-learning is used to solve the weight parameter optimization problem of the task order gene. Q-learning is a value-based, model-free, off-policy and offline reinforcement learning algorithm. It uses Bellman equation iteration to update and solve, as shown in the following formula (20).

[0172]

[0173] where Q(S,A) is the value function, and a is the learning rate.

[0174] The complete algorithm flow of Q-learning is as follows:

[0175] Step 1: Initialize the learning rate a, the exploration rate e, the discount rate g, the iteration number and the Q value in the table. Set the total iteration number N and start learning and training.

[0176] Step 2: Initialize the state sequence.

[0177] Step 3: Randomly select the behavior with a size of e with a probability of exploration, and select the behavior with the maximum Q value with a probability of 1-e for decision making.

[0178] Step 4: Execute the action After the action is executed, the reward value R j is calculated according to the new environment of the system obtained by state transition. At this time, it is judged whether it is the final state of the system. If it is the final state S n+1 , then If it is not the final state, then

[0179] Step 5: Update the value of the Q table with the learning rate a.

[0180]

[0181] Step 6 judges whether the current state is the final state S n+1 , if yes, the production task scheduling control is completed, the current iteration is finished, the iteration number is increased by 1, and step 7 is performed, otherwise, step 3 is performed;

[0182] Step 7 judges whether the iteration number reaches N times, if yes, the iteration is finished, otherwise, the next iteration is started.

[0183] S7, in combination with the received control optimization parameters, the tasks in the rolling window which are not processed are reordered through the operation mechanism of the task gene control network model, and the production processing is performed according to the reordered processing sequence; and then S5 is returned.

[0184] S8, end.

[0185] In order to facilitate those skilled in the art to better understand the control mechanism in the present application, the production scheduling control mechanism based on GRN (gene regulatory network) is described as follows:

[0186] The production scheduling control mechanism based on GRN is described as shown in Figure 4 When the control link is triggered, the production task gene will express the task sequence and resource selection through the gene regulation equation in combination with the real-time data of the production area equipment and the average repair time (MTTR) of the faulty equipment, so as to determine the sequence of the tasks and the matching relationship between the tasks and the equipment. The production scheduling control will simulate the translation process of the ribosome on the RNA, and adopt the rolling window mechanism to reschedule the tasks to be processed according to the task gene expression results until the number of the production scheduling in the single rolling window reaches saturation. The saturation conditions of the rolling window are as follows:

[0187] ①The next step of all task types is arranged on a device waiting for processing.

[0188] ②There is no production task that can be inserted between the maximum processing time of each device and the maximum completion time of the scheduled scheme under the condition that the first piece is satisfied.

[0189] When the rolling window is saturated, the equipment will be put into production in stages. When the last task in the rolling window is about to be completed, the rolling window will be lowered, the task gene network will be updated, the sequence of production task genes and the sequence of resource selection in the network will be re-expressed in combination with the latest data of the equipment, and according to the expression result, the production will be arranged again in the rolling window until all the production tasks are completely arranged. If the task in the rolling window is activated again during the execution process, the rolling window will be reset at the task that activates the control start sequence, and the gene will be re-expressed in combination with the real-time information of the production area equipment.

[0190] The prior art is a "cloud + end" operation mode, that is, unified management and control is performed through the cloud, real-time sensing of underlying production process information is difficult, and timely and effective optimization of equipment resources during production execution cannot be achieved. The present application creatively proposes a new cloud-edge scheduling and control method, which monitors and controls through the edge, and uses the cloud to optimize the value of the edge model. Through such processing, the specific configuration optimization and process monitoring in the production process are implemented by the local production area edge task gene control network model. And when the preset update condition is triggered, the cloud optimizes the related parameters. In this way, first, the computing power of the cloud can be fully utilized, and when an exception occurs, such as a disturbance in the production process, the optimization and update of related parameters can be quickly completed, ensuring the effectiveness of task control during specific production; second, since the cloud is only responsible for the optimization of related parameters, and the detection and control in the production process are executed by the local production area edge task gene control network model, the timeliness of specific control can be ensured; third, using this method, only when related parameters need to be optimized during production, local and cloud communication will be performed, and since the cloud only optimizes related parameters, the data volume of a single communication is also very small. This way can also effectively and significantly reduce communication energy consumption.

[0191] In addition, in the specific production process, the method can collect the processing information in the rolling window of a predetermined size, and determine whether the related parameters need to be updated. When reordering is needed (the trigger factor meets the condition or the rolling window slides down), only the updated parameters are used to reorder the tasks that have not been processed in the rolling window. Such an updating mechanism can ensure the speed and flexibility of the entire reordering process. In addition, due to the uncertainty of the cloud manufacturing task production quality in a high-frequency disturbance environment, the application uses a hybrid method of real-time monitoring and prediction based on historical data to design the control trigger factor, thereby balancing the timeliness and effectiveness of the control. In addition, the trigger condition in the application can ensure that when reordering is needed (the trigger factor meets the condition or the rolling window slides down), it can be identified in a timely and accurate manner. Furthermore, the application reorders the tasks in combination with the gene expression of the tasks. Such a method takes into account various specific situations of the tasks, thereby ensuring the reliability of subsequent adjustments.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit the technical solutions. Those of ordinary skill in the art should understand that modifications or equivalent replacements of the technical solutions of the application without departing from the spirit and scope of the technical solutions should be covered in the scope of the claims of the application.

Claims

1. A GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment, characterized by, The method comprises the following steps: S1, constructing a task gene regulation network model for monitoring and regulating production at the edge of the production area; The task gene regulation network model comprises trigger factors and operation mechanisms, the trigger factors are used for monitoring the production process, and the operation mechanisms are used for sequencing the tasks and selecting processing equipment for the tasks; S2, constructing a regulation parameter optimization model on a cloud platform to optimize the regulation parameters of the task gene regulation network model; S3, training the parameter optimization model according to historical data to obtain initial regulation parameters of the task gene regulation network model; S4, sequencing the production tasks in the production area based on the initial regulation parameters and the operation mechanisms of the task gene regulation network model to obtain an initial processing sequence, and performing production processing according to the initial processing sequence; S5, collecting processing information in a preset size of a rolling window in a rolling window manner, processing and analyzing the processing information by the task gene regulation network model to obtain real-time trigger factors, and judging whether the values of the trigger factors meet regulation conditions; if yes, proceeding to S6; if there is an executed task reaching the bottom of the rolling window, the rolling window will slide down and proceed to S6; if all unprocessed tasks are included in the current rolling window and all tasks in the current rolling window have been completed, proceeding to S8; S6, sending real-time production information to the cloud platform, obtaining real-time regulation optimization parameters by the regulation parameter optimization model of the cloud platform, and sending the regulation optimization parameters to the task gene regulation network model; the production information comprises device processing parameters and task processing information; S7, resequencing the unprocessed tasks in the rolling window by the operation mechanisms of the task gene regulation network model in combination with the received regulation optimization parameters, performing production processing according to the resequenced processing sequence, and returning to S5; S8, ending.

2. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 1, characterized in that: The regulation equation of the trigger factors of the task gene regulation network model is: In the formula, the parameter δ r represents an execution process monitoring operator of task r, used to reflect the execution progress of task r; represents an execution quality prediction operator of task r of device i, used to predict the machining quality fluctuation of device i; dev r represents the deviation degree of the actual machining duration monitoring value of task r from the static expected value; represents the quality early warning prediction value of task r of device i; κ is a direct trigger operator, which is assigned a preset maximum value to trigger the regulation condition when an acute disturbance event occurs, the acute disturbance event comprising an urgent order and an urgent order.

3. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 2, characterized in that: deviation dev r The calculation formula is: In the formula, T r represents the static processing time of task r, D r represents the production time limit of task r, represents the expected time of the jth process of task r, is the actual time of the jth process of task r, and J is the current process number.

4. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 3, characterized in that: Early warning prediction value The calculation formula is: wherein W is the maximum quality loss expected for device i during execution of task r, and W is the quality loss alarm value.

5. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 4, characterized in that: In S5, the trigger condition is: In the formula, R represents the number of task types; S and A are trigger thresholds, the values of S and A being obtained by the parameter optimization model of the cloud platform, and the initial regulation parameters in S3 comprising the values of S and A.

6. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 5, characterized in that: The operation mechanisms of the task gene regulation network model comprise task sequence expression control and resource selection expression control; the task sequence expression control is used for sequencing the tasks, and the resource selection expression control is used for selecting processing equipment for the tasks.

7. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 6, characterized in that: The content of the task sequence expression control comprises: average processing man-hours of the computing task in each round In the formula, tur represents the current round, k tur·n represents the process number to which task n has progressed in the turth round. is 1 if the kth operation of task n can be processed on device m, otherwise 0. tur·n is 1 if the kth operation of task n can be processed on device m, otherwise 0. kth pass of task n on machine m tur·n machining time of pass k of task n on machine m Completion time of the computing task before the current round where the first term is the total processing time of task n for all processes before this round; y nkm is the decision variable, which is 1 if the kth process of task n is processed on machine m, otherwise 0; t nkm represents the processing time of the kth process of task n on machine m; the second term indicates the total adjustment time required for machine switching to process tasks, which is 0 if tur=1; and calculate the total time difference between the delivery date and the time consumed after the processing of the round for indicating the urgency of a task: In the formula, T n represents the delivery date of task n; ​​​​ The task sequence gene regulation equation is recalculated as follows: In the formula, λ1, λ2 and λ3 are sequence expression weights, the initial values and subsequent optimization values of λ1, λ2 and λ3 being obtained by the parameter optimization model of the cloud platform, the initial regulation parameters in S3 comprising the initial values of λ1, λ2 and λ3, and the regulation optimization parameters in S6 comprising the latest optimization values of λ1, λ2 and λ3. The order gene regulation expression value of the next process task of all the to-be-processed task types in the current round is calculated every round, and the obtained task sequence set is arranged in ascending order as the task scheduling regulation order of the current round:

8. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 7, characterized in that: The content of resource selection expression control includes: Computing task n at turth wheel selects device m's processing man-hours Processing energy consumption And device load After that, and Normalization is performed to obtain the corresponding and Recalculating the task gene resource selection expression regulation equation, the corresponding gene resource selection expression value is obtained In the formula, ω1, ω2, ω3 are resource selection index weights, which are calculated by subjective analytic hierarchy process; After calculating the gene resource selection expression value of all optional devices of task n in the tur round, the device with the minimum gene is the final selected device In the formula, Refers to the task n The available device set of the path process; After the gene expression of the tasks in the current round is completed, the task gene network is generated according to the order information of the task gene expression and the resource selection information, and the start and end times of the current task on the equipment are calculated according to the previous task processing completion time, task adjustment time and average repair time of the fault equipment, and the production tasks are sequentially arranged in the rolling window.

9. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 8, characterized in that: The process of obtaining the trigger threshold S and A through the parameter optimization model includes: a. Obtain task and equipment processing information, and train the equipment quality prediction curve; b. Randomly set a group of control operation parameters, initialize threshold value S = 0, A = 0, set the increase step of threshold value S, A and threshold upper limit S max , A max ; c、 Start production simulation n times to determine threshold S, record the average number of overdue tasks; d、 Start to determine the threshold value A production simulation n times, record the average task quality loss prediction value; e. If this is the first record go to step f, g; if S < S max then go to step f, else go to step h; if A < A max then go to step g, else go to step k; f. Repeat step c; g. Repeat step d; h. Summarize the data, sort the performance, and optimize the threshold S; f. Summarize the data, sort the performance, and optimize the threshold A.

10. The GRN-RL-based cloud-edge production scheduling and regulation method in a high-frequency disturbance environment according to claim 9, characterized in that: The parameter optimization model obtains the order expression weights λ1, λ2, λ3 of the task order gene through reinforcement learning training; The behaviors in the reinforcement learning include the following types: Among them, the rule T is: in order to avoid the long waiting time of the subsequent tasks, the priority of the task order with shorter processing time is higher; The rule K is: in order to avoid the idle of the equipment, the priority of the task order which is completed earlier is higher; The rule S is: the closer the task is to the delivery period, the higher the order priority is.

Images