A multi-station assembly plant intelligent scheduling method fusing proactive evaluation

Abstract

The application discloses a kind of multi-station assembly workshop intelligent scheduling methods of fusing proactive evaluation, material processing data, warehouse data, distribution system information and assembly progress etc. in combination with production line management and control system, the material of workshop is complete and the proactive evaluation of assembly progress is carried out;Based on discrete event simulation, simulation workshop simulates the running condition of real workshop, initializes simulation environment using proactive evaluation result and workshop related process data etc., designs multi-agent negotiation model to generate scheduling scheme and resource demand list;For abnormal events occurring in production process, a response plan combining artificial and agent simulation is developed.The application can to some extent avoid the problem of frequent changes in plan caused by material shortage and other disturbance events, and can quickly generate the globally optimal workshop scheduling result.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of workshop planning and scheduling, and particularly relates to an intelligent production scheduling method for multi-station assembly workshops by fusing proactive evaluation. BACKGROUND

[0002] Workshop production planning has become an important decision-making task to support and improve the production efficiency of manufacturing enterprises, logistics and service providers. The workshop scheduling problem can be described as a process of allocating limited resources to various production tasks according to certain constraints based on the master plan. The production tasks of a multi-station assembly workshop are divided according to stations, and there are complex assembly sequence relationships within and between stations. As the amount of tasks increases, the planning and scheduling of the workshop becomes more complex. In actual production, relying on manual experience can only give a rough plan node, and cannot give a detailed production plan. In addition, manual experience in formulating plans not only has difficulty in comprehensively considering factors such as personnel characteristics, personnel load balancing, personnel and automatic equipment cooperation and other factors in an uncertain production environment, but also has low scheduling efficiency, and the scheduling results lack scientificity and rationality, which seriously restricts the planning and management level of the workshop. Heuristic algorithms take a long time to solve when facing large-scale problems, and it is difficult to converge to the optimal solution. Deep reinforcement learning algorithms are mostly applied to flexible workshops or flow shop scenarios, and are less applied to multi-station large-scale assembly workshops.

[0003] There are many materials in the assembly workshop and the sources are complex, and the delay in the supply of key materials will cause the production line to stop or stop production. The traditional planning method plans in an ideal state without considering the material completeness and possible progress lag, and it is difficult to ensure the executability of the plan. SUMMARY

[0004] The application provides an intelligent production scheduling method for multi-station assembly workshops by fusing proactive evaluation, constructs a workshop simulation model based on discrete events, avoids certain risk events through proactive evaluation of materials and progress, designs a multi-agent decision-making model with offline training and online matching and a composite reward mechanism, and realizes intelligent decision-making for workshop scheduling.

[0005] Technical scheme: The intelligent production scheduling method for multi-station assembly workshops by fusing proactive evaluation, comprises the following steps:

[0006] (1) Construct a simulation workshop that reflects the physical workshop operating state and operating logic, decompose the workshop scheduling process, construct agents and decision actions based on assembly instructions, and build a Markov process model;

[0007] (2) Using the material matching time evaluation model to analyze and evaluate the matching of materials required for assembly in the workshop, using the assembly progress evaluation model to evaluate the future progress of the workshop; initializing the simulation workshop with the evaluation results, workshop process information, master plan, personnel information, and equipment information to form a complete agent learning environment;

[0008] (3) The interaction between the agent and the simulation environment is through the Socket interface and TCP communication protocol. The agent obtains the observation state, state transition, and immediate reward from the simulation environment and feeds back the action taken in the corresponding state to the environment. The interaction trajectory is composed of sample data in the form of five-tuples and stored in the experience replay pool using a binary tree structure;

[0009] (4) When the number of samples in the experience replay pool exceeds a certain number, the agent samples from its respective experience replay pool. Each time, a batch of data is sampled according to the sampling probability of the sample for training, and the future cumulative return of each sample data is predicted through the multi-step prediction method;

[0010] (5) Update the value function network of the agent using the sample data and cumulative return prediction value in step (4);

[0011] (6) Repeat steps (3)-(5) until the specified round is reached, and each agent converges to the corresponding optimal value function. Fix the network parameters of the agent and deploy it on the workshop scheduling solution.

[0012] Further, the material matching time evaluation model of step (2) evaluates the assembly instruction list containing essential parts. Each assembly instruction has a corresponding material list. The assembly instruction can only be executed when the material list reaches a certain matching rate and does not lack essential parts. Determine the matching rate of each assembly instruction through warehouse information. If the matching rate meets the standard, determine whether the essential parts are in stock. If they are in stock, set the status of the instruction to executable. Otherwise, evaluate the matching time of the essential parts. After the matching time, the status of the instruction becomes executable.

[0013] Further, the material matching time evaluation model of step (2) takes the state of the processing workshop as input, including material list information, buffer state, machine state, and work-in-process state. The GRU network layer is used to extract the time sequence features, and the specific expression is as follows:

[0014] r t =σ(W r ·[h t-1 ,x t ])

[0015] z t =v(W Z xt +U z h t-1 )

[0016]

[0017] wherein, r t represents an update gate, z t represents a reset gate, represents a hidden state of a node, h t represents an output of the node; W r , W Z , W h , U z respectively represent corresponding parameter matrices, x t is an input of the layer, h t-1 is a hidden state; after extracting the timing feature, the full connection network layer outputs the kit time of the material.

[0018] Further, the assembly progress evaluation model in step (2) relies on the simulated workshop built, synchronizes the current assembly progress of the real workshop, simulates the workshop production in a future period of time, judges whether all assembly instructions can be completed before the delivery node; if not, corresponding adjustments are made; the specific adjustment methods include increasing the working hours of assembly personnel per day, increasing the number of assembly personnel in the production line, changing the production line process, synchronizing the adjusted workshop related data to the simulation environment for verification until the delivery node requirement is met.

[0019] Further, the agent in step (3) includes an importance evaluation agent, a matching personnel agent, and a decision sequence agent; the importance evaluation agent obtains state information related to the instruction, and gives priority feedback of the assembly instruction to be decided to the simulation environment; the matching personnel agent obtains the state of the assembly instruction to be matched and the related information and state of all assembly personnel, gives a decision priority rule, and matches the assembly personnel according to the experience rule of the agent decision, and feeds back to the workshop simulation environment; the decision sequence agent obtains the production state of the workshop, including the progress of each station, the state of the buffer area and the storage area, and the instruction group information to be decided, and selects the next assembly instruction to be executed according to the experience rule of the agent decision; wherein, the input state AgentS1 of the importance evaluation agent is:

[0020] AgentS1={SA}

[0021]

[0022] wherein, SA represents features related to the assembly instruction, represents the number of the i-th assembly instruction in the k-th assembly stage of the station S. represents the man-hour of the assembly instruction, represents the total man-hour of all instructions in parallel with the instruction, represents the current available station of the station where the assembly instruction is located, represents the total man-hour of unexecuted instructions of the current production batch of the station where the assembly instruction is located, represents the criticality of the assembly instruction, represents the parallel number of the assembly instruction; the output is the priority of the assembly instruction, i.e. a number from 0 to 9, the larger the number, the higher the importance and priority of the assembly instruction;

[0023] The input state of the matching personnel agent AgentS2 is:

[0024] AgentS2 = {SP, SA}

[0025] SP = {P 1,num , P 1,at , P 1,e , P 2,num , P 2,at , P 2,e ,..., P p,num , P p,at , P p,e , S 1,pro , S 2,pro ,..., S s,pro}

[0026]

[0027] wherein SP represents the characteristics related to personnel, P j,num represents the number of remaining tasks of the jth assembly personnel, P j,at represents the total man-hour of the remaining tasks of the jth assembly personnel, P j,e represents the work rate of the jth assembly personnel; the assembly progress S S,pro of the assembly station S depends on the number of AOs to be completed in the task of the stage S S,aon and the number of AOs completed at the current time S S,fn ;

[0028] The input state of the decision sequence agent AgentS3 is:

[0029] AgentS3 = {SS, SPL}

[0030] SS = {S 1,num , S 1,pro , S 1,avn , S 1,bn , S 1,bat , S 1,strn , S 1,strnt ,

[0031] S 2,num ,S 2,pro ,S 2,avn ,S 2,bn ,S 2,bat ,S 2,strn ,S 2,strnt ,...,

[0032] S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt}

[0033] SPL={PL an ,PL wt ,PL ast ,PL 1,num ,PL 2,num ,...,PL s,num}

[0034] Wherein, SS represents the state related to the station position of the workshop, S S,num represents the station number of the Sth station position, S S,pro represents the assembly progress of the Sth station position, S S,avn represents the number of available workstations of the Sth station position, S S,bn represents the number of AOs in the buffer area of the Sth station position, S S,bat represents the average AO working hours in the buffer area of the Sth station position, S S,strn represents the number of AOs in the storage area of the Sth station position, S s,strnt represents the average AO working hours in the storage area of the Sth station position; SPL represents the state related to the list of assembly personnel, PL an represents the number of AOs in the assembly personnel task list, PL wt represents the average waiting time of AOs in the assembly personnel task list, PL ast represents the average assembly time of AOs in the assembly personnel task list, PL S,num represents the number of AOs belonging to the Sth station position in the assembly personnel task list.

[0035] Further, the reward obtained by the intelligent agent is a progressive reward, the sum of the assembly progress of each station is used to weight the reward, so that the reward obtained is greater when it is closer to the completion of the task, and the learning process of the intelligent agent is accelerated; the specific calculation method of the reward is:

[0036]

[0037] Wherein, R is the reward function, E is the average work rate of the worker, and V is the worker load variance.

[0038] Further, the experience pool capacity in step (4) is fixed, and sampling is performed from the experience pool by improving the priority experience replay method; the time of experience storage and the number of times of sampling are added when calculating the sampling probability, which improves the utilization efficiency of the sample and the learning speed of the agent, and the probability of sampling the sample is:

[0039]

[0040] Wherein, p i represents the probability of sampling, δ i represents the time difference error, and c is a constant greater than 0 for controlling the proportion of the number of times of sampling; t represents the current total number of times of sampling, N t (i) represents the number of times of sampling of the sample, and the initial value is 1; when the sample is newly stored in the experience pool, the sampling probability thereof is set to a fixed larger constant pc, so as to ensure that the newly stored sample is preferentially sampled;

[0041] In order to avoid the priority replay from changing the sample distribution in an uncontrollable manner, an importance weight is used to correct the bias:

[0042]

[0043] Wherein, w i represents the importance sampling weight, N represents the number of samples, p i represents the probability of sampling, and β represents the degree of compensation for the bias introduced by non-uniform probability sampling; when β = 1, the compensation is complete.

[0044] Further, the step (5) is implemented as follows:

[0045] Initialize the evaluation network, randomly generate the weight w, initialize the target network, and the weight w' of the target network = w;

[0046] Initialize the environment state S t , feed back the environment state to the agent, generate the action of the agent by using the noise network layer combined with the greedy strategy, receive the reward r t and the new state S t+1 from the environment, and store the sample track (S t ,a t ,r t ,S t+1 ) in the experience pool.

[0047] Randomly extract a batch of sample tracks from the experience pool, and update by using the gradient descent method; the weight of the target network is updated every fixed step.

[0048] Further, the noise network layer is specifically calculated as follows:

[0049] y = wx + b

[0050] w = u w + sigma w ⊙ epsilon w

[0051] b = mu b + sigma b ⊙ epsilon b

[0052] Wherein, y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weight and bias of the noise layer respectively, epsilon w And epsilon b It is the Gaussian distribution noise of w and b, which is randomly generated and does not participate in the process of gradient update.

[0053] Further, the greedy strategy is naive exploration, which preferentially selects the behavior currently considered to be the highest value, and the specific calculation method is as follows:

[0054]

[0055] Wherein, a represents the decision action of the agent, Q(a) represents the value of action a, and epsilon represents the probability value of randomly selecting action.

[0056] Beneficial effects: compared with the prior art, the beneficial effects of the present application are:

[0057] 1. The present application is based on the characteristics of multi-station workshop, decomposes the workshop scheduling process and constructs the decision action of the agent based on the transfer of assembly instructions, realizes the Markov process model, and is more in line with the production logic of the actual workshop, and is suitable for solving large-scale multi-station scheduling problems;

[0058] 2. The present application integrates the proactive evaluation technology to schedule the workshop, realizes the information interaction between each production workshop in time and effectively, avoids the frequent changes of plan caused by material distribution delay and progress anomaly through the evaluation of material matching time and assembly progress, and ensures the rationality and executability of the plan;

[0059] 3. The present application constructs a composite reward mechanism combining production progress and worker production efficiency, improves the priority experience playback, considers the time of sample entering the experience pool and the number of samples being sampled during sampling, improves the utilization rate of samples, and speeds up the learning speed of the agent. DETAILED DESCRIPTION

[0060] Figure 1 is the flowchart of the present application;

[0061] Figure 2 is a production rule and operation logic diagram of a multi-station assembly workshop of the present application;

[0062] Figure 3 is a diagram of input, output and state transition of each agent of the present application;

[0063] Figure 4 is a diagram of the interaction relationship between each agent and the simulation environment of the present application;

[0064] Figure 5 is a diagram of part of the initialization data of the present application;

[0065] Figure 6 is a diagram of part of the personnel plan solved by the present application;

[0066] Figure 7 is a diagram of part of the station plan solved by the present application. DETAILED DESCRIPTION

[0067] The present application will be further described in detail below with reference to the accompanying drawings.

[0068] The present application provides a multi-station assembly workshop intelligent scheduling method integrating proactive evaluation, and the specific implementation method is as shown in Figure 1 , which comprises the following steps:

[0069] Step 1: Construct a simulation workshop that can truly reflect the operation rule and operation logic of the physical workshop, decompose the workshop scheduling process, construct agents and corresponding actions based on the transfer of assembly instructions, and realize the Markov process model.

[0070] The operation logic and operation rule of the workshop are for the multi-station assembly workshop, and the assembly instructions guide the actual assembly process. As shown in Figure 2 , the main operation rule thereof includes that the assembly sequence between each station needs to be satisfied, when a batch of manufacturing tasks is executed, an activity can only start after a group of previous tasks are completed, that is, only when the assembly of station 26A is completed, the assembly of station 25A can be started. The execution of assembly instructions in the station has strict serial and parallel constraints, the instructions of the same T level are parallel instructions, and the instructions between different T levels are in serial relationship according to the size of the T level.

[0071] The simulation workshop includes personnel, equipment, assembly instructions, tooling, component groups and their corresponding attributes, uses discrete event simulation, acquires the state data of the workshop through data acquisition devices in the physical workshop and manual input, and updates the current operation state of the workshop to the simulation model through a specific communication protocol.

[0072] Step 2: The material matching time evaluation model is used to analyze and evaluate the material matching of the assembly in the workshop. The assembly progress evaluation model is used to evaluate the workshop and future progress. The simulation workshop is initialized with the evaluation results, workshop process information, master plan, personnel information, equipment information, etc. to form a complete agent learning environment. As shown in FIG. 8, the process information and material matching information imported into the simulation model are shown. Figure 3

[0073] The material matching time evaluation model evaluates the assembly instruction list containing necessary parts. Each assembly instruction has a corresponding material list. The instruction can be executed only when the material list reaches a certain matching rate and does not lack necessary parts. The material state (the required material is currently in the warehouse, processing, distribution, has been distributed, etc.) is derived from the warehouse, machining workshop and distribution system. The warehouse information is used to judge the matching rate of each assembly instruction. If the matching rate meets the standard, it is judged whether the necessary parts are in stock. If the necessary parts are in stock, the state of the instruction is set to executable. Otherwise, the matching time of the necessary parts needs to be evaluated. After the matching time, the state of the instruction is executable.

[0074] The material matching time evaluation model takes the state of the machining workshop as input, including material list information, buffer zone state, machine tool state, work-in-process state, extracts time sequence features through GRU network layer, and its specific expression is as follows:

[0075] r t =σ(W r ·[h t-1 ,x t ])

[0076] z t =σ(W Z x t +U z h t-1 )

[0077]

[0078] wherein, r t represents an update gate, z t represents a reset gate, represents the hidden state of a node, h t represents the output of the node. W r , W Z , W h , U z respectively represent the corresponding parameter matrix, x t is the input of the layer, and h t-1 is the hidden state. After extracting the time sequence features, the material matching time is output through the fully connected network layer.

[0079] ​The assembly progress evaluation model relies on the built simulation workshop, synchronizes the current assembly progress of the real workshop, simulates the production situation of the workshop in a future period of time, and judges whether all assembly instructions can be completed before the delivery node under ideal conditions. If not, make appropriate adjustments, including increasing the working hours of assembly personnel per day, increasing the number of assembly personnel in the production line, changing the production line process, and synchronizing the changed data to the simulation environment for verification.

[0080] Step 3: The interaction between the agent and the simulation environment is through the Socket interface and the TCP communication protocol. The agent obtains the observation state, state transition and immediate reward from the simulation environment, and feeds back the action taken by the corresponding state of the environment. The interaction trajectory is composed of sample data in the form of five-tuple, and stored in the experience replay pool using a binary tree structure.

[0081] As shown in Figure 4 , the agent is composed of three agents, and the three agents are in a complete cooperative relationship. Agent 1 is an importance evaluation agent that obtains state information related to instructions and provides priority feedback for pending assembly instructions to the simulation environment. Agent 2 is a matching personnel agent that obtains the state of the assembly instructions to be matched and the relevant information and state of all assembly personnel, provides a decision-making priority rule, and matches assembly personnel according to the experience rule of agent decision-making, and feeds back to the workshop simulation environment. Agent 3 is a decision sequence agent that obtains the production state of the workshop, including the progress of each station, the state of the buffer area and the storage area, and the information of the instruction group to be decided, and selects the next assembly instruction to be executed according to the experience rule of agent decision-making. As shown in Figure 5 , the agent is located in the same environment and makes decisions at different time points. Different agents can interact with each other, and the agents are in a complete cooperative relationship. The state transition of the environment can occur on the same agent or between different agents. Among them, the input state of the importance evaluation agent AgentS1 is:

[0082] AgentS1={SA}

[0083]

[0084] Among them, SA represents the characteristics related to the assembly instruction, represents the number of the i-th assembly instruction in the k-th assembly stage of the station S, represents the assembly instruction working hours, represents the total working hours of all instructions in parallel with the instruction, represents the current available workstation of the station where the assembly instruction is located, represents the total working hours of unexecuted instructions of the current production batch of the station where the assembly instruction is located, Indicates the criticality of the assembly instruction, Indicates the parallel number of the assembly instruction. The output is the priority of the assembly instruction, i.e. a number from 0 to 9, the larger the number, the more important and the higher the priority of the assembly instruction.

[0085] The input state of the matching personnel agent AgentS2 is:

[0086] AgentS2 = {SP, SA}

[0087] SP = {P 1,num , P 1,at , P 1,e , P 2,num , P 2,at , P 2,e ,..., P p,num , P p,at , P p,e , S 1,pro , S 2,pro ,..., S s,pro}

[0088]

[0089] wherein SP represents the personnel-related features, P j,num represents the number of remaining tasks of the jth group of assembly personnel, P j,at represents the total man-hours of the remaining tasks of the jth group of assembly personnel, P j,e represents the work rate of the jth group of assembly personnel. The assembly progress S S,pro of the assembly station S depends on the number of AOs to be completed in the task S S,aon and the number of AOs completed at the current time S S,fn .

[0090] The input state of the decision sequence agent AgentS3 is:

[0091] AgentS3 = {SS, SPL}

[0092] SS = {S 1,num , S 1,pro , S 1,avn , S 1,bn , S 1,bat , S 1,strn , S 1,strnt ,

[0093] S 2,num , S 2,pro , S 2,avn , S 2,bn , S 2,bat , S 2,strn , S2,strnt ,...,

[0094] S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt}

[0095] SPL={PL an ,PL wt ,PL ast ,PL 1,num ,PL 2,num ,...,PL s,num}

[0096] Wherein, SS represents a state related to the station position of the workshop, S S,num represents the station number of the Sth station position, S S,pro represents the assembly progress of the Sth station position, S S,avn represents the number of available workstations of the Sth station position, S S,bn represents the number of AOs in the buffer area of the Sth station position, S S,bat represents the average AO working hours in the buffer area of the Sth station position, S S,strn represents the number of AOs in the storage area of the Sth station position, S s,strnt represents the average AO working hours in the storage area of the Sth station position. SPL represents a state related to the assembly personnel list, PL an represents the number of AOs in the assembly personnel task list, PL wt represents the average waiting time of AOs in the assembly personnel task list, PL ast represents the average assembly time of AOs in the assembly personnel task list, PL S,num represents the number of AOs belonging to the Sth station position in the assembly personnel task list.

[0097] The actions of the matching personnel agent and the decision sequence agent are composed of experience rules.

[0098] The reward obtained by the agent is a compound reward, and the optimization goal of the agent is to minimize the maximum completion time and minimize the work load variance of the assembly personnel. The higher the worker work rate is, the shorter the corresponding completion time should be. The average work rate of the worker and the load variance of the worker are used as the reward of the agent. When the reward is obtained, a progressive reward is designed in the application, the sum of the assembly progress of each station is used to weight the reward, so that the reward obtained is greater when it is closer to the completion of the task, and the learning process of the agent is accelerated. The specific calculation method of the reward is:

[0099]

[0100]

[0101] where R is the reward function, E is the average work rate of the worker, and V is the worker load variance.

[0102] Step 4: When the number of samples in the experience replay pool exceeds a certain number, the agent samples from the respective experience replay pool, each time sampling a batch of data according to the sampling probability of the sample for training, and using a multi-step prediction method to predict the future cumulative return of each sample data.

[0103] The experience pool capacity is fixed, and the samples are sampled from the experience pool by improving the priority experience replay method. When calculating the sampling probability, the time of experience storage and the number of times of sampling are added to improve the utilization efficiency of samples and the learning speed of the agent. The probability of sampling a sample is:

[0104]

[0105] where p i represents the probability of being sampled, δ i represents the time difference error, c is a constant greater than 0 for controlling the proportion of the number of times of sampling; t represents the current total number of sampling times, N t (i) represents the number of times the sample is sampled, and the initial value is 1. When a sample is newly stored in the experience pool, its sampling probability is set to a fixed large constant pc, which ensures that the newly stored sample is preferentially sampled. This can avoid the situation that the sample is removed from the experience pool before being sampled.

[0106] To avoid the priority replay from changing the sample distribution in an uncontrollable manner, importance weights are used to correct this bias:

[0107]

[0108] where w i represents the importance sampling weight, N represents the number of samples, p i represents the probability of being sampled, and β represents the degree of compensation for the bias introduced by non-uniform probability sampling. When β = 1, it is completely compensated.

[0109] Step 5: Update the value function network of the agent using the sample data and the cumulative return prediction value in step 4. The specific process is as follows:

[0110] Initialize the evaluation network and randomly generate the weight w. Initialize the target network, and the target network weight w' = w.

[0111] Initialize the environment state S tThe environment state is fed back to the agent, and the action of the agent is generated by combining the noise network with the greedy strategy, and the reward r from the environment is received t and the new state S t+1 The sample trajectory (S t ,a t ,r t ,S t+1 ) is stored in the experience pool.

[0112] The calculation method of the noise network layer is as follows:

[0113] y = wx + b

[0114] w = u w + σ w ⊙ ε w

[0115] b = μ b + σ b ⊙ ε b

[0116] Where y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weights and biases of the noise layer respectively, and ε w and ε b are Gaussian distributed noises of w and b, which are randomly generated and do not participate in the gradient update process.

[0117] The greedy strategy is naive exploration, which preferentially selects the behavior currently considered to be the highest value, and its specific calculation method is as follows:

[0118]

[0119] Where a represents the decision action of the agent, Q(a) represents the value of action a, and ε represents the probability of randomly selecting an action.

[0120] A batch of sample trajectories is randomly extracted from the experience pool, and the gradient descent method is used for updating, and the weights of the target network are updated every fixed step.

[0121] Step 6: Repeat steps 3 to 5 until the specified round is reached, and each agent converges to the corresponding optimal value function. Fix the network parameters of the agent and deploy it on the workshop scheduling solution.

[0122] After deploying the parameters of the agent, the process information, master plan, personnel information, equipment information and material matching time of the assembly task to be scheduled are synchronized to the simulation model, and the simulation model generates the final scheduling scheme by interacting with the agent, including the production plan of the assembly personnel and the station production plan, such as Figure 6 a part of the personnel production plan diagram, such as Figure 7A production plan diagram for a certain station position.

[0123] The above merely illustrates the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered as falling within the protection scope of the present application.

Claims

1. A multi-station assembly shop intelligent scheduling method fusing proactive assessment, characterized in that, The method comprises the following steps: (1) constructing a simulation workshop of reaction physical workshop operation state and operation logic, decomposing workshop scheduling process, constructing an agent based on assembly instruction transfer and decision action, and building a Markov process model; the workshop operation logic and operation law are multi-station assembly workshops, and the assembly instruction is used to guide the actual assembly process; (2) using a material set-up time evaluation model to analyze and evaluate the material set-up required by the workshop assembly, using an assembly progress evaluation model to evaluate the future progress of the workshop; the evaluation results, workshop process information, master plan, personnel information and equipment information are used to initialize the simulation workshop to form a complete agent learning environment; (3) the agent and the simulation environment interact through the Socket interface and the TCP communication protocol, the agent obtains the observation state, state transition and instant reward from the simulation environment, and feeds back the action taken to the corresponding state of the environment; the interaction trajectory is composed of sample data in the form of five-tuple, and stored in the experience replay pool in the structure of binary tree; the agent comprises an importance evaluation agent, a matching personnel agent and a decision sequence agent; the importance evaluation agent obtains the state information related to the instruction, and gives the priority of the assembly instruction to be decided to the simulation environment; the matching personnel agent obtains the assembly instruction state to be matched and the related information and state of all assembly personnel, gives the decision priority rule, and matches the assembly personnel according to the experience rule of the agent decision, and feeds back to the workshop simulation environment; the decision sequence agent obtains the production state of the workshop, including the progress of each station, the state of the buffer area and the storage area, and the instruction group information to be decided, and selects the next assembly instruction to be executed according to the experience rule of the agent decision; the reward obtained by the agent is a compound reward, and the optimization target of the agent is to minimize the maximum completion time and minimize the work load variance of the assembly personnel; the reward obtained by the agent is a progressive reward, the sum of the assembly progress of each station is used to weight the reward, so that the reward obtained when the task is completed is larger, and the learning process of the agent is accelerated; (4) when the number of samples in the experience replay pool exceeds a certain number, the agent samples from the respective experience replay pool, samples a batch of data according to the sampling probability of the sample each time for training, and predicts the future cumulative return of each sample data by a multi-step prediction method; (5) using the sample data and the cumulative return prediction value in step (4) to update the value function network of the agent; (6) repeating steps (3)-(5) until a specified round is reached, and each agent converges to the corresponding optimal value function; the network parameters of the fixed agent are deployed on the workshop scheduling solution. The material fitting time evaluation model of step (2) evaluates the assembly instruction list containing necessary components; each assembly instruction has a corresponding material list, and the assembly instruction can be executed only when the material list reaches a certain fitting rate and does not lack necessary components; the fitting rate of each assembly instruction is judged through warehouse information, and if the fitting rate meets the standard, it is judged whether the necessary components are in stock; if in stock, the state of the assembly instruction is set to executable, otherwise the fitting time of the necessary components needs to be evaluated, and the state of the assembly instruction is executable after the fitting time; The assembly progress evaluation model of step (2) relies on the simulated workshop built, synchronizes the current assembly progress of the real workshop, simulates the workshop production in the future period of time, judges whether all assembly instructions can be completed before the delivery node; if not, make corresponding adjustments; the specific adjustment methods include increasing the working hours of assembly personnel per day, increasing the number of assembly personnel in the production line, changing the production line process, synchronizing the adjusted workshop related data to the simulation environment for verification until the delivery node requirement is met.

2. The method of claim 1, wherein the method is a method of intelligent scheduling of a multi-station assembly shop with proactive assessment of fusion, characterized in that, The material fitting time evaluation model of step (2) takes the state of the processing workshop as input, including material list information, buffer zone state, machine tool state, work-in-process state, extracts time sequence features through GRU network layer, and its specific expression is as follows: r t = σ(W r · [h t-1 , x t ]) z t = σ(W Z x t + U z h t-1 ) where r t represents an update gate, z t represents a reset gate, represents the hidden state of the node, h t represents the output of the node; W r , W Z , W h , U z respectively represent the corresponding parameter matrix, x t is the input of the layer, h t-1 is the hidden state; after extracting the timing characteristics, the full connection network layer outputs the full set time of the material.

3. The method of claim 1, wherein the method is a method of intelligent scheduling of a multi-station assembly shop with proactive assessment of fusion, characterized in that, The input state AgentS1 of the importance evaluation agent is: AgentS1={SA} wherein SA represents the characteristic related to the assembly instruction, represents the number of the i-th assembly instruction of the k-th assembly stage of the station S, represents the man-hour of the assembly instruction, represents the total man-hour of all instructions in parallel with the instruction, represents the current available workstations of the station where the assembly instruction is located, represents the total man-hour of unexecuted instructions of the current production batch of the station where the assembly instruction is located, represents the criticality of the assembly instruction, represents the parallel number of the assembly instruction; the output is the priority of the assembly instruction, i.e. a number from 0 to 9, the larger the number, the higher the importance and priority of the assembly instruction. The input state AgentS2 of the matching personnel agent is: AgentS2={SP,SA} SP = {P 1,num ,P 1,at ,P 1,e ,P 2,num ,P 2,at ,P 2,e ,...,P p,num ,P p,at ,P p,e ,S 1,pro ,S 2,pro ,...,S s,pro} wherein SP represents a feature related to a person, P j,num represents the number of remaining tasks of the jth assembly worker, P j,at represents the total man-hours of the remaining tasks of the jth assembly worker, P j,e represents the work rate of the jth assembly worker; the assembly progress S of the assembly station S S,pro depends on the number of AOs that need to be completed in the tasks of the stage S S,aon and the number of AOs that have been completed at the current time S S,fn ; The input state AgentS3 of the decision sequence agent is: AgentS3={SS,SPL} SS = {S 1,num , S 1,pro , S 1,avn , S 1,bn , S 1,bat , S 1,strn , S 1,strnt , S 2,num ,S 2,pro ,S 2,avn ,S 2,bn ,S 2,bat ,S 2,strn ,S 2,strnt ,..., S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt} SPL = {PL an , PL wt , PL ast , PL 1,num , PL 2,num ,..., PL s,num} wherein SS represents a state related to the station positions of the workshop, S S,num represents the station number of the Sth station position, S S,pro represents the assembly progress of the Sth station position, S S,avn represents the number of available workstations in the Sth station position, S S,bn represents the number of AOs in the buffer area of the Sth station position, S S,bat represents the average AO working hours in the buffer area of the Sth station position, S S,strn represents the number of AOs in the storage area of the Sth station position, S s,strnt represents the average AO working hours in the storage area of the Sth station position; SPL represents a state related to the assembly personnel list, PL an represents the number of AOs in the assembly personnel task list, PL wt represents the average waiting time of AOs in the assembly personnel task list, PL ast represents the average assembly time of AOs in the assembly personnel task list, PL S,num represents the number of AOs belonging to the Sth station position in the assembly personnel task list.

4. The method of claim 3, wherein the method further comprises: The reward obtained by the agent is a progressive reward, which adopts the sum of assembly progress of each station to weight the reward, so that the reward obtained is greater when it is closer to the task completion, which speeds up the learning process of the agent; the specific calculation method of the reward is: Wherein, R is the reward function, E is the average work rate of workers, and V is the worker load variance.

5. The method of claim 1, wherein, The experience replay pool capacity of step (4) is fixed, and the improved priority experience replay is used to sample from the experience pool; the time of experience storage and the number of times of sampling are added when calculating the sampling probability, which improves the sample utilization efficiency and the learning speed of the agent, and the probability of sampling the sample is: where p i represents the probability of being sampled, δ i represents the timing difference error, c is a constant greater than 0 for controlling the proportion of the number of samplings; t represents the current total number of samplings, N t (i) represents the number of times the sample is sampled, the initial value is 1; when the sample is newly stored in the experience pool, the sampling probability is set to a fixed larger constant pc, ensuring that the newly stored sample is preferentially sampled; In order to avoid the priority replay from changing the sample distribution in an uncontrollable way, importance weight is used to correct this bias: where w i represents the importance sampling weight, N represents the number of samples, p i represents the probability of being sampled, and β represents the degree of compensation for the bias introduced by non-uniform probability sampling, and when β = 1, the compensation is complete.

6. The method of claim 1, wherein, The implementation process of step (5) is as follows: Initialize the evaluation network, randomly generate the weight w, initialize the target network, and the weight w' of the target network is w; Initialize the environment state S t , the agent's action is generated by combining the noise network layer with the greedy strategy, and the reward r from the environment is received t , and the new state S t+1 , and the sample trajectory (S t ,a t ,r t ,S t+1 ) is stored in the experience pool; Randomly extract a batch of sample trajectories from the experience pool, and update them using the gradient descent method. Update the weight of the target network every fixed step.

7. The method of claim 6, wherein the method further comprises: The specific calculation method of the noise network layer is as follows: y=wx+b w = u w +σ w ⊙ε w b = μ b + σ b ⊙ ε b where y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weight and bias of the noise layer, respectively, and ε w and ε b are Gaussian distribution noises of w and b, respectively, randomly generated in the process not participating in gradient update.

8. The method of claim 6, wherein the method further comprises: The greedy strategy is naive exploration, which preferentially selects the behavior currently considered to be the highest value, and its specific calculation method is as follows: Wherein, a represents the decision action of the agent, Q(a) represents the value of action a, and ε represents the probability value of randomly selecting action.

Images