Multi-station assembly workshop intelligent production scheduling method fusing proactive evaluation

Abstract

The invention discloses a multi-station assembly workshop intelligent production scheduling method fused with proactive evaluation, and the method comprises the steps: carrying out the proactive evaluation of the material neatness and assembly progress of a workshop through combining the material processing data, warehouse data, distribution system information, assembly progress and other information in a production line management and control system; establishing a simulation workshop based on discrete event simulation to simulate a real workshop operation condition, initializing a simulation environment by using a proactive evaluation result, workshop related process data and the like, and designing a multi-agent negotiation model to generate a production scheduling scheme and a resource demand list; a coping scheme combining manual simulation and intelligent agent simulation is formulated according to abnormal events occurring in the production process. According to the method, the problem of frequent plan change caused by disturbance events such as material shortage can be avoided to a certain extent, and a globally optimal workshop production scheduling result can be quickly generated.

Description

Technical Field

[0001] The present invention belongs to the field of workshop planning and scheduling, and in particular relates to an intelligent production scheduling method for a multi-station assembly workshop integrating proactive evaluation. Background Art

[0002] Workshop production planning has become an important decision-making task to support and improve the production efficiency of manufacturing enterprises, logistics and service providers. The workshop scheduling problem can be described as the process of allocating limited resources to each production task according to certain constraints in the master plan. The production tasks of a multi-station assembly workshop are divided according to the stations. There is a complex assembly sequence relationship within the stations and between the stations. As the number of tasks increases, the planning and scheduling of the workshop becomes more and more complicated. In actual production, relying on manual experience can only give a rough planning node, and it is impossible to give a detailed production plan. In addition, it is not only difficult to comprehensively consider factors such as personnel characteristics, personnel load balancing, and coordination between personnel and automatic equipment in an uncertain production environment, but also the scheduling efficiency is low, and the scheduling results lack scientific rationality, which seriously restricts the planning management level of the workshop. When facing large-scale problems, heuristic algorithms take a long time to solve and are difficult to converge to the optimal solution. Deep reinforcement learning algorithms are mostly used in scenarios such as flexible workshops or flow workshops, but are rarely used in scenarios such as multi-station large-scale assembly workshops.

[0003] There are many materials in the assembly workshop and their sources are complex. Delays in the supply of key materials can cause the production line to stagnate or even shut down. Traditional planning methods plan under ideal conditions without considering the completeness of materials and possible delays in progress, making it difficult to ensure the executability of the plan. Summary of the invention

[0004] Purpose of the invention: In view of the above-mentioned deficiencies of the existing problems, the present invention provides a method for intelligent scheduling of multi-station assembly workshops integrating proactive evaluation, constructs a workshop simulation model based on discrete events, avoids certain risk events through proactive evaluation of materials and progress, designs a multi-agent decision-making model with offline training and online matching and a compound reward mechanism, and realizes intelligent decision-making on workshop scheduling.

[0005] Technical solution: The intelligent production scheduling method for a multi-station assembly workshop integrating proactive evaluation described in the present invention comprises the following steps:

[0006] (1) Build a simulation workshop that reflects the operating status and operation logic of the physical workshop, decompose the workshop scheduling process, build intelligent agents and decision-making actions based on the transfer of assembly instructions, and build a Markov process model;

[0007] (2) Use the material matching time assessment model to analyze and evaluate the matching of materials required for workshop assembly, and use the assembly progress assessment model to evaluate the future progress of the workshop; initialize the simulation workshop with the assessment results, workshop process information, master plan, personnel information, and equipment information to form a complete intelligent agent learning environment;

[0008] (3) The intelligent agent and the simulation environment exchange information through the Socket interface and TCP communication protocol. The intelligent agent obtains the observed state, state transition, and immediate reward from the simulation environment, and feeds back the action taken by the environment in the corresponding state. The interaction trajectory is composed of sample data in the form of five-tuples and is stored in the experience replay pool using a binary tree structure.

[0009] (4) When the number of samples in the experience replay pool exceeds a certain number, the agents sample from their respective experience replay pools, sampling a batch of data for training each time according to the sampling probability of the samples, and predicting the future cumulative return of each sample data through a multi-step prediction method;

[0010] (5) Using the sample data in step (4) and the cumulative reward prediction value, update the value function network of the intelligent agent;

[0011] (6) Repeat steps (3)-(5) until the specified round is reached and each agent converges to the corresponding optimal value function; fix the network parameters of the agent and deploy it to solve the workshop production scheduling problem.

[0012] Furthermore, the material completeness time evaluation model in step (2) evaluates the assembly instruction list containing necessary parts; each assembly instruction has a corresponding bill of materials, and the assembly instruction can be executed only when the bill of materials reaches a certain completeness rate and there are no missing necessary parts; the completeness rate of each assembly instruction is determined by warehouse information, and if the completeness rate meets the standard, it is determined whether the necessary parts are in stock; if they are in stock, the status of the instruction is set to executable, otherwise the completeness time of the necessary parts needs to be evaluated, and the status of the instruction is executable after the completeness time.

[0013] Furthermore, the material set time assessment model described in step (2) takes the state of the processing workshop as input, including bill of materials information, buffer state, machine tool state, and work-in-progress state, and extracts time series features through the GRU network layer. Its specific expression is as follows:

[0014] r t =σ(W r ·[h t-1 , x t ])

[0015] z t =v(W Z xt +U z h t-1 )

[0016]

[0017] Among them, r t represents the update gate, z t Represents the reset gate, represents the hidden state of the node, h t Represents the output of the node; W r , W Z , W h , U z Represent the corresponding parameter matrices, x t is the input of this layer, h t-1 It is a hidden state; after extracting the time series features, the complete set time of the materials is output through the fully connected network layer.

[0018] Furthermore, the assembly progress evaluation model described in step (2) relies on the constructed simulation workshop, synchronizes the current assembly progress of the real workshop, simulates the workshop production situation in the future, and determines whether the assembly operations of all assembly instructions can be completed before the delivery node; if not, corresponding adjustments are made; specific adjustment methods include increasing the working hours of assembly personnel per day, increasing the number of assembly personnel in the production line, changing the production line process, and synchronizing the adjusted workshop-related data to the simulation environment for verification until the delivery node requirements are met.

[0019] Furthermore, the agent in step (3) includes an importance evaluation agent, a matching agent, and a decision sequence agent; the importance evaluation agent obtains state information related to the instruction, and gives the priority of the assembly instruction to be decided and feeds it back to the simulation environment; the matching agent obtains the state of the assembly instruction to be matched and the relevant information and state of all assemblers, gives the priority rule of decision, and matches the assemblers according to the empirical rules of the agent decision, and feeds it back to the workshop simulation environment; the decision sequence agent obtains the production state of the workshop, including the progress of each station, the state of the buffer area and the storage area, and the information of the instruction group to be decided, and selects the next assembly instruction to be executed according to the empirical rules decided by the agent; wherein, the input state AgentS of the importance evaluation agent 1 for:

[0020] AgentS 1 ={SA}

[0021]

[0022] Among them, SA represents the characteristics related to the assembly instruction, represents the number of the i-th assembly instruction at the k-th assembly stage at station S, Indicates the assembly instruction working hours, Indicates the total working hours of all instructions parallel to this instruction. Indicates the currently available workstations where the assembly instruction is located. Indicates the total working hours of the unexecuted instructions of the current production batch at the station where the assembly instruction is located. Indicates the importance of assembly instructions. Indicates the number of parallel assembly instructions; the output is the priority of the assembly instruction, that is, a number from 0 to 9. The larger the value, the higher the importance and priority of the assembly instruction;

[0023] Matching the input state of the agent AgentS 2 for:

[0024] AgentS 2 ={SP,SA}

[0025] SP = {P 1,num ,P 1,at ,P 1,e ,P 2,num ,P 2,at ,P 2,e ,...,P p,num ,P p,at ,P p,e ,S 1,pro ,S 2,pro ,...,S s,pro}

[0026]

[0027] Among them, SP represents the characteristics related to personnel, P j,num represents the number of remaining tasks for the jth group of assemblers, P j,at represents the total remaining task time of the jth group of assemblers, P j,e represents the work rate of the jth group of assemblers; the assembly progress S of assembly station S S,pro Depends on the number of AOs that need to be completed in this stage of the task S S,aon And the number of AOs completed at the current moment S S,fn ;

[0028] The input state of the decision-making sequence agent AgentS 3 for:

[0029] AgentS 3 ={SS,SPL}

[0030] SS={S 1,num ,S 1,pro ,S1,avn ,S 1,bn ,S 1,bat ,S 1,strn ,S 1,strnt ,

[0031] S 2,num ,S 2,pro ,S 2,avn ,S 2,bn ,S 2,bat ,S 2,strn ,S 2,strnt ,...,

[0032] S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt}

[0033] SPL = {PL an ,PL wt ,PL ast ,PL 1,num ,PL 2,num ,...,PL s,num}

[0034] Among them, SS represents the status related to the workshop station, S S,num Indicates the station number of the Sth station, S S,pro represents the assembly progress of the Sth station, S S,avn represents the number of available workstations at the Sth station, S S,bn represents the number of AOs in the cache of the Sth station, S S,bat represents the average AO working time in the buffer area of ​​the Sth station, S S,strn Indicates the number of AOs in the storage area of ​​the Sth station, S s,strnt represents the average AO working time of the Sth station storage area; SPL represents the status related to the assembly personnel list, PL an Indicates the number of AOs in the assembler's task list, PL wt represents the average waiting time of AO in the assembler task list, PL ast represents the average assembly time of AO in the assembler's task list, PL S,num Indicates the number of AOs belonging to the S station in the assembler's task list.

[0035] Furthermore, the reward obtained by the agent is a progressive reward, and the reward is weighted by the sum of the assembly progress of each station, so that the closer the task is to completion, the greater the reward obtained, which speeds up the learning process of the agent; the specific calculation method of the reward is:

[0036]

[0037] Among them, R is the reward function, E is the average work rate of workers, and V is the variance of worker load.

[0038] Furthermore, the capacity of the experience pool described in step (4) is fixed, and sampling is performed from the experience pool by improving the priority experience playback method; the time when the experience is stored and the number of times it is sampled are added when calculating the sampling probability, which improves the utilization efficiency of the sample and the learning speed of the intelligent agent. The probability of the sample being sampled is:

[0039]

[0040] Among them, p i represents the probability of being sampled, δ i represents the timing difference error, c is a constant greater than 0 and is used to control the proportion of sampling times; t represents the current total number of sampling times, N t (i) indicates the number of times the sample is sampled, with an initial value of 1. When a sample is newly stored in the experience pool, its sampling probability is set to a fixed large constant pc to ensure that the newly stored sample is sampled first.

[0041] To prevent prioritized replay from changing the sample distribution in an uncontrolled way, importance weights are used to correct for this bias:

[0042]

[0043] Among them, w i represents the importance sampling weight, N represents the number of samples, and p i represents the probability of being sampled, β represents the degree of compensation for the deviation introduced by non-uniform probability sampling, and when β=1, it is fully compensated.

[0044] Furthermore, the implementation process of step (5) is as follows:

[0045] Initialize the evaluation network, randomly generate weights w, initialize the target network, and the target network weights w'=w;

[0046] Initialize the environment state S t , feeds back the environment state to the agent, generates the agent's actions by combining the noise network layer with the greedy strategy, and receives the reward r from the environment t And the new state S t+1 , and the sample trajectory (S t ,a t ,r t ,S t+1 ) is stored in the experience pool;

[0047] A batch of sample trajectories is randomly selected from the experience pool and updated using the gradient descent method to update the weights of the target network at fixed steps.

[0048] Furthermore, the noise network layer is specifically calculated as follows:

[0049] y=wx+b

[0050] w=u w +σ w ⊙ε w

[0051] b=μ b +σ b ⊙ε b

[0052] Among them, y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weight and bias of the noise layer respectively, ε w and ε b Gaussian noises are w and b respectively, which are randomly generated and do not participate in the gradient update process.

[0053] Furthermore, the greedy strategy is a naive exploration, which gives priority to the behavior that is currently considered to have the highest value. The specific calculation method is as follows:

[0054]

[0055] Among them, a represents the decision action of the agent, Q(a) represents the value of action a, and ε represents the probability value of randomly selecting an action.

[0056] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. Based on the characteristics of multi-station workshops, the present invention decomposes the workshop production scheduling process and constructs intelligent agent decision actions based on the transfer of assembly instructions to realize the Markov process model, which is more in line with the production logic of the actual workshop and is suitable for solving large-scale multi-station production scheduling problems;

[0058] 2. The present invention integrates proactive assessment technology to schedule production in workshops, and timely and effectively realizes information exchange between various production workshops. By evaluating the material set-up time and assembly progress, it effectively avoids frequent plan changes caused by material delivery delays and progress abnormalities, and ensures the rationality and executability of the plan;

[0059] 3. The present invention constructs a composite reward mechanism that combines production progress and worker productivity, improves the priority experience playback, considers the time when the sample enters the experience pool and the number of times it is sampled when sampling, improves the utilization rate of the sample, and accelerates the learning speed of the intelligent agent. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of the present invention;

[0061] Figure 2 It is a schematic diagram of the production rules and operation logic of a multi-station assembly workshop of the present invention;

[0062] Figure 3 is a schematic diagram of the input, output and state transition of each intelligent agent of the present invention;

[0063] Figure 4 is a schematic diagram of the interaction relationship between each intelligent agent and the simulation environment of the present invention;

[0064] Figure 5 It is a schematic diagram of some initialization data of the present invention;

[0065] Figure 6 It is a schematic diagram of some personnel plans solved by the present invention;

[0066] Figure 7 It is a schematic diagram of a partial station plan solved by the present invention. DETAILED DESCRIPTION

[0067] The present invention is further described in detail below with reference to the accompanying drawings.

[0068] The present invention proposes to provide a multi-station assembly workshop intelligent production scheduling method integrating proactive evaluation, and the specific implementation method is as follows: Figure 1 As shown, the following steps are included:

[0069] Step 1: Build a simulation workshop that can truly reflect the operating rules and logic of the physical workshop, decompose the workshop scheduling process, build intelligent agents and corresponding actions based on the transfer of assembly instructions, and realize the Markov process model.

[0070] The operation logic and operation rules of the workshop are multi-station assembly workshops, and the actual assembly process is guided by assembly instructions. Figure 2 As shown in the figure, its main operating rules include that each station must meet the established assembly sequence. When executing a batch of manufacturing tasks, an activity can only start when and only after a set of previous tasks are completed, that is, 25A can start assembly only when and only after 26A at station 26 completes assembly. The execution of assembly instructions within a station has strict serial and parallel constraints. Instructions at the same T level are parallel instructions, and instructions at different T levels are serial according to the size of the T level.

[0071] The simulated workshop includes personnel, equipment, assembly instructions, tooling, components and their corresponding attributes. It uses discrete event simulation to collect workshop status data through data acquisition devices in the physical workshop and manual entry, and updates the current operating status of the workshop to the simulation model through a specific communication protocol.

[0072] Step 2: Use the material set time assessment model to analyze and assess the material set required for workshop assembly, and use the assembly progress assessment model to assess the workshop and future progress. Initialize the simulation workshop with the assessment results, workshop process information, master plan, personnel information, equipment information, etc. to form a complete intelligent agent learning environment. Figure 3 The figure shows the process information and material set information imported into the simulation model.

[0073] The material completeness time evaluation model evaluates the assembly instruction list containing necessary parts. Each assembly instruction has a corresponding bill of materials. The instruction can be executed only when the bill of materials reaches a certain completeness rate and there are no missing necessary parts. The material status (the required materials are currently in the warehouse, in processing, in distribution, delivered, etc.) comes from the warehouse, machining workshop and distribution system. The completeness rate of each assembly instruction is determined by the warehouse information. If the completeness rate meets the standard, it is determined whether the necessary parts are in stock. If they are in stock, the status of the instruction is set to executable. Otherwise, the completeness time of the necessary parts needs to be evaluated. After the completeness time, the status of the instruction is executable.

[0074] The material set time assessment model takes the status of the processing workshop as input, including bill of materials information, buffer status, machine tool status, and work-in-progress status. The GRU network layer extracts the timing features. The specific expression is as follows:

[0075] r t =σ(W r ·[h t-1 , x t ])

[0076] z t =σ(W Z x t +U z h t-1 )

[0077]

[0078] Among them, r t represents the update gate, z t Represents the reset gate, represents the hidden state of the node, h t Represents the output of this node. r , W Z , W h, U z Represent the corresponding parameter matrices, x t is the input of this layer, h t-1 After extracting the time series features, the fully connected network layer outputs the time of material completion.

[0079] The assembly progress evaluation model relies on the built simulation workshop, synchronizes the current assembly progress of the real workshop, simulates the workshop production situation in the future, and determines whether all assembly instructions can be completed before the delivery node under ideal conditions. If not, make corresponding adjustments, including increasing the working hours of assembly personnel every day, increasing the number of assembly personnel in the production line, changing the production line process, and synchronizing the changed data to the simulation environment for verification.

[0080] Step 3: The agent and the simulation environment exchange information through the Socket interface and TCP communication protocol. The agent obtains the observed state, state transition, and immediate reward from the simulation environment, and feeds back the action taken by the environment in the corresponding state. The interaction trajectory is composed of sample data in the form of five-tuples and is stored in the experience replay pool using a binary tree structure.

[0081] like Figure 4 As shown in the figure, the intelligent agent is composed of three agents, and the three agents are in a completely cooperative relationship. Agent 1 is an importance assessment agent, which obtains status information related to instructions, and gives the priority of assembly instructions to be decided and feeds it back to the simulation environment. Agent 2 is a matching personnel agent, which obtains the status of assembly instructions to be matched and the relevant information and status of all assemblers, gives priority rules for decision-making, and matches assemblers according to the empirical rules of agent decision-making, and feeds back to the workshop simulation environment. Agent 3 is a decision-making sequence agent, which obtains the production status of the workshop, including the progress of each station, the status of the buffer area and storage area, and the information of the instruction group to be decided, and selects the next assembly instruction to be executed according to the empirical rules decided by the agent. As shown in the figure Figure 5 As shown in Figure 1, the agents are in the same environment and make decisions at different time points. Different agents can exchange information and have a complete cooperative relationship with each other. The state transfer of the environment can occur on the same agent or between different agents. 1 for:

[0082] AgentS 1 ={SA}

[0083]

[0084] Among them, SA represents the characteristics related to the assembly instruction, represents the number of the i-th assembly instruction at the k-th assembly stage at station S, Indicates the assembly instruction working hours, Indicates the total working hours of all instructions parallel to this instruction. Indicates the currently available workstations where the assembly instruction is located. Indicates the total working hours of the unexecuted instructions of the current production batch at the station where the assembly instruction is located. Indicates the importance of assembly instructions. Indicates the number of parallel assembly instructions. The output is the priority of the assembly instruction, that is, a number from 0 to 9. The larger the value, the higher the importance and priority of the assembly instruction.

[0085] Matching the input state of the agent AgentS 2 for:

[0086] AgentS 2 ={SP,SA}

[0087] SP = {P 1,num ,P 1,at ,P 1,e ,P 2,num ,P 2,at ,P 2,e ,...,P p,num ,P p,at ,P p,e ,S 1,pro ,S 2,pro ,...,S s,pro}

[0088]

[0089] Among them, SP represents the characteristics related to personnel, P j,num represents the number of remaining tasks for the jth group of assemblers, P j,at represents the total remaining task time of the jth group of assemblers, P j,e represents the work rate of the jth group of assemblers. The assembly progress S of assembly station S S,pro Depends on the number of AOs that need to be completed in this stage of the task S S,aon And the number of AOs completed at the current moment S S,fn .

[0090] The input state of the decision-making sequence agent AgentS 3 for:

[0091] AgentS 3 ={SS,SPL}

[0092] SS={S 1,num ,S 1,pro ,S1,avn ,S 1,bn ,S 1,bat ,S 1,strn ,S 1,strnt ,

[0093] S 2,num ,S 2,pro ,S 2,avn ,S 2,bn ,S 2,bat ,S 2,strn ,S 2,strnt ,...,

[0094] S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt}

[0095] SPL = {PL an ,PL wt ,PL ast ,PL 1,num ,PL 2,num ,...,PL s,num}

[0096] Among them, SS represents the status related to the workshop station, S S,num Indicates the station number of the Sth station, S S,pro represents the assembly progress of the Sth station, S S,avn represents the number of available workstations at the Sth station, S S,bn represents the number of AOs in the cache of the Sth station, S S,bat represents the average AO working time in the buffer area of ​​the Sth station, S S,strn Indicates the number of AOs in the storage area of ​​the Sth station, S s,strnt Indicates the average AO working time of the Sth station storage area. SPL indicates the status related to the assembly personnel list, PL an Indicates the number of AOs in the assembler's task list, PL wt represents the average waiting time of AO in the assembler task list, PL ast represents the average assembly time of AO in the assembler's task list, PL S,num Indicates the number of AOs belonging to the S station in the assembler's task list.

[0097] The actions of the matching agent and the decision sequence agent are composed of empirical rules.

[0098] The reward obtained by the agent is a composite reward, and the optimization goal of the agent is to minimize the maximum completion time and minimize the workload variance of the assemblers. The higher the worker's work rate, the shorter the corresponding completion time should be. The average work rate of the workers and the variance of the workers' load are used as the reward of the agent. When obtaining rewards, the present invention designs a progressive reward, which uses the sum of the assembly progress of each station to weight the reward, so that the closer the task is to completion, the greater the reward obtained, which speeds up the learning process of the agent. The specific calculation method of the reward is:

[0099]

[0100]

[0101] Among them, R is the reward function, E is the average work rate of workers, and V is the variance of worker load.

[0102] Step 4: When the number of samples in the experience replay pool exceeds a certain number, the agents sample from their respective experience replay pools, sampling a batch of data for training each time according to the sampling probability of the samples, and using a multi-step prediction method to predict the future cumulative return of each sample data.

[0103] The capacity of the experience pool is fixed, and sampling is performed from the experience pool by improving the priority experience playback method. The time when the experience is stored and the number of times it is sampled are added when calculating the sampling probability, which improves the utilization efficiency of the sample and the learning speed of the agent. The probability of a sample being sampled is:

[0104]

[0105] Among them, p i represents the probability of being sampled, δ i represents the timing difference error, c is a constant greater than 0 and is used to control the proportion of sampling times; t represents the current total number of sampling times, N t (i) indicates the number of times the sample is sampled, with an initial value of 1. When a sample is newly stored in the experience pool, its sampling probability is set to a fixed large constant pc to ensure that the newly stored sample is sampled first. This can avoid the situation where the sample exits the experience pool before being sampled.

[0106] To prevent prioritized replay from changing the sample distribution in an uncontrolled way, importance weights are used to correct for this bias:

[0107]

[0108] Among them, w i represents the importance sampling weight, N represents the number of samples, and p irepresents the probability of being sampled, β represents the degree of compensation for the deviation introduced by non-uniform probability sampling, and when β=1, it is fully compensated.

[0109] Step 5: Use the sample data and cumulative reward prediction values ​​in step 4 to update the value function network of the agent. The specific process is as follows:

[0110] Initialize the evaluation network, randomly generate weights w, initialize the target network, and the target network weights w'=w.

[0111] Initialize the environment state S t , feeds back the environment state to the agent, generates the agent's actions by combining the noise network with the greedy strategy, and receives the reward r from the environment t And the new state S t+1 , and the sample trajectory (S t ,a t ,r t ,S t+1 ) is stored in the experience pool.

[0112] The noise network layer is calculated as follows:

[0113] y=wx+b

[0114] w=u w +σ w ⊙ε w

[0115] b=μ b +σ b ⊙ε b

[0116] Among them, y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weight and bias of the noise layer respectively, ε w and ε b Gaussian noises are w and b respectively, which are randomly generated and do not participate in the gradient update process.

[0117] The greedy strategy is a naive exploration, which gives priority to the behavior that is currently considered to have the highest value. The specific calculation method is as follows:

[0118]

[0119] Among them, a represents the decision action of the agent, Q(a) represents the value of action a, and ε represents the probability of randomly selecting an action.

[0120] A batch of sample trajectories is randomly selected from the experience pool and updated using the gradient descent method to update the weights of the target network at fixed steps.

[0121] Step 6: Repeat steps 3 to 5 until the specified round is reached and each agent converges to the corresponding optimal value function. Fix the network parameters of the agent and deploy it to solve the workshop production scheduling problem.

[0122] After the parameters of the intelligent agent are deployed, the process information, master plan, personnel information, equipment information, and material set-up time of the assembly tasks to be scheduled are synchronized to the simulation model. The simulation model interacts with the intelligent agent to generate the final production scheduling plan, including the production plan of the assembly personnel and the production plan of the station, such as Figure 6 A partial schematic diagram of the personnel production plan, such as Figure 7 This is a schematic diagram of the production plan for one of the stations.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-station assembly workshop intelligent production scheduling method integrating proactive evaluation, characterized in that: The following steps are involved: (1) Build a simulation workshop that reflects the operating status and operation logic of the physical workshop, decompose the workshop scheduling process, build intelligent agents and decision-making actions based on the transfer of assembly instructions, and build a Markov process model; (2) Use the material matching time assessment model to analyze and evaluate the matching of materials required for workshop assembly, and use the assembly progress assessment model to evaluate the future progress of the workshop; initialize the simulation workshop with the assessment results, workshop process information, master plan, personnel information, and equipment information to form a complete intelligent agent learning environment; (3) The intelligent agent and the simulation environment exchange information through the Socket interface and TCP communication protocol. The intelligent agent obtains the observed state, state transition, and immediate reward from the simulation environment, and feeds back the action taken by the environment in the corresponding state. The interaction trajectory is composed of sample data in the form of five-tuples and is stored in the experience replay pool using a binary tree structure. (4) When the number of samples in the experience replay pool exceeds a certain number, the agents sample from their respective experience replay pools, sampling a batch of data for training each time according to the sampling probability of the samples, and predicting the future cumulative return of each sample data through a multi-step prediction method; (5) Using the sample data in step (4) and the cumulative reward prediction value, update the value function network of the intelligent agent; (6) Repeat steps (3)-(5) until the specified round is reached and each agent converges to the corresponding optimal value function; fix the network parameters of the agent and deploy it to solve the workshop production scheduling problem.

2. According to the method of intelligent production scheduling of multi-station assembly workshops integrating proactive evaluation according to claim 1, it is characterized in that: The material completeness time evaluation model in step (2) evaluates the assembly instruction list containing the necessary parts; each assembly instruction has a corresponding bill of materials, and the assembly instruction can be executed only when the bill of materials reaches a certain completeness rate and there are no missing necessary parts; the completeness rate of each assembly instruction is determined by warehouse information, and if the completeness rate meets the standard, it is determined whether the necessary parts are in stock; if they are in stock, the status of the instruction is set to executable, otherwise the completeness time of the necessary parts needs to be evaluated, and the status of the instruction is executable after the completeness time.

3. According to the method of intelligent production scheduling of multi-station assembly workshops integrating proactive evaluation according to claim 1, it is characterized in that: The material set time assessment model described in step (2) takes the state of the processing workshop as input, including bill of materials information, buffer state, machine tool state, and work-in-progress state, and extracts time series features through the GRU network layer. Its specific expression is as follows: r t =σ(W r ·[h t-1 ,x t ]) z i =σ(W z x t +U z h t-1 ) Among them, r t represents the update gate, z t Represents the reset gate, represents the hidden state of the node, h t Represents the output of the node; W r , W Z , W h , U z Represent the corresponding parameter matrices, x t is the input of this layer, h t-1 It is a hidden state; after extracting the time series features, the complete set time of the materials is output through the fully connected network layer.

4. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 1, characterized in that: The assembly progress evaluation model described in step (2) relies on the constructed simulation workshop, synchronizes the current assembly progress of the real workshop, simulates the workshop production situation in the future, and determines whether the assembly operations of all assembly instructions can be completed before the delivery node; if not, make corresponding adjustments; specific adjustment methods include increasing the working hours of assembly personnel per day, increasing the number of assembly personnel in the production line, changing the production line process, and synchronizing the adjusted workshop-related data to the simulation environment for verification until the delivery node requirements are met.

5. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 1, characterized in that: The agents in step (3) include an importance evaluation agent, a matching agent, and a decision sequence agent; the importance evaluation agent obtains status information related to the instructions, and gives the priority of the assembly instructions to be decided and feeds it back to the simulation environment; the matching agent obtains the status of the assembly instructions to be matched and the relevant information and status of all assemblers, gives the priority rules for decision-making, and matches the assemblers according to the empirical rules of the agent decision, and feeds back to the workshop simulation environment; the decision sequence agent obtains the production status of the workshop, including the progress of each station, the status of the buffer area and the storage area, and the information of the instruction group to be decided, and selects the next assembly instruction to be executed according to the empirical rules decided by the agent; wherein, the input state AgentS1 of the importance evaluation agent is: AgentS1 = {SA} Among them, SA represents the characteristics related to the assembly instruction, represents the number of the i-th assembly instruction at the k-th assembly stage at station S, Indicates the assembly instruction working hours, Indicates the total working hours of all instructions parallel to this instruction. Indicates the currently available workstations where the assembly instruction is located. Indicates the total working hours of the unexecuted instructions of the current production batch at the station where the assembly instruction is located. Indicates the importance of assembly instructions. Indicates the number of parallel assembly instructions; the output is the priority of the assembly instruction, that is, a number from 0 to 9. The larger the value, the higher the importance and priority of the assembly instruction; The input state of the matching agent AgentS2 is: AgentS2={SP,SA} SP={P 1,num ,P 1,at ,P 1,e ,P 2,num ,P 2,at ,P 2,e ,...,P p,num ,P p,at ,P p,e ,S 1,pro ,S 2,pro ,...,S s,pro } Among them, SP represents the characteristics related to personnel, P j,num represents the number of remaining tasks for the jth group of assemblers, P j,at represents the total remaining task time of the jth group of assemblers, P j,e represents the work rate of the jth group of assemblers; the assembly progress S of assembly station S S,pro Depends on the number of AOs that need to be completed in this stage of the task S S,aon And the number of AOs completed at the current moment S S,fn ; The input state of the decision-making sequence agent AgentS3 is: AgentS3={SS,SPL} SS={S 1,num ,S 1,pro ,S 1,avn ,S 1,bn ,S 1,bat ,S 1,strn ,S 1,strnt , S 2,num ,S 2,pro ,S 2,avn ,S 2,bn ,S 2,bat ,S 2,strn ,S 2,strnt ,..., S s,num ,S s,pro ,S s,avn ,S s,bn ,S s,bat ,S s,strn ,S s,strnt } SPL={PL an ,PL wt ,PL ast ,PL 1,num ,PL 2,num ,...,PL s,num } Among them, SS represents the status related to the workshop station, S S,num Indicates the station number of the Sth station, S S,pro represents the assembly progress of the Sth station, S S,avn represents the number of available workstations at the Sth station, S S,bn represents the number of AOs in the cache of the Sth station, S S,bat represents the average AO working time in the buffer area of ​​the Sth station, S S,strn Indicates the number of AOs in the storage area of ​​the Sth station, S s,strnt represents the average AO working time of the Sth station storage area; SPL represents the status related to the assembly personnel list, PL an Indicates the number of AOs in the assembler's task list, PL wt represents the average waiting time of AO in the assembler task list, PL ast represents the average assembly time of AO in the assembler's task list, PL S,num Indicates the number of AOs belonging to the S station in the assembler's task list.

6. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 1, characterized in that: The reward obtained by the agent is a progressive reward, which is weighted by the sum of the assembly progress of each station, so that the closer the task is to completion, the greater the reward obtained, which speeds up the learning process of the agent; the specific calculation method of the reward is: Among them, R is the reward function, E is the average work rate of workers, and V is the variance of worker load.

7. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 1, characterized in that: The capacity of the experience pool described in step (4) is fixed, and sampling is performed from the experience pool by improving the priority experience playback method; the time when the experience is stored and the number of times it is sampled are added when calculating the sampling probability, which improves the utilization efficiency of the sample and the learning speed of the intelligent agent. The probability of the sample being sampled is: Among them, p i represents the probability of being sampled, δ i represents the timing difference error, c is a constant greater than 0 and is used to control the proportion of sampling times; t represents the current total number of sampling times, N t (i) indicates the number of times the sample is sampled, with an initial value of 1. When a sample is newly stored in the experience pool, its sampling probability is set to a fixed large constant pc to ensure that the newly stored sample is sampled first. To prevent prioritized replay from changing the sample distribution in an uncontrolled way, importance weights are used to correct for this bias: Among them, w i represents the importance sampling weight, N represents the number of samples, and p i represents the probability of being sampled, β represents the degree of compensation for the deviation introduced by non-uniform probability sampling, and when β=1, it is fully compensated.

8. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 1, characterized in that: The implementation process of step (5) is as follows: Initialize the evaluation network, randomly generate weights w, initialize the target network, and the target network weights w'=w; Initialize the environment state S t , feeds back the environment state to the agent, generates the agent's actions by combining the noise network layer with the greedy strategy, and receives the reward r from the environment t And the new state S t+1 , and the sample trajectory (S t ,a t ,r t ,S t+1 ) is stored in the experience pool; A batch of sample trajectories is randomly selected from the experience pool and updated using the gradient descent method, updating the weights of the target network at fixed steps.

9. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 8, characterized in that: The noise network layer is specifically calculated as follows: y=wx+b w=u w +s w ⊙e w b=μ b +s b ⊙e b Among them, y is the output of the noise network layer, x is the input of the noise network layer, w and b are the weight and bias of the noise layer respectively, ε w and ε b Gaussian noises are w and b respectively, which are randomly generated and do not participate in the gradient update process.

10. The method for intelligent production scheduling of a multi-station assembly workshop integrating proactive evaluation according to claim 8, characterized in that: The greedy strategy is a naive exploration, which gives priority to the behavior that is currently considered to have the highest value. The specific calculation method is as follows: Among them, a represents the decision action of the agent, Q(a) represents the value of action a, and ε represents the probability value of randomly selecting an action.

Images