Man-machine cooperation task allocation method and device based on bilateral assembly line balance, medium and product

Abstract

The invention discloses a man-machine cooperation task allocation method and device based on double-side assembly line balance, a medium and a product, and relates to the field of task allocation. Establishing an integer programming model of double-side assembly line balance task allocation considering man-machine cooperation; according to different types of product information and integer programming models, a multi-agent system is constructed based on an Actor-Critic algorithm; the multi-agent system comprises a task agent, a station agent and a mode agent. Respectively interacting the multi-agent system with different parallel environments, and performing state conversion and reward evaluation to obtain a trained multi-agent system; and performing man-machine cooperation task allocation by using the trained multi-agent system. The production efficiency of the assembly line and the man-machine cooperation effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of task allocation, and in particular to a method, device, medium and product for human-machine collaborative task allocation based on bilateral assembly line balancing. Background Art

[0002] In modern manufacturing, assembly line production has become a key technology for large-scale, high-efficiency production. However, due to the diverse nature of the objects being assembled, assembly lines often need to support the assembly of multiple models (product types), making traditional single-line assembly solutions difficult to adapt to the rapid switching between different product models. Furthermore, in a multi-model production environment, assembly tasks must not only be assigned to machines but also be completed in collaboration with human operators.

[0003] Existing technologies for multi-model human-machine collaborative assembly line balancing often employ traditional heuristic algorithms and linear programming methods. However, these methods are limited in effectiveness when faced with dynamic environments and uncertainties. Multi-agent reinforcement learning technology, with its adaptive learning capabilities in complex systems, offers a novel solution, but its effective application for optimizing two-sided assembly line balancing is still lacking.

[0004] Therefore, based on the above problems, there is an urgent need to provide a multi-model bilateral assembly line balanced human-machine collaborative task allocation method to improve the production efficiency of the assembly line and the effect of human-machine collaboration. Summary of the Invention

[0005] The purpose of this application is to provide a method, equipment, medium and product for human-machine collaborative task allocation based on bilateral assembly line balancing, which can improve the production efficiency of the assembly line and the effect of human-machine collaboration.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for allocating tasks of human-machine collaboration based on bilateral assembly line balancing, the method comprising:

[0008] Based on the task data of a bilateral assembly line with human-robot collaboration and information about different types of products, an integer programming model for balanced task allocation on a bilateral assembly line taking human-robot collaboration into account is established. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balancing rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-machine operation constraints on workstation pairings, and task execution sequence constraints as constraint conditions. The task data includes: an assembly task list, the standard operation time of each task, the task priority relationship, and parameters comparing the efficiency of manual and robot operations.

[0009] Based on different types of product information and integer programming models, a multi-agent system is constructed based on the Actor-Critic algorithm. The multi-agent system includes: a task agent, a site agent, and a pattern agent. The task agent uses the task state as a local observation and the task state and site state as a global state. The site agent uses the site state as a local observation and the task state and site state as a global state. The pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state.

[0010] The multi-agent system interacts with different parallel environments, performs state transitions and reward evaluations, and obtains a trained multi-agent system.

[0011] Utilize the trained multi-agent system for human-machine collaborative task allocation.

[0012] Optionally, the integer programming model specifically includes:

[0013] The optimization function F is determined using the formula F = min(α·CT+β·EC-γ·LE); where CT is the cycle time of the assembly line, EC is the energy consumption of the machines on the assembly line, including the working energy consumption and standby energy consumption of the robots, LE is the load balance rate of all workstations on the assembly line, min is the minimum function, α is the weight coefficient of the cycle time of the assembly line, β is the weight coefficient of the energy consumption of the machines on the assembly line, and γ is the weight coefficient of the load balance rate of all workstations on the assembly line;

[0014] Using the formula Determine the assembly line constraints; where K is the set of product models, K = {1, 2, 3, ..., k}, Y kt It means that if the assembly line processes the product of product model k at time t, it is 1, otherwise it is 0;

[0015] Using the formula and formula Determine the execution task constraints; where t s ik is the start time of task i processing product of product model k, t e ik is the end time of task i for processing product model k, ψ is a positive number, X ikwp If product task i is assigned to the p side of workstation w for processing product model k, the value is 1; otherwise, the value is 0; p is the paired workstation; when p = 1, it means the left side; when p = 2, it means the right side;

[0016] Using the formula formula formula and formula Determine the workstation allocation constraints; where J is the workstation set, J = {1, 2, 3, ..., w}, I is the task set, I = {1, 2, 3, ..., i}, A(L) is the set of tasks that must be assigned to the left site, A(L)∈I; A(R) is the set of tasks that must be assigned to the right site, A(R)∈I; A(E) is the set of tasks without direction restrictions, A(E)∈I, and satisfies I = A(L)∪A(R)∪A(E). Use the formula Determine the operation mode constraints; where Z H ikwp If task i is assigned to the p side of workstation w to process product model k and manual operation is selected, the value is 1; otherwise, the value is 0; Z R ikwp If task i is assigned to the p side of workstation w to process product model k and robot operation is selected, the value is 1; otherwise, the value is 0; Z C ikwp If the product task i for processing product model k is assigned to the p-side of workstation w and human-machine collaboration is selected, the value is 1; otherwise, the value is 0;

[0017] Using the formula formula formula formula formula and formula Determine the human-machine operation constraints on the workstation's paired station; where X ikw1 To process product model k, task i is assigned to the left side of workstation w, X ikw2 To process product model k, task i is assigned to the right side of workstation w, X jkw1 To process product model k, task j is assigned to the left side of workstation w. jkw2 Product task j for processing product model k is assigned to the right side of workstation w;

[0018] Using the formula formula formula formula formula formula formula formula and formula Determine the execution order constraints of tasks; where R ij If task i and task j are assigned to the same workstation, the value is 1; otherwise, the value is 0. P(i) is the set of predecessor tasks of task i; t iks is the start time of task i of product k, t ik e is the end time of task i of product k, w i 、w j The site number assigned to tasks i and j.

[0019] Optionally, the step of establishing an integer programming model for balancing task allocation of the bilateral assembly line considering human-machine collaboration based on task data and product information of the bilateral assembly line may further include:

[0020] Different types of product information are constructed into information dictionaries respectively; the information dictionary includes: task numbers for defining task set indexes, task execution times under different operation modes, priority relationships between tasks, and task direction requirements.

[0021] Optionally, the task agent, site agent and pattern agent all include: an actor network and a critic network; the actor network and the critic network both include: a multi-layer perceptron and a long short-term memory network.

[0022] Optionally, the multi-agent system is allowed to interact with different parallel environments respectively to perform state transition and reward evaluation to obtain a trained multi-agent system, specifically including:

[0023] Using the formula formula and formula Determine the immediate reward; where r ct For cycle time bonus, CT t is the cycle time at the current moment, CT t-1 is the cycle time of the previous moment, r ec For energy consumption rewards, EC t is the energy consumption at the current moment, EC t-1 is the energy consumption at the previous moment, r td For load balancing rewards, TD t is the load difference at the current moment, TD t-1 is the load difference at the previous moment, λ is the discount factor;

[0024] Using formula R final =ω1·R CT +ω2·R EC +ω3·R LE +ω4·R HRB Determine the termination reward; where R CT For cycle time rewards, CT is the cycle time of the assembly line, t ikis the operation time of task i for processing product model k, where I is the task set, I = {1, 2, 3, ..., i}, and R EC Rewards for energy consumption, OE is the energy consumption per unit time when the robot is operating, EC is the energy consumption of the machine on the assembly line, including the robot's working energy consumption and standby energy consumption, R LE For load balancing rewards, t w is the working time of workstation w, R HRB is the load balancing rate of the robot, T H is the total operation time for workers to complete production, T R is the total operation time of the robot to complete the production, ε is a very small positive number, ω1, ω2, ω3, ω4 are all weight coefficients, satisfying ω1+ω2+ω3+ω4=1.

[0025] Optionally, the trained multi-agent system specifically includes:

[0026] Using the formula Loss q_net =|Q(S,a t )-r| 2 Determine the loss function Loss of the action value network q_net ; Among them, Q(S,a t ) is the action value of selecting task t, r is the reward at the current time, S is the global state obtained by splicing the task state and the station state, a t For output actions;

[0027] Using the formula Determine the loss function L(θ) of the actor network; where B is the number of batches, n is the time step of the trajectory, The probability ratio is clipped to the interval [1-∈1,1+∈2], is the probability ratio of the new and old strategies, is the advantage estimate of the k-th step, η is the entropy reward coefficient, For strategy in state The action distribution entropy under ,∈1 is the clipping threshold hyperparameter;

[0028] Using the formula Determine the loss function L(φ) of the critic network; where, is the current value function network state The estimated value of is the old value function network state The estimated value of R i Status The actual return of ,∈2 is the clipping threshold hyperparameter.

[0029] In a second aspect, the present application provides a human-machine collaborative task allocation device based on bilateral assembly line balancing, the human-machine collaborative task allocation device based on bilateral assembly line balancing comprising:

[0030] A model building module is used to establish an integer programming model for balanced task allocation of a bilateral assembly line taking into account human-robot collaboration based on task data of the bilateral assembly line and different types of product information. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balance rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-machine operation constraints at paired workstations, and task execution sequence constraints as constraint conditions. The task data includes: an assembly task list, the standard operation time of each task, the task priority relationship, and a comparison parameter of manual and robot operation efficiency.

[0031] A multi-agent system building module is used to build a multi-agent system based on the Actor-Critic algorithm according to different types of product information and integer programming models. The multi-agent system includes: a task agent, a site agent, and a pattern agent. The task agent uses the task state as a local observation and the task state and site state as a global state. The site agent uses the site state as a local observation and the task state and site state as a global state. The pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state.

[0032] The multi-agent system training module is used to interact with different parallel environments, perform state transitions and reward evaluation, and obtain a trained multi-agent system.

[0033] The human-machine collaborative task allocation module is used to use the trained multi-agent system to perform human-machine collaborative task allocation.

[0034] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human-machine collaborative task allocation method based on bilateral assembly line balancing.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human-machine collaborative task allocation method based on bilateral assembly line balancing.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the human-machine collaborative task allocation method based on bilateral assembly line balancing.

[0037] According to the specific embodiments provided in this application, this application has the following technical effects:

[0038] This application provides a method, device, medium and product for human-machine collaborative task allocation based on bilateral assembly line balancing. By establishing an integer programming model for bilateral assembly line balancing task allocation considering human-machine collaboration, and based on the above integer programming model and the Actor-Critic algorithm, a multi-agent system is constructed, which can be applied to production environments with multiple product models and requiring human-machine collaboration on bilateral assembly lines. In order to achieve efficient balance of the assembly line, this application combines a multi-agent reinforcement learning method, and through collaborative decision-making between agents, reasonably allocates and adjusts the task distribution of each workstation to optimize production rhythm and reduce resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 This is a flow chart of a method for allocating tasks of human-machine collaboration based on bilateral assembly line balancing in one embodiment of the present application;

[0041] Figure 2 The figure is a schematic diagram of the overall process;

[0042] Figure 3 It is a schematic diagram of the state;

[0043] Figure 4 Schematic diagram of the intelligent agent structure. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0046] In an exemplary embodiment, Figure 1 and Figure 2 As shown, a method for human-machine collaborative task allocation based on bilateral assembly line balancing is provided, which includes the following S101 to S104.

[0047] S101. Based on the task data of a bilateral assembly line with human-robot collaboration and information about different types of products, an integer programming model for balanced task allocation on a bilateral assembly line taking into account human-robot collaboration is established. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balance rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-robot operation constraints at workstation pairings, and task execution sequence constraints as constraints. Task data includes: a list of assembly tasks, standard operation time for each task, task priority relationships, and parameters comparing manual and robot operation efficiency. Task data is obtained by reviewing academic papers in related fields and using publicly available assembly line datasets. To ensure the reliability and representativeness of task data, the publicly available data is screened, processed, and standardized.

[0048] The integer programming model specifically includes:

[0049] The optimization function F is determined using the formula F = min(α·CT+β·EC-γ·LE); where CT is the cycle time of the assembly line, The aim is to minimize the cycle time of the assembly line. The cycle time is determined by the workstation with the longest working time on the assembly line. EC is the energy consumption of the machine on the assembly line, including the working energy consumption and standby energy consumption of the robot. It aims to minimize the energy consumption of machines on the assembly line. SE is the energy consumption per unit time in the standby mode of the robot, and LE is the load balancing rate of all workstations in the assembly line. Aims to maximize the load balance rate of all workstations on the assembly line, min is the minimum function, α is the weight coefficient of the cycle time of the assembly line, β is the weight coefficient of the energy consumption of the machine on the assembly line, and γ is the weight coefficient of the load balance rate of all workstations on the assembly line;

[0050] Using the formula Determine the assembly line constraints, stipulating that at the same time, the assembly line can process the same model of products; where K is the set of product models, K = {1, 2, 3, ..., k}, Y kt It means that if the assembly line processes the product of product model k at time t, it is 1, otherwise it is 0;

[0051] Using the formula and formula Determine the execution task constraints, which means that when a certain type of product is produced, the corresponding task will be executed; where t s ik is the start time of task i processing product of product model k, t e ik is the end time of task i for processing product model k, ψ is a positive number, X ikwp If product task i is assigned to the p side of workstation w for processing product model k, the value is 1; otherwise, the value is 0; p is the paired workstation; when p = 1, it means the left side; when p = 2, it means the right side;

[0052] Using the formula formula formula and formula Determine workstation allocation constraints; Formula It ensures that each task of each model will be assigned to the workstation, and will only be assigned to the station on one side of the workstation. The other two formulas stipulate that tasks that can only be assembled at the left station cannot be assigned to the right, and tasks that can only be assembled at the right station cannot be assigned to the left. The paired stations of the workstation are assigned an employee and a collaborative robot. Tasks can be selected from three operation modes: manual operation, machine operation, and human-machine collaboration; where J is the workstation set, J = {1,2,3,...,w}, I is the task set, I = {1,2,3,...,i}, A(L) is the set of tasks that must be assigned to the left station, A(L)∈I; A(R) is the set of tasks that must be assigned to the right station, A(R)∈I; A(E) is the set of tasks without direction restrictions, A(E)∈I, and satisfies I = A(L)∪A(R)∪A(E);

[0053] Using the formula Determine the operation mode constraints, limiting each task to only one operation mode; where Z H ikwp If task i is assigned to the p side of workstation w to process product model k and manual operation is selected, the value is 1; otherwise, the value is 0; Z R ikwp If task i is assigned to the p side of workstation w to process product model k and robot operation is selected, the value is 1; otherwise, the value is 0; Z C ikwp If the product task i for processing product model k is assigned to the p-side of workstation w and human-machine collaboration is selected, the value is 1; otherwise, the value is 0;

[0054] Using the formula formula formula formula formula and formula Determine the human-machine operation constraints on the pairing station of the workstation, which means that the human-machine operation on the pairing station is restricted. The first two formulas indicate that at the pairing station on the same side of the same workstation, the human and the robot cannot perform their respective tasks in parallel due to the lack of sufficient safety distance. One party must wait for the other to complete the task before proceeding to the next assembly task. The middle two formulas indicate that when the task is carried out in a human-machine collaborative manner, both parties are allowed to work simultaneously. The last two formulas indicate that when tasks are assigned to different pairing stations on the same workstation, parallel work is allowed due to sufficient safety distance. Among them, X ikw1 To process product model k, task i is assigned to the left side of workstation w, X ikw2 To process product model k, task i is assigned to the right side of workstation w, X jkw1 To process product model k, task j is assigned to the left side of workstation w. jkw2 Product task j for processing product model k is assigned to the right side of workstation w;

[0055] Using the formula formula formula formula formula formula formula formula and formula Determine the execution order constraints of tasks, which means that the execution order of tasks with priority relationships is constrained; the first four formulas indicate that in the task allocation stage, if two tasks with priority relationships are assigned to different sites, the site that executes the higher priority task should be ahead of the site that executes the lower priority task; the remaining formulas indicate that if two tasks with priority relationships are assigned to the same site, regardless of whether the two tasks are assigned to the same side, the higher priority task needs to be executed first over the lower priority task; Among them, R ij If task i and task j are assigned to the same workstation, the value is 1; otherwise, the value is 0. P(i) is the set of predecessor tasks of task i; t ik s is the start time of task i of product k, t ik e is the end time of task i of product k, w i 、w j The site number assigned to tasks i and j.

[0056] S101 and earlier also included:

[0057] Product information of multiple different models is collected and differentiated by model; different types of product information are constructed into information dictionaries; the information dictionaries include: task numbers used to define task set indexes, task execution times under different operating modes (input of the time parameter matrix in the integer programming model), priority relationships between tasks, and task direction requirements (limiting the feasibility of assigning tasks to the left / right assembly line).

[0058] S102, constructing a multi-agent system based on the Actor-Critic algorithm according to different types of product information and integer programming models; the multi-agent system includes: a task agent, a site agent, and a pattern agent; the task agent uses the task state as a local observation and the task state and site state as a global state; the site agent uses the site state as a local observation and the task state and site state as a global state; the pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state;

[0059] Among them, the task agent, site agent and pattern agent all include: actor network and critic network; the actor network and critic network both include: multi-layer perceptron (MLP) and long short-term memory network (LSTM) modules, which encode the current observation features through MLP and memorize the historical action information through LSTM. Combining the fusion of current observation features and historical information, the actor network outputs the probability distribution of the action through a linear layer, while the critic network outputs the global state value through a linear layer. According to the function of the agent, it is divided into task agent, site agent and pattern agent. The structural diagram of the agent is as follows Figure 4 shown.

[0060] Product task allocation is performed in parallel within each environment. For each type of product task allocation, allocation and action selection are performed as follows:

[0061] 1. Constructing Agent Observation: Task allocation in bilateral assembly line balancing includes two state designs: task state and station state. Task state is the basic information of each task in the product. Specifically, a one-dimensional vector is used to encode each task, including the following information: task allocation, task priority, task model, task operation direction, task operation method, and task estimated operation time. The information vectors of all tasks are combined into a two-dimensional array, which is the task state. tThe station state is the working information of each station in the assembly line. Specifically, a one-dimensional vector is used to encode each station, including the following information: the model number currently assembled by the workstation, the load distribution and energy consumption of the stations on the left and right sides of the workstation, the overall time cost and overall energy consumption of the workstation, etc. The information vectors of all stations are combined into a two-dimensional array, which is the station state O s In the initial state, since all stations are unloaded, they are represented by an array of 0. The state diagram is as follows Figure 3 shown.

[0062] 2. Change the task status to O t and site status O s As local observations of the task agent and the site agent, they are input into the actor network of the corresponding agent for action selection. Action selection follows the following method:

[0063] (1) Task selection: The task agent is used to select tasks. Before selecting tasks, all tasks are first screened to build a task pool to reduce the action space of the task agent. The specific construction method is as follows:

[0064] 1) Based on the priority relationship between tasks, extract the selectable tasks at the current time step, that is, the tasks that meet the priority order and are not assigned at the current moment. Use a one-hot encoding to represent the task number of each task. Put the task codes that meet the conditions into the candidate sequence.

[0065] 2) Combine the task state and the site state as the global state S, where S = {O t ,O s The global state S and each task in the sequence to be selected are input into the action value network respectively, and the action value of the corresponding network is output. The action value network is an MLP module, which takes the global state and task encoding as input and outputs the action value Q(S,a t ), where a t The action of selecting a task.

[0066] 3) Sort the action values ​​of the tasks in the selection sequence in descending order. Based on the task pool capacity k, retain the task vectors of the first k tasks with the highest action values ​​and place them in the task pool for selection. If the number of tasks is less than the task pool capacity, fill it with zero vectors.

[0067] After the task pool is built, the task sequence information in the task pool is combined with the task status O t Splicing is performed as the local observation O of the task agent t '. t′ is input to the actor network of the task agent, and the probability distribution of the optional tasks in the task pool is output. A task is selected through random sampling, which is the action a of the task agent. t ', and calculate the logarithm of the probability of the action.

[0068] (2) Site selection: The site agent is used to select the site. The site state is used as a local observation of the site agent and input into the actor network of the site agent. Since the execution of tasks needs to meet the priority relationship of tasks, the site assigned to the current task must be located below the site assigned to its priority task or both are at the same site. Therefore, a mask is needed to cover the sites that do not meet the conditions. After applying the mask, the actor network of the site agent outputs the probability distribution of each site, where the probability of selection of sites that do not meet the conditions is automatically set to 0 after being masked. Selecting a site through random sampling is the action a of the site agent. s , and calculate the logarithm of the probability of the action.

[0069] (3) Selecting the operation mode: The mode agent is used to select the operation mode for the current task. Its local observation is the observation of the task vector of the task selected by the task agent and the site vector of the site selected by the site agent. m ,like Figure 3 As shown, O m Input into the actor network of the pattern agent. Due to the differences in tasks, not all tasks can be completed by human-machine collaboration or robot independently. Therefore, a mask is used to cover the operation modes that do not meet the conditions. The actor network of the pattern agent outputs the probability distribution of three operation modes, among which the probability of the operation modes that do not meet the conditions is automatically set to 0 after the mask. After random sampling, one operation mode is selected, which is the action a of the pattern agent. m , and calculate the logarithm of the probability of the action.

[0070] After the three agents complete their action selection, they combine all their actions into a joint action A, where A = {a t ',a s ,a m}.

[0071] S103, interacting the multi-agent system with different parallel environments, performing state transition and reward evaluation, and obtaining a trained multi-agent system;

[0072] The task state transfer environment marks the corresponding task in the global task state array as assigned according to the selected task in the joint action; retrieves the direct successor task of the current task and increases the priority of all successor tasks by one level.

[0073] The site state transfer is when the environment updates the corresponding site information according to the selected site and the selected operation mode in the joint action, so the load and energy consumption on both sides of the site need to be recalculated;

[0074] Recalculating the load and energy consumption on both sides of the site specifically includes:

[0075] (1) Select the operation direction for the task. Some tasks have clear operation direction requirements (requiring assembly on the left or right side), while some tasks do not specify the operation direction (either left or right). For the latter, it is necessary to compare the load of the paired stations on the left and right sides of the current site, that is, the working time of both sides, and assign the task to the paired station with the smaller load; if the load of the paired stations is the same, it is assigned to the paired station on the left by default.

[0076] (2) Update the load and total energy consumption of the pairing station in the current direction. After being assigned a new task, the load of the pairing station depends on the completion time of the new task, and the start time of the new task depends on the completion time of all its predecessor tasks. First, it is necessary to traverse all tasks in the current station, execute all tasks in the current station in order, and store the end time of all predecessor tasks of the current task. The earliest start time of the new task is the maximum end time of all predecessor tasks in the station; if there is no predecessor task of the current station in the station, the earliest start time of the new task is the end time of the previous task. Since the pairing stations on the same side do not allow people and machines to perform assembly work in parallel, if the task is completed manually, the actual start time of the new task is the maximum of the end time of the task to be completed by the human and the earliest start time; if the task is completed by the machine, the actual start time of the new task is the maximum of the end time of the task to be completed by the robot and the earliest start time; if it is human-machine collaboration, the actual start time of the new task is the maximum of the three. At the same time, it is necessary to recalculate the energy consumption of the pairing station, which is the sum of the energy consumption of the robot in the pairing station when working and the energy consumption when standing by.

[0077] (3) Update the total load and total energy consumption of the station. Since the pairing stations allow parallel operation, the total load of the station is the maximum of the loads of the left and right pairing stations. The load of the pairing station depends on the maximum completion time of the people and machines in the station. At the same time, the total energy consumption of the station is the sum of the energy consumption of the left and right pairing stations.

[0078] Based on the optimization objective, an immediate reward and a terminal reward are designed. At each time step, the agent receives an immediate reward, and at the last time step, it receives both an immediate reward and a terminal reward. Since the agents are in a cooperative relationship, they share the same reward function.

[0079] Using the formula formula and formula Determine the immediate reward; where r ct is the cycle time reward, r ct After each task assignment, the change in the current assembly line cycle time is calculated and the change rate is used as the cycle time reward. The reward is normalized to avoid the impact of different time scales. t is the cycle time at the current moment, CT t-1 is the cycle time of the previous moment, r ec Energy consumption reward is calculated by calculating the change in energy consumption between the current moment and the previous moment, and taking the change rate as the energy consumption reward. This reward also needs to be normalized, EC t is the energy consumption at the current moment, EC t-1 is the energy consumption at the previous moment, r td The load balancing reward is calculated by calculating the change in the load difference between the current moment and the previous moment. The load difference is defined as the difference between the time of the site with the longest time consumption and the time of the site with the shortest time consumption. The change rate is used as the load balancing reward, TD t is the load difference at the current moment, TD t-1 is the load difference at the previous moment; the three formulas use the same discount factor λ;

[0080] Using formula R final =ω1·R CT +ω2·R EC +ω3·R LE +ω4·R HRB Determine the termination reward. The termination reward is used to provide feedback on the agent's performance after all tasks are completed, helping the agent evaluate its strategy effectiveness throughout the entire task process and further promote strategy optimization. After the last task is assigned, reward values ​​are set according to each target value; where R CT is the cycle time reward, that is, to minimize the cycle time, the cycle time of the assembly line is calculated according to the final allocation, CT is the cycle time of the assembly line, t ik is the operation time of task i for processing product model k, where I is the task set, I = {1, 2, 3, ..., i}, and R EC The energy consumption reward is to minimize the energy consumption of the collaborative robot and calculate the energy consumption of the assembly line according to the final allocation. OE is the energy consumption per unit time when the robot is operating, EC is the energy consumption of the machine on the assembly line, including the robot's working energy consumption and standby energy consumption, R LE is the load balancing reward, that is, to maximize the balance rate of the assembly line, and calculate the line efficiency of the assembly line according to the final allocation, t w is the working time of workstation w, R HRBis the load balance rate of the robot, that is, to maximize the load balance rate between the worker and the collaborative robot to avoid overloading of one of them. T H is the total operation time for workers to complete production, T R is the total operation time of the robot to complete the production, ε is a very small positive number, ω1, ω2, ω3, ω4 are all weight coefficients, satisfying ω1+ω2+ω3+ω4=1.

[0081] After interacting with the environment, new state and reward information is obtained. The experience of each time step is stored in a central experience pool. The storage content includes the global state, all local observations, joint actions, rewards, action masks, the logarithm of the probability of the corresponding action, and the termination flag. This continues until the task allocation for the current model is completed.

[0082] Collect all parallel environments and the interaction trajectories of multiple models in the environment and integrate them into a central experience pool. When the entire trajectory is completed, that is, when all current tasks are assigned, the network update phase begins.

[0083] The global terminal state value is calculated using the critic network, and the trajectory return is calculated using GAE.

[0084]

[0085]

[0086] δ t =r t +γV(s t+1 )-V(s t );

[0087] Among them, G t is the discounted return at time t, V(s t ) is state s t The value function, V(s t+1 ) is the next state s t The value function A GAE t is the generalized advantage estimate at time step t, λ is the GAE hyperparameter, T is the total length of the trajectory, l is the future time offset, δ t+l is the time series difference error at time t+l; γ is the discount factor, r t is the immediate reward at time step t;

[0088] The advantage function for each time step is calculated as follows:

[0089] A t =G t -V(s t );

[0090] Extract B experiences from the experience pool. Each experience includes the global state, all local observations, joint action, reward, action mask, logarithm of the probability of the corresponding action, and a termination flag. Extract the advantage value for the corresponding time step, use the state from the experience as input to the network to construct the computational graph, and calculate the loss function based on the network type.

[0091] Using the formula Loss q_net =|Q(S,a t )-r| 2 Determine the loss function Loss of the action value network q_net ; The action value network is used to screen tasks and update the network using gradient descent; where Q(S,a t ) is the action value of selecting task t, r is the reward at the current time, S is the global state obtained by splicing the task state and the station state, a t For output actions;

[0092] Using the formula Determine the loss function L(θ) of the actor network; the actor network is updated using the basic framework of MAPPO, and the difference between the current strategy and the old strategy is measured by the importance sampling ratio; the clipping operation is used to limit the update amplitude of the ratio, but the ratio is controlled in [1-∈1,1+∈1], which prevents the strategy update from being too drastic, thereby achieving the stability of the strategy update, and combined with the advantage function A i (k) To evaluate the quality of actions, the strategy is pushed to select actions that can improve the cumulative reward. At the same time, by maximizing the entropy of the strategy, the strategy is encouraged to maintain a certain degree of exploration, avoid overly deterministic behavior, promote exploration, and prevent falling into local optimal solutions. Where B is the number of batches of data, n is the time step of the trajectory, The probability ratio is clipped to the interval [1-∈1,1+∈1], is the probability ratio of the new and old strategies, is the advantage estimate of the k-th step, η is the entropy reward coefficient, For strategy in state The action distribution entropy under ,∈1 is the clipping threshold hyperparameter;

[0093] Using the formula Determine the loss function L(φ) of the critic network; the loss function of the critic network calculates the error through two main parts: first, calculate the current value function With target return Secondly, the clipping technique is used to constrain the predicted value of the value function to ensure that it is consistent with the old value function. The difference does not exceed the preset threshold, which is controlled at Then calculate the square error with the target return; where, is the current value function network state The estimated value of is the old value function network state The estimated value of R i Status The actual return of ,∈2 is the clipping threshold hyperparameter;

[0094] This process continues until the reward of the trajectory stabilizes or the maximum number of cycles is reached, and the weights are retained. During the application phase, due to differences in data information among various models, especially different constraints, complete model information and weights must be input, and a certain amount of training time is required to ultimately obtain a trained multi-agent system.

[0095] S104, using the trained multi-agent system to perform human-machine collaborative task allocation.

[0096] Based on the same inventive concept, the embodiment of the present application also provides a human-machine collaborative task allocation device based on bilateral assembly line balancing for implementing the human-machine collaborative task allocation method based on bilateral assembly line balancing involved above. The implementation solution provided by the device is similar to the implementation solution described in the above method. Therefore, the specific limitations in the embodiments of one or more human-machine collaborative task allocation devices based on bilateral assembly line balancing provided below can be found in the above limitations on the human-machine collaborative task allocation method based on bilateral assembly line balancing, and will not be repeated here.

[0097] In an exemplary embodiment, a human-machine collaborative task allocation device based on double-sided assembly line balancing is provided, comprising:

[0098] A model building module is used to establish an integer programming model for balanced task allocation of a bilateral assembly line taking into account human-robot collaboration based on task data of the bilateral assembly line and different types of product information. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balance rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-machine operation constraints at paired workstations, and task execution sequence constraints as constraint conditions. The task data includes: an assembly task list, the standard operation time of each task, the task priority relationship, and a comparison parameter of manual and robot operation efficiency.

[0099] A multi-agent system building module is used to build a multi-agent system based on the Actor-Critic algorithm according to different types of product information and integer programming models. The multi-agent system includes: a task agent, a site agent, and a pattern agent. The task agent uses the task state as a local observation and the task state and site state as a global state. The site agent uses the site state as a local observation and the task state and site state as a global state. The pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state.

[0100] The multi-agent system training module is used to interact with different parallel environments, perform state transitions and reward evaluation, and obtain a trained multi-agent system.

[0101] The human-machine collaborative task allocation module is used to use the trained multi-agent system to perform human-machine collaborative task allocation.

[0102] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a human-machine collaborative task allocation method based on bilateral assembly line balancing is implemented.

[0103] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0104] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0105] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0107] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0108] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0109] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0110] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A human-machine collaborative task allocation method based on bilateral assembly line balancing, characterized in that: The human-machine collaborative task allocation method based on bilateral assembly line balancing includes: Based on the task data of a bilateral assembly line with human-robot collaboration and information about different types of products, an integer programming model for balanced task allocation on a bilateral assembly line taking human-robot collaboration into account is established. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balancing rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-machine operation constraints on workstation pairings, and task execution sequence constraints as constraint conditions. The task data includes: an assembly task list, the standard operation time of each task, the task priority relationship, and parameters comparing the efficiency of manual and robot operations. Based on different types of product information and integer programming models, a multi-agent system is constructed based on the Actor-Critic algorithm. The multi-agent system includes: a task agent, a site agent, and a pattern agent. The task agent uses the task state as a local observation and the task state and site state as a global state. The site agent uses the site state as a local observation and the task state and site state as a global state. The pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state. The multi-agent system interacts with different parallel environments, performs state transitions and reward evaluations, and obtains a trained multi-agent system. Utilize the trained multi-agent system for human-machine collaborative task allocation.

2. The human-machine collaborative task allocation method based on bilateral assembly line balancing according to claim 1 is characterized in that: The integer programming model specifically includes: The optimization function F is determined using the formula F = min(α·CT+β·EC-γ·LE); where CT is the cycle time of the assembly line, EC is the energy consumption of the machines on the assembly line, including the working energy consumption and standby energy consumption of the robots, LE is the load balance rate of all workstations on the assembly line, min is the minimum function, α is the weight coefficient of the cycle time of the assembly line, β is the weight coefficient of the energy consumption of the machines on the assembly line, and γ is the weight coefficient of the load balance rate of all workstations on the assembly line; Using the formula Determine the assembly line constraints; where K is the set of product models, K = {1, 2, 3, ..., k}, Y kt It means that if the assembly line processes the product of product model k at time t, it is 1, otherwise it is 0; Using the formula and formula Determine the execution task constraints; where t s ik is the start time of task i processing product of product model k, t e ik is the end time of task i for processing product model k, ψ is a positive number, X ikwp If product task i is assigned to the p side of workstation w for processing product model k, the value is 1; otherwise, the value is 0; p is the paired workstation; when p = 1, it means the left side; when p = 2, it means the right side; Using the formula Formula X ikwp =0, ,p=2、Formula X ikwp =0, p=1 and formula X ikwp ∈{0,1}, Determine the workstation allocation constraints; where J is the workstation set, J = {1, 2, 3, ..., w}, I is the task set, I = {1, 2, 3, ..., i}, A(L) is the set of tasks that must be assigned to the left site, A(L)∈I; A(R) is the set of tasks that must be assigned to the right site, A(R)∈I; A(E) is the set of tasks without direction restrictions, A(E)∈I, and satisfies I = A(L) ∪ A(R) ∪ A(E); Using the formula Determine the operation mode constraints; where Z H ikwp If task i is assigned to the p side of workstation w to process product model k and manual operation is selected, the value is 1; otherwise, the value is 0; Z R ikwp If task i is assigned to the p side of workstation w to process product model k and robot operation is selected, the value is 1; otherwise, the value is 0; Z C ikwp If the product task i for processing product model k is assigned to the p-side of workstation w and human-machine collaboration is selected, the value is 1; otherwise, the value is 0; Using the formula w∈J,p∈{1,2},i≠j, formula w∈J,p∈{1,2},i≠j, formula formula formula and formula Determine the human-machine operation constraints on the workstation's paired station; where X ikw1 To process product model k, task i is assigned to the left side of workstation w, X ikw2 To process product model k, task i is assigned to the right side of workstation w, X jkw1 To process product model k, task j is assigned to the left side of workstation w. jkw2 Product task j for processing product model k is assigned to the right side of workstation w; Using the formula Formula w i -w j ≤ψ(1-R ij ), Formula w j -w i ≤ψ(1-R ij ), Formula w i ≤w j +ψ(1-R ij ), formula formula formula formula and formula Determine the execution order constraints of tasks; where R ij If task i and task j are assigned to the same workstation, the value is 1; otherwise, the value is 0. P(i) is the set of predecessor tasks of task i; t ik s is the start time of task i of product k, t ik e is the end time of task i of product k, w i 、w j The site number assigned to tasks i and j.

3. The method for human-machine collaborative task allocation based on bilateral assembly line balancing according to claim 1 is characterized in that: The method of establishing an integer programming model for balancing task allocation of a bilateral assembly line considering human-machine collaboration based on task data and product information of a bilateral assembly line involving human-machine collaboration also includes: Different types of product information are constructed into information dictionaries respectively; the information dictionary includes: task numbers for defining task set indexes, task execution times under different operation modes, priority relationships between tasks, and task direction requirements.

4. The method for human-machine collaborative task allocation based on bilateral assembly line balancing according to claim 1 is characterized in that: The task agent, site agent and pattern agent all include: actor network and critic network; the actor network and critic network both include: multi-layer perceptron and long short-term memory network.

5. The method for human-machine collaborative task allocation based on bilateral assembly line balancing according to claim 1 is characterized in that: The multi-agent system is interacted with different parallel environments to perform state transition and reward evaluation to obtain a trained multi-agent system, specifically including: Using the formula formula and formula Determine the immediate reward; where r ct For cycle time bonus, CT t is the cycle time at the current moment, CT t-1 is the cycle time of the previous moment, r ec Reward for energy consumption, EC t is the energy consumption at the current moment, EC t-1 is the energy consumption at the previous moment, r td For load balancing rewards, TD t is the load difference at the current moment, TD t-1 is the load difference at the previous moment, λ is the discount factor; Using formula R final =ω1·R CT +ω2·R EC +ω3·R LE +ω4·R HRB Determine the termination reward; where R CT For cycle time rewards, CT is the cycle time of the assembly line, t ik is the operation time of task i for processing product model k, where I is the task set, I = {1, 2, 3, ..., i}, and R EC Rewards for energy consumption, OE is the energy consumption per unit time when the robot is operating, EC is the energy consumption of the machine on the assembly line, including the robot's working energy consumption and standby energy consumption, R LE For load balancing rewards, t w is the working time of workstation w, R HRB is the load balancing rate of the robot, T H is the total operation time for workers to complete production, T R is the total operation time of the robot to complete the production, ε is a very small positive number, ω1, ω2, ω3, ω4 are all weight coefficients, satisfying ω1+ω2+ω3+ω4=1.

6. The method for human-machine collaborative task allocation based on bilateral assembly line balancing according to claim 1 is characterized in that: The trained multi-agent system specifically includes: Using the formula Loss q_net =|Q(S,a t )-r| 2 Determine the loss function Loss of the action value network q_net ; Among them, Q(S,a t ) is the action value of selecting task t, r is the reward at the current time, S is the global state obtained by splicing the task state and the station state, a t For output actions; Using the formula Determine the loss function L(θ) of the actor network; where B is the number of batches, n is the time step of the trajectory, The probability ratio is clipped to the interval [1-∈1,1+∈2], is the probability ratio of the new and old strategies, is the advantage estimate of the kth step, η is the entropy reward coefficient, S[π θ (o i (k) )] is the strategy in state o i (k) The action distribution entropy under ,∈1 is the clipping threshold hyperparameter; Using the formula Determine the loss function L(φ) of the critic network; where V φ (s i (k) ) is the current value function network for state s i (k) The estimated value of V φold (s i (k) ) is the old value function network for state s i (k) The estimated value of R i For state s i (k) The actual return of ,∈2 is the clipping threshold hyperparameter.

7. A human-machine collaborative task allocation device based on bilateral assembly line balancing, characterized in that: The human-machine collaborative task allocation device based on bilateral assembly line balancing includes: A model building module is used to establish an integer programming model for balanced task allocation of a bilateral assembly line taking into account human-robot collaboration based on task data of the bilateral assembly line and different types of product information. The integer programming model uses minimizing the cycle time of the assembly line, minimizing the energy consumption of the machines on the assembly line, and maximizing the load balance rate of all workstations on the assembly line as optimization functions, and uses assembly line constraints, execution task constraints, workstation allocation constraints, operation mode constraints, human-machine operation constraints at paired workstations, and task execution sequence constraints as constraint conditions. The task data includes: an assembly task list, the standard operation time of each task, the task priority relationship, and a comparison parameter of manual and robot operation efficiency. A multi-agent system building module is used to build a multi-agent system based on the Actor-Critic algorithm according to different types of product information and integer programming models. The multi-agent system includes: a task agent, a site agent, and a pattern agent. The task agent uses the task state as a local observation and the task state and site state as a global state. The site agent uses the site state as a local observation and the task state and site state as a global state. The pattern agent uses the task vector of the task selected by the task agent and the site vector of the site selected by the site agent as local observations and the task state and site state as a global state. The multi-agent system training module is used to interact with different parallel environments, perform state transitions and reward evaluation, and obtain a trained multi-agent system. The human-machine collaborative task allocation module is used to use the trained multi-agent system to perform human-machine collaborative task allocation.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human-machine collaborative task allocation method based on bilateral assembly line balancing according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the human-machine collaborative task allocation method based on bilateral assembly line balancing according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the human-machine collaborative task allocation method based on bilateral assembly line balancing according to any one of claims 1 to 6 is implemented.

Images