Intelligent team construction method and system based on deep reinforcement learning

Abstract

The invention proposes an intelligent team construction method and system based on deep reinforcement learning, and aims to construct an efficient intelligent team, and the method comprises the steps: building a Markov decision process model which comprises a state space, an action space, a transition probability and a reward function, fusing the social network information of team members through employing a graph embedding technology, and carrying out the calculation of the state space, the action space, the transition probability and the reward function; a social network, member skills and project requirements are used as input, training is performed through an adaptive actor network based on an attention mechanism, and in a team construction process, the network selects a member with the highest selection probability from candidate members in each round until a selected team can completely meet the skill requirements of a project; in practical application, related data is input into the pre-trained adaptive actor network, and the team is constructed step by step by using the beam search strategy, so that end-to-end intelligent team construction is realized, the team construction efficiency is greatly improved, and the potential synergistic effect in the social network is fully mined.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and reinforcement learning, and specifically to a method and system for building an intelligent team based on deep reinforcement learning. Background Art

[0002] In modern society, team building is widely used in many fields such as corporate management, scientific research cooperation, and project development. Its core lies in achieving task goals through member skill combination optimization and social network analysis. Efficient team building requires solving NP-hard combinatorial optimization problems, which is crucial to the quality and efficiency of tasks. Traditional team building methods are usually based on the minimum covering set theory or skill matrix matching in graph theory, relying on heuristic experience decision-making or regular expression-based rule matching. Such methods are not effective in social networks with a scale of more than 10 3 When the nodes are large, there are problems such as exponential growth of adjacency matrix storage space and incomplete modeling of collaborative costs, which can easily lead to low computational efficiency and local Pareto optimality.

[0003] Current research on the team formation problem (TFP) focuses on three computational paradigms: approximate algorithms based on greedy strategies, iterative optimization algorithms based on swarm intelligence, and exact solutions based on mathematical programming. Greedy strategies often use priority queue data structures to achieve local optimal selection, combined with dynamic weight adjustment mechanisms to balance skill coverage and communication costs. While their time complexity can be controlled to a certain extent, they are limited by the short-sightedness of Markov decision processes. Genetic algorithms, among iterative strategies, use binary encoding to represent member selection, improving solution quality through tournament selection strategies and adaptive crossover operators. However, population initialization relies on random sampling, leading to repeated computations. While tabu search-based methods can leverage some historical experience, their neighborhood operations still face complexity bottlenecks when traversing the collaborative relationship graph.

[0004] For precise solution, a mixed integer programming model uses branch-and-bound to find the global optimum. Its constraints include skill coverage, social radius threshold, and role conflict detection. To improve scalability, researchers have attempted to decompose the main problem into subproblems using column generation algorithms. However, this still requires rebuilding the constraint matrix when dealing with dynamic member availability or changes in task requirements. Recent research has begun exploring multi-objective optimization frameworks that integrate deep reinforcement learning. These approaches extract member embedding features through graph neural networks and reuse experience through Monte Carlo tree search. These hybrid approaches can handle multidimensional constraints in heterogeneous graph data while maintaining time complexity.

[0005] In recent years, deep reinforcement learning (DRL), an emerging approach for solving combinatorial optimization problems, has been increasingly applied to complex tasks such as team building. Reinforcement learning, through continuous interaction with the environment and real-time policy adjustments, is capable of handling the dynamic changes and complexity of tasks. This approach, through policy adjustments based on immediate feedback, enables the system to autonomously learn the inherent structure of the problem, demonstrating high adaptability and efficiency in the face of new challenges. Summary of the Invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a team building method based on deep reinforcement learning to improve the intelligence level of team building and optimize team collaboration efficiency.

[0007] To achieve the above objectives, in the first aspect, the present invention provides a method for building an intelligent team based on deep reinforcement learning, which describes the intelligent team building process as a Markov decision process and , member skill sets and project requirements collection The input is based on a trained adaptive actor network, and the team is gradually constructed using beam search, including: S01, calculate the probability of each member being selected; S02, at time step When the beam search is used to select the top The candidates with the highest probability are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team; at time step When , the member with the highest current probability is selected to join the corresponding candidate team, and the candidate team is dynamically updated; S03, for each candidate team, based on the newly added team members, update the dynamic skills and project requirements information of the members; at time step Selected members Afterwards, remove the member from the unselected member's skills. Update members' dynamic skill information when joining a team and meeting project skill requirements; and remove members from project requirement information. Provided skill requirements and updated project requirements information; S04, determine whether the current team meets all project requirements: If not, return to step S01 and continue selecting members until the team meets all requirements. If so, proceed to step S05; S05, final team screening, if successfully generated candidate teams, the communication costs of all candidate teams are calculated, and the team with the smallest communication cost is selected as the final intelligent team.

[0008] Furthermore, the state, action, transition probability and reward function of the Markov decision process are designed as follows: At each time step ,in, Indicates the time step when all requirements in the project are met, the state It can be expressed as:

[0009] in, is an undirected weighted membership social network, each node Represents a member, each edge Indicates members and members The direct communication relationship between them, the weight matrix Records the communication cost in the network, where Representative members With members The communication cost between them. A smaller value indicates more efficient communication, while a larger value indicates higher communication cost. Represents the time step The dynamic skill set of all members at the time, where is a member At time step The skill set you have, Represents the total number of members, at time step Selected members Afterwards, from the dynamic skill set Remove members At time step Skill sets offered , get the updated dynamic skill set ; Indicates that at time step Dynamic project demand set; Indicates that at time step Previously selected team member collection:

[0010] in, Indicates that at time step Selected members ; At each time step , according to the current state Select to perform an action ,action represents a member of the member's social network; Transition probability Indicates that in a given current state and the current action Under the condition, transfer to the next state probability; At time step The reward function is defined as:

[0011] When the time step Less than When the project requirements have not been met, the reward value is 0. equal When all the requirements of the project are met, the difference between the historical optimal team and the team communication cost predicted by the adaptive actor network is Multiply as the reward value, represents the best team found so far, represents the team generated by the Markov decision process, and the parameter Represents the sensitivity factor of the reward signal, which is used to adjust the degree of response of the reward signal to changes.

[0012] Furthermore, the probability of each member being selected is calculated by converting the member social network using a graph embedding method to generate a high-dimensional vector for each member. , and concatenate to get the static embedding vector ,in is the number of members, To embed the dimension, the member skill set and project requirements set Converted into member skill vectors respectively and the project requirement vector , concatenate the skill vectors of all members and the project requirement vectors to obtain the dynamic embedding vector ; Embed static and dynamic embedding vectors Input static encoder and dynamic encoder respectively. If the current time step ,Since the team members have not been selected, the decoder input is empty; when a team member is already selected, the static embedding vector of the currently selected team member is input into the decoder; The results of the static encoder, dynamic encoder, and decoder are input into the attention mechanism to calculate the probability of each member being selected.

[0013] Furthermore, the adaptive actor network is trained, including using the attention mechanism to select the member with the highest selection probability from the candidate members to join the team in each round of selection. After each selection, the member's skill information, project requirement information, and the currently selected team members are updated until the selected team can meet all the skill requirements of the project. The specific steps include: (1) Based on member social networks , member skill sets and project requirements collection , use graph embedding method to transform member social networks and generate high-dimensional vectors for each member , and concatenate to get the static embedding vector ,in is the number of members, is an embedding dimension. and project requirements set Converted into member skill vectors respectively and the project requirement vector , and concatenate the skill vectors of all members with the project requirement vectors to obtain the dynamic embedding vector ; (2) Input the static embedding and dynamic embedding into the static encoder and dynamic encoder respectively, and calculate the optimal intelligent team through the attention mechanism; (3) Calculate the reward value based on the current team's performance and communication costs , and update the historical optimal solution; (4) Calculate the loss function based on the reward value , and use back propagation to optimize network parameters; (5) Determine whether the maximum number of iterations has been reached: if not, return to step (2) and continue to use the adaptive actor network to build the team; if reached, execute step (6); (6) Output the optimal parameters of the adaptive actor network.

[0014] Furthermore, the historical optimal solution and reward function are as follows: Each time a new team is generated, the total communication cost of the team is calculated and compared with the historical optimal team of the current project. If the communication cost of the team generated by the adaptive actor network is lower than the historical optimal team, the historical optimal team is updated; otherwise, the original historical optimal team remains unchanged. The communication cost of a team is measured by calculating the sum of the shortest path weights between team members in the social network. Depend on members, then the communication cost The definition is as follows:

[0015] in, and Representative Team Members of Indicates members With members In the figure The sum of the edge weights on the shortest path in , if there is a reachable path between two members, then is equal to the sum of the weights on its shortest path; otherwise, set Indicates that effective communication cannot be established, and the shortest path is calculated using Dijkstra .

[0016] Back propagation is used to optimize the network parameters according to the following formula:

[0017] is the gradient of the policy network, are network parameters, is the state vector, As a benchmark, it is used to reduce the variance of the gradient estimate. represents the skill vector of all members at the initial moment, represents the project demand vector at the initial moment, Represents the action set at the initial moment; Representative Team The communication cost is For the The team corresponding to the sample, is the initial state vector of the sample, is the total number of samples in the training set; The value of is used to generate an initial solution using a greedy algorithm or other heuristics , as a reference benchmark for the optimal solution, For the The historical optimal team corresponding to each sample updates the historical optimal solution in real time by comparing the solution generated by the adaptive actor network with the historical optimal solution.

[0018] Furthermore, the attention mechanism is used to return the probability of selecting each member. Through the static encoder, the structural features of the members are extracted and obtained. , used for subsequent team selection and remains unchanged during training; the static encoder is implemented using a one-dimensional convolutional network, which uses the vector With each member's skill vector The way to perform splicing, specifically, at the time step ,vector Calculated by the following formula:

[0019] in, are the parameters of the convolutional network, Represents the project at time step Dynamic skill requirements when Represents the time step member The dynamic skills, at time step When using the matrix represents the output of the dynamic encoder, where is the encoding vector of each member. In the embodiment of the present invention, it is assumed that at time step When selecting a member , the input vector of the dynamic module and The selected members will be updated accordingly and the matrix will be generated by the dynamic encoder ,The dynamic encoder is implemented using a one-dimensional convolutional network; For the decoder, at each time step A member index is generated at time step The action set is }, the decoder input is represented as The decoder module is also a convolutional network used to transform the selected members The embedding vector is converted into a new vector, denoted as , which is calculated as follows:

[0020] represents the parameters of the convolutional network, is in the time step action The static embedding vector of Denotes the decoder at time step Output.

[0021]

[0022]

[0023] Furthermore, the rules for updating the dynamic skills and project requirements of team members are as follows: Time step , the policy network selects members , and based on the index, the corresponding member is determined to join the team, and then at time step , the project's skill requirements set Based on the selected members Update the skills you have, remove the skills you have already met, and satisfy the following relationship:

[0024] express Current Member The skill set you possess, when selecting members After that, the skill sets of the remaining members that were not selected are adjusted accordingly. Specifically, at the time step Select members Afterwards, for the unselected members , whose skill set Update as follows: .

[0025] In a second aspect, the present invention provides an intelligent team building system based on deep reinforcement learning, comprising a calculation module, an update module, a check module, and a selection module; The calculation module is used to calculate the probability of each member being selected; The update module is used to update the When the beam search is used to select the top The candidates with the highest probability are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team; at time step When , the member with the highest current probability is selected to join the corresponding candidate team, and the candidate team is dynamically updated; For each candidate team, based on the newly added team members, update the dynamic skills and project requirements information of its team members; at time step Selected members Afterwards, from the unselected member skills Remove members Join the team and meet the project skill requirements, get ; and from the project requirements information Remove members Provided skills requirements, ; The check module is used to determine whether the current team meets all project requirements: If not, continue to select members until the team meets all requirements. If so, make the selection; The selection module is used for final team screening. If it is successfully generated candidate teams, the communication costs of all candidate teams are calculated, and the team with the smallest communication cost is selected as the final intelligent team.

[0026] In a third aspect, the present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store computer executable programs, and the processor reads part or all of the computer executable programs from the memory and executes them. When the processor executes part or all of the computer executable programs, the above-mentioned intelligent team building method based on deep reinforcement learning can be implemented.

[0027] A computer-readable storage medium may also be provided, in which a computer program is stored. When the computer program is executed by a processor, the method for building an intelligent team based on deep reinforcement learning can be implemented.

[0028] Compared with the prior art, the present invention has at least the following beneficial effects: This method integrates member social network information to mine potential information from their social relationship networks. Through an end-to-end approach, this information, along with their skills and project information, is fed into an adaptive actor network to directly derive the probability of selecting each member. This probability is then used to select members for the team. This method also proposes an adaptive critic network, incorporating an attention mechanism, to dynamically evaluate team selection results in real time, continuously optimizing the strategy through adaptive adjustments and iterative updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of an embodiment of the present invention.

[0030] Figure 2 This is the network structure diagram of the present invention.

[0031] Figure 3 Using flowcharts for deep reinforcement learning. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are only used to illustrate the present invention and are not used to limit the scope of protection of the present invention.

[0033] Example 1 like Figure 1 and Figure 3 As shown, the method for constructing an intelligent team based on deep reinforcement learning in this embodiment includes the following steps: S1: Construct a Markov decision process, including defining states, actions, transition probabilities, and rewards to describe the decision-making process in different environments.

[0034] The Markov decision process described in step S1 includes state space, action space, reward function and transition probability. Figure 2 , the design of state, action, transition probability and reward function is as follows: (1) State space

[0035] At each time step ,in, Indicates the time step when all requirements in the project are met, the state It can be expressed as:

[0036] in: is an undirected weighted membership social network, each node Represents a member, each edge Indicates members and members The direct communication relationship between them. Weight matrix Records the communication cost in the network, where Representative members With members The communication cost between them. The smaller the value, the more efficient the communication, and the larger the value, the higher the communication cost.

[0037] Represents the time step The dynamic skill set of all members at the time, where is a member At time step The skill set you have, Indicates the total number of members. Selected members Afterwards, from the dynamic skill set Remove members At time step Skill sets offered , get the updated dynamic skill set Specifically, for the unselected members (i.e. members of ), whose skill sets Update as follows:

[0038] Indicates that at time step The dynamic requirements set of the project. Selected members Afterwards, from the project requirements set at the previous moment Remove members Provided skills requirements, thus obtaining updated , this process satisfies the following relationship:

[0039] Indicates that at time step Previously selected team member collection:

[0040] in, Indicates that at time step Selected members .

[0041] (2) Action Space

[0042] At each time step , according to the current state Select to perform an action .action Represents a member in a member's social network.

[0043] (3) Transition probability Transition probability Indicates that in a given current state and the current action Under the condition, transfer to the next state probability.

[0044] (4) Reward Function At time step The reward function can be defined as:

[0045] When the time step Less than When the project requirements have not been met, the reward value is 0. equal When all the requirements of the project are met, the difference between the historical optimal team and the team communication cost predicted by the adaptive actor network is Multiplied as the reward value. In the formula, represents the best team found so far, represents the team generated by this Markov decision process. Represents the sensitivity factor of the reward signal, which is used to adjust the degree of response of the reward signal to changes.

[0046] S2: Input data, including: input member social networks, member skills and project requirements.

[0047] Furthermore, step S2 includes inputting member social networks, member skills, and project requirements, as follows: (1) Member skills represents the skill set of all members at the initial moment (time step 0). Indicates members At time step Possess a set of skills.

[0048] (2) Project requirements Represents the set of skills required at the initial moment, and every skill must be met in the project.

[0049] (3) Member social network is an undirected weighted member social network, where Represents a member set, Represents the connection relationship between members. Each node in the graph represents a member, and each edge It means members and members The direct communication relationship between them. Weight matrix Records the communication cost in the network, where Representative members With members The communication cost between them. The smaller the value, the more efficient the communication, and the larger the value, the higher the communication cost.

[0050] S3: Training the Adaptive Actor Network. This involves using an attention mechanism to select the member with the highest probability of selection from the candidate pool in each round of selection. After each selection, the member's skill information, project requirements, and currently selected team members are updated until the selected team meets all the project's skill requirements.

[0051] Furthermore, the specific steps of step S3 to train the adaptive actor network are as follows: S31, input training data, including member social networks , member skill sets and project requirements collection . Use graph embedding method to transform member social networks and generate high-dimensional vectors for each member , and concatenate to get the static embedding vector ,in is the number of members, is an embedding dimension. and project requirements set Converted into member skill vectors respectively and the project requirement vector . And concatenate the skill vectors of all members with the project requirement vectors to obtain the dynamic embedding vector .

[0052] Further, in S31, the specific details are as follows: vector Each component of is used to indicate the demand status of the corresponding skill. Specifically, when the value of a component is 1, it means that the skill is a required skill; when the value is 0, it means that the skill is not a required skill. The vector Each component is used to indicate the member At time step Skills possessed. When the value of a component is 1, it means that the member Possess this skill; when the value is 0, it means that the member Does not have this skill.

[0053] S32, computing an intelligent team using an adaptive actor network, includes the following steps: inputting a static embedding vector and a dynamic embedding vector into a static encoder and a dynamic encoder, respectively, and computing an optimal intelligent team using an attention mechanism. It is worth noting that the team selection method during training remains consistent with the actual application selection method, so it will not be detailed here.

[0054] S33, calculate the reward value and update the historical optimal solution. Including: calculate the reward value based on the current team's performance and communication costs , and update the historical optimal solution.

[0055] Furthermore, the historical optimal solution and reward function described in step S33 are as follows: Each time a new team is generated, the total communication cost of the team is calculated and compared with the historical optimal team of the current project: If the communication cost of the team generated by the adaptive actor network is lower than the historical optimal team, the historical optimal team is updated; otherwise, the original historical optimal team remains unchanged.

[0056] The communication cost of a team is measured by calculating the sum of the shortest path weights between team members in their social networks. Depend on members, then the communication cost The definition is as follows:

[0057] in, and Representative Team Members of Indicates members With members In the figure The sum of the edge weights on the shortest path in . If there is a reachable path between two members, then is equal to the sum of the weights on its shortest path; otherwise, set Indicates that effective communication cannot be established. The shortest path is calculated using Dijkstra .

[0058] S34, calculate the loss function And update the network parameters. This includes: calculating the loss function based on the reward value and optimizing the network parameters using backpropagation.

[0059] Furthermore, step S34 calculates the loss function gradient as follows: in, represents the gradient of the policy network, are network parameters. The state vector is defined as ,in, As a benchmark ( baseline ), which is used to reduce the variance of gradient estimation and improve training stability. Represents the skill vector of all members at the initial moment. Represents the project demand vector at the initial moment. Represents the action set at the initial moment.

[0060]

[0061] Monte Carlo sampling is widely used in policy gradient methods in reinforcement learning. In this method, Representative Team The communication cost is For the The team corresponding to the sample, is the initial state vector of the sample, is the total number of samples in the training set.

[0062] The value of is usually estimated by a critic network. However, due to the discrete nature of the solution space, it is difficult to calculate Therefore, this method uses greedy or other heuristics to generate the initial solution. , as a reference benchmark for the optimal solution. For the The policy network compares the solutions generated by the adaptive actor network with the historical best solutions and updates them in real time, continuously improving the decision-making capabilities of the adaptive actor network during training. By fully leveraging existing knowledge during training and using the current best solution as a reference, the policy network can effectively learn the structural characteristics of the problem, generating more efficient and accurate solutions without relying on the critic network.

[0063] S35, determine whether the maximum number of iterations has been reached: if not, return to step S32 and continue to use the adaptive actor network to build the team; if reached, execute step S36.

[0064] S36 outputs the optimal parameters of the adaptive actor network. These parameters can be used in subsequent team building tasks to achieve intelligent team recommendation.

[0065] S4: Generate intelligent teams using adaptive actor networks. This involves inputting project requirements and member skills into a trained adaptive actor network and gradually building a team using beam search.

[0066] Furthermore, the specific steps of using the adaptive actor network in step S4 are as follows: S41, input data. Including member social network , member skill sets and project requirements collection . Use graph embedding method to transform member social networks and generate high-dimensional vectors for each member , and concatenate to get the static embedding vector ,in is the number of members, is an embedding dimension. and project requirements set Converted into member skill vectors respectively and the project requirement vector . And concatenate the skill vectors of all members with the project requirement vectors to obtain the dynamic embedding vector .

[0067] S42, embed static and dynamic embedding Input static encoder and dynamic encoder respectively. If the current time step , since no team members have been selected yet, the decoder input is empty; when a team member already exists, the static embedding of the currently selected team member is used. Enter the decoder.

[0068] S43, input the results of the static encoder, dynamic encoder and decoder into the attention mechanism to calculate the probability of each member being selected. Among them, the output of the static encoder is , the output of the dynamic encoder is , the output of the decoder is .

[0069] At time step When using beam search to select the top The candidates with the highest probability (determined by the beam width parameter) are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team.

[0070] At time step When , the member with the highest current probability is selected to join the corresponding candidate team to dynamically update the candidate team.

[0071] S44, for each candidate team, based on the newly added team members, update the dynamic skills and project requirement information of its team members to reflect the changes in the current status of the team.

[0072] S45, team requirements check, to determine whether the current team meets all project requirements: If not, return to step S42 and continue selecting members until the team meets all requirements. If so, proceed to step S46.

[0073] S46, final team screening, if successfully generated candidate teams, calculate the communication costs of all candidate teams and select the team with the lowest communication cost as the final intelligent team. , then return to step S42 and continue to generate new candidate teams until the required number is met.

[0074] Further, in step S41, the specific details are as follows: vector Each component of is used to indicate the demand status of the corresponding skill. Specifically, when the value of a component is 1, it means that the skill is a required skill; when the value is 0, it means that the skill is not a required skill. The vector Each component is used to indicate the member At time step Skills possessed. When the value of a component is 1, it means that the member Possess this skill; when the value is 0, it means that the member Does not have this skill.

[0075] Furthermore, step S42 obtains the outputs of the static encoder, dynamic encoder, and decoder. The specific steps are as follows: Afterwards, the attention mechanism is used to return the probability of selecting each member. Through the static encoder, the structural features of the members are extracted and obtained. , used for subsequent team selection and remains unchanged during the training process. In the embodiment of the present invention, the static encoder is implemented using a one-dimensional convolutional network. In order to ensure that the dynamic skill requirements of the project can be effectively integrated into the strategy network, the vector With each member's skill vector Specifically, at the time step ,vector Calculated by the following formula:

[0076] in, are the parameters of the convolutional network, Represents the project at time step Dynamic skill requirements when Represents the time step member Dynamic skills. At time step When using the matrix To represent the output of the dynamic encoder, where is the encoding vector of each member. In the embodiment of the present invention, it is assumed that at time step When selecting a member (Right now ), then the input vector of the dynamic module and The selected members will be updated accordingly and the matrix will be generated by the dynamic encoder In an embodiment of the present invention, the dynamic encoder is implemented using a one-dimensional convolutional network.

[0077] For the decoder, at each time step A member index is generated. The action set is }, which is the index of the action set that is a member, so the input to the decoder can be expressed as Specifically, the decoder module is also a convolutional network, whose function is to transform the selected members The embedding vector is converted into a new vector, denoted as , which is calculated as follows:

[0078] represents the parameters of the convolutional network, is in the time step action Static embedding vector of . Using Denotes the decoder at time step Output.

[0079] Furthermore, in step S43, the attention mechanism is used to calculate the probability of selecting the next member. The specific expression is as follows:

[0080] in, and are the trained parameters, is a vector Extension to match dimension. The method to obtain is as follows, the vector yes 、 and , and then apply a nonlinear transformation. and are the trained parameters.

[0081]

[0082] Furthermore, the update rules described in step S44 are as follows: Time step , the policy network selects members , and based on the index, the corresponding member is determined to join the team. , the project's skill requirements set Based on the selected members Update the skills you have, remove the skills you have already met, and satisfy the following relationship:

[0083] express Current Member Skill set possessed. After that, the skill sets of the remaining members that were not selected are adjusted accordingly. Specifically, at time step Select members Afterwards, for the unselected members (i.e. members of ), whose skill sets Update as follows:

[0084] In order to verify the effectiveness of the method of the present invention, the method of Example 1 of the present invention was used for comparison with GA, Greedy and A1. Among them, the genetic algorithm (GA) is an optimization strategy based on natural selection and genetic mechanisms. It simulates the biological evolution process and uses operations such as crossover, mutation and selection to continuously optimize the quality of the solution, thereby finding a better solution. Greedy is greedy. Every time a team member is selected, it will give priority to members who can meet the project skill requirements to the greatest extent and join the team in turn. A1 adopts the traditional actor-critic, but does not update member skill information and project requirement information. A1 and the present invention use beam search, and the numbers in brackets represent the beam width parameters, while in GA, the numbers in brackets represent the number of generations.

[0085] In the experiment, a member social network was constructed based on the DBLP dataset. First, keywords were extracted from paper abstracts as member skills. Co-authors of co-authored papers were considered to have a collaborative relationship, forming an edge in the member social network. After all collaborative relationships were integrated, the edge weight in the network was set to the inverse of the number of collaborations.

[0086] To accommodate problems of varying sizes, subgraphs containing 10, 20, 30, 40, 50, and 100 members were selected from the DBLP dataset. During the initialization phase, a greedy strategy was used to initialize the historical optimal solution. To evaluate the performance of different methods, 1,000 projects were generated. The number of skills required for each project followed a Poisson distribution. A portion of the skill matrix was randomly selected using a uniform distribution and labeled to represent the skill requirements. The teams generated by each method were compared and their average communication costs were analyzed. Lower communication costs indicate higher communication efficiency. The experimental results are shown in Table 1, where "size" represents the number of members in the member's social network.

[0087] Table 1 Performance comparison of different methods on real datasets

[0088] Experimental results show that our method effectively reduces the average communication cost of teams in social networks of varying sizes. Compared to the GA, Greedy, and A1 methods, our method achieves the lowest communication cost in most cases. This cost advantage is particularly pronounced at larger scales (50 and 100 members), demonstrating strong team optimization capabilities and applicability.

[0089] In Example 2, the present invention provides an intelligent team building system based on deep reinforcement learning, comprising a calculation module, an update module, a check module, and a selection module; The calculation module is used to calculate the probability of each member being selected; The update module is used to update the When the beam search is used to select the top The candidates with the highest probability are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team; at time step When , the member with the highest current probability is selected to join the corresponding candidate team, and the candidate team is dynamically updated; For each candidate team, based on the newly added team members, update the dynamic skills and project requirements information of its team members; at time step Selected members Afterwards, from the unselected member skills Remove members Join the team and meet the project skill requirements, get ; and from the project requirements information Remove members Provided skills requirements, ; The check module is used to determine whether the current team meets all project requirements: If not, continue to select members until the team meets all requirements. If so, make the selection; The selection module is used for final team screening. If it is successfully generated candidate teams, the communication costs of all candidate teams are calculated, and the team with the smallest communication cost is selected as the final intelligent team.

[0090] On the other hand, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the intelligent team building method based on deep reinforcement learning described in the present invention.

[0091] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the intelligent team building method based on deep reinforcement learning described in the present invention.

[0092] The computer device may be a laptop computer, a desktop computer or a workstation.

[0093] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf field-programmable gate array (FPGA).

[0094] The memory of the present invention may be an internal storage unit of a laptop computer, desktop computer or workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0095] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSD) or optical disks, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).

[0096] The above content is only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. An intelligent team building method based on deep reinforcement learning, characterized by: The intelligent team building process is described as a Markov decision process, where the members' social network , member skill sets and project requirements collection The input is based on a trained adaptive actor network and uses beam search to build a team, including: S01, calculate the probability of each member being selected; S02, at time step When the beam search is used to select the top The candidates with the highest probability are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team; at time step When , the member with the highest current probability is selected to join the corresponding candidate team, and the candidate team is dynamically updated; S03, for each candidate team, update the dynamic skills and project requirements information of the members based on the newly added team members; S04: If the current team does not meet all project requirements, return to step S01 and continue selecting members until the team meets all requirements. If the current team meets all project requirements, proceed to step S05; S05, final team screening, if successfully generated candidate teams, the communication costs of all candidate teams are calculated, and the team with the smallest communication cost is selected as the final intelligent team.

2. The method for building an intelligent team based on deep reinforcement learning according to claim 1, characterized in that: The design of the state, action, transition probability and reward function of the Markov decision process is as follows: At each time step ,in, Indicates the time step when all requirements in the project are met, the state It can be expressed as: in, is an undirected weighted membership social network, each node Represents a member, each edge Indicates members and members The direct communication relationship between them, the weight matrix Records the communication cost in the network, where Representative members With members The communication cost between them. A smaller value indicates more efficient communication, while a larger value indicates higher communication cost. Represents the time step The dynamic skill set of all members at the time, where is a member At time step The skill set you have, Represents the total number of members, at time step Selected members Afterwards, from the dynamic skill set Remove members At time step Skill sets offered , get the updated dynamic skill set ; Indicates that at time step Dynamic project demand set; Indicates that at time step Previously selected team member collection: in, Indicates that at time step Selected members ; At each time step , according to the current state Select to perform an action ,action represents a member of the member's social network; Transition probability Indicates that in a given current state and the current action Under the condition, transfer to the next state probability; At time step The reward function is defined as: When the time step Less than When the project requirements have not been met, the reward value is 0. equal When all the requirements of the project are met, the difference between the historical optimal team and the team communication cost predicted by the adaptive actor network is Multiply as the reward value, represents the best team found so far, represents the team generated by the Markov decision process, and the parameter Represents the sensitivity factor of the reward signal, which is used to adjust the degree of response of the reward signal to changes.

3. The method for building an intelligent team based on deep reinforcement learning according to claim 1, characterized in that: Calculating the probability of each member being selected includes: using graph embedding methods to transform the member social network and generate a high-dimensional vector for each member , and concatenate to get the static embedding vector ,in is the number of members, To embed the dimension, the member skill set and project requirements set Converted into member skill vectors respectively and the project demand vector , concatenate all members’ skill vectors and project requirement vectors to obtain dynamic embedding vectors ; Embed static and dynamic embedding vectors Input static encoder and dynamic encoder respectively. If the current time step ,Since the team members have not been selected, the decoder input is empty; when a team member is already selected, the static embedding vector of the currently selected team member is input into the decoder; The results of the static encoder, dynamic encoder, and decoder are input into the attention mechanism to calculate the probability of each member being selected.

4. The method for building an intelligent team based on deep reinforcement learning according to claim 1, characterized in that: Training the adaptive actor network involves using an attention mechanism to select the member with the highest probability of selection from the candidate pool in each round of selection. The network also updates the member's skill information, project requirements, and currently selected team members after each selection, until the selected team meets all the project's skill requirements. This includes the following steps: (1) Based on member social networks , member skill sets and project requirements collection , use graph embedding method to transform member social networks and generate high-dimensional vectors for each member , and concatenate to get the static embedding vector ,in is the number of members, To embed the dimension, the member skill set and project requirements set Converted into member skill vectors respectively and the project demand vector , and concatenate the skill vectors of all members with the project requirement vectors to obtain the dynamic embedding vector ; (2) Input the static embedding and dynamic embedding into the static encoder and dynamic encoder respectively, and calculate the optimal intelligent team through the attention mechanism; (3) Calculate the reward value based on the current team's performance and communication costs , and update the historical optimal solution; (4) Calculate the loss function based on the reward value , and use back propagation to optimize network parameters; (5) Determine whether the maximum number of iterations has been reached: if not, return to step (2) and continue to use the adaptive actor network to build the team; if reached, execute step (6); (6) Output the optimal parameters of the adaptive actor network.

5. The method for building an intelligent team based on deep reinforcement learning according to claim 4, characterized in that: The historical optimal solution and reward function are as follows: Each time a new team is generated, the total communication cost of the team is calculated and compared with the historical optimal team of the current project. If the communication cost of the team generated by the adaptive actor network is lower than the historical optimal team, the historical optimal team is updated; otherwise, the original historical optimal team remains unchanged. The communication cost of a team is measured by calculating the sum of the shortest path weights between team members in the social network. Depend on members, then the communication cost The definition is as follows: in, and Representative Team Members of Indicates members With members In the figure The sum of the edge weights on the shortest path in , if there is a reachable path between two members, then is equal to the sum of the weights on its shortest path; otherwise, set Indicates that effective communication cannot be established, and the shortest path is calculated using Dijkstra ; Back propagation is used to optimize the network parameters according to the following formula: is the gradient of the policy network, are network parameters, is the state vector, As a benchmark, it is used to reduce the variance of the gradient estimate. represents the skill vector of all members at the initial moment, represents the project demand vector at the initial moment, Represents the action set at the initial moment; Representative Team The communication cost is For the The team corresponding to the sample, is the initial state vector of the sample, is the total number of samples in the training set; The value of is used to generate an initial solution using a greedy algorithm or other heuristics , as a reference benchmark for the optimal solution, For the The historical optimal team corresponding to each sample updates the historical optimal solution in real time by comparing the solution generated by the adaptive actor network with the historical optimal solution.

6. The method for building an intelligent team based on deep reinforcement learning according to claim 1, characterized in that: Use the attention mechanism to return the probability of selecting each member. Through the static encoder, the structural features of the members are extracted and obtained. , used for subsequent team selection and remains unchanged during training; the static encoder is implemented using a one-dimensional convolutional network, which uses the vector With each member's skill vector The way to perform splicing, specifically, at the time step ,vector Calculated by the following formula: in, are the parameters of the convolutional network, Represents the project at time step Dynamic skill requirements when Represents the time step member The dynamic skills, at time step When using the matrix represents the output of the dynamic encoder, where is the encoding vector of each member. In the embodiment of the present invention, it is assumed that at time step When selecting a member , the input vector of the dynamic module and The selected members will be updated accordingly and the matrix will be generated by the dynamic encoder ,The dynamic encoder is implemented using a one-dimensional convolutional network; For the decoder, at each time step A member index is generated at time step The action set is }, the decoder input is represented as The decoder module is also a convolutional network used to transform the selected members The embedding vector is converted into a new vector, denoted as , which is calculated as follows: represents the parameters of the convolutional network, is in the time step action The static embedding vector of Denotes the decoder at time step The output, 。 7. The method for building an intelligent team based on deep reinforcement learning according to claim 6, characterized in that: The rules for updating the dynamic skills and project requirements information of its team members are as follows: Time step , the policy network selects members , and based on the index, the corresponding member is determined to join the team, and then at time step , the project's skill requirements set Based on the selected members Update the skills you have, remove the skills you have already met, and satisfy the following relationship: express Current Member The skill set you possess, when selecting members After that, the skill sets of the remaining members that were not selected are adjusted accordingly. Specifically, at the time step Select members Afterwards, for the unselected members , whose skill set Update as follows: 。 8. An intelligent team building system based on deep reinforcement learning, characterized by: Including calculation module, update module, check module and selection module; The calculation module is used to calculate the probability of each member being selected; The update module is used to update the When the beam search is used to select the top The candidates with the highest probability are selected and added to different candidate teams respectively, and independent project requirements and member skills are initialized for each candidate team; at time step When , the member with the highest current probability is selected to join the corresponding candidate team, and the candidate team is dynamically updated; For each candidate team, based on the newly added team members, update the dynamic skills and project requirements information of its team members; at time step Selected members Afterwards, from the unselected member skills Remove members Join the team and meet the project skill requirements, get ; and from the project requirements information Remove members Provided skills requirements, ; The check module is used to determine whether the current team meets all project requirements: If not, continue to select members until the team meets all requirements. If so, make the selection; The selection module is used for final team screening. If it is successfully generated candidate teams, the communication costs of all candidate teams are calculated, and the team with the smallest communication cost is selected as the final intelligent team.

9. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store computer executable programs, the processor reads part or all of the computer executable programs from the memory and executes them, and when the processor executes part or all of the computer executable programs, it can implement the intelligent team building method based on deep reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the intelligent team building method based on deep reinforcement learning described in any one of claims 1 to 7.

Images