A human resource management permission control system and method based on reinforcement learning

Abstract

This invention relates to the field of human resource management technology and discloses a human resource management access control system and method based on reinforcement learning, comprising the following steps: acquiring business authorization information and task information; identifying permission overlap conflicts and generating conflict states; establishing isolated permission business copies for rehearsing candidate access schemes; generating adjustment actions for each access source based on MATD3 and forming joint candidate schemes; executing single-source schemes and joint candidate schemes to generate combined difference data; executing joint candidate schemes and reversing rehearsals to generate forward execution trajectories and recovery execution trajectories; updating the dual Critic network based on the two types of trajectories and delaying the Actor network based on the combined difference data; outputting the target access scheme, submitting for approval, and recording the execution results. This invention achieves accurate identification and secure rehearsal of access conflicts, improving the reliability and traceability of access schemes.

Description

Technical Field

[0001] This invention relates to the field of human resource management technology, and in particular to a human resource management access control system and method based on reinforcement learning. Background Technology

[0002] Existing human resource management systems typically configure user permissions based on job position, department, and business task, and complete the granting and revoke of permissions through role-based access control, permission matrices, and manual approval. When a user simultaneously assumes multiple job responsibilities or business tasks, the system generally merges the authorization scope corresponding to different permission sources to determine the final access permissions.

[0003] Existing technologies primarily control permissions from a single source, making it difficult to identify implicit permission conflicts arising from the overlap of multiple permission sources. While individual permissions may meet business requirements when used individually, their combination can result in data access capabilities exceeding the needs of the current task. Using permission union can easily lead to an expansion of the permission scope, while using permission intersection may cause the target business process to malfunction.

[0004] Existing permission schemes typically need to be implemented in the formal business environment before they can be verified. This can easily lead to problems such as insufficient permissions, unauthorized access, or incomplete permission revocation during actual execution. Furthermore, it is difficult to distinguish the specific impact of different permission sources on the conflict outcome, and there is a lack of means to evaluate the execution process and revocation / recovery process of candidate permission schemes separately.

[0005] Therefore, how to provide a human resource management access control system and method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a human resource management permission control system and method based on reinforcement learning. This invention performs forward pre-playing and reversal re-playing of candidate permission schemes through permission business copies, and uses a multi-actor network to process different permission sources respectively. It has the advantages of accurate permission conflict identification, secure scheme verification, traceable permission adjustment and strong business adaptability.

[0007] A human resource management access control method based on reinforcement learning according to an embodiment of the present invention includes the following steps: Obtain human resources business authorization information and task information, identify permission conflicts caused by overlapping permission sources, and generate conflict status; Based on the business environment, establish isolated permission business copies, write conflict states into the permission business copies, and form an execution rehearsal environment for candidate permission schemes; Based on MATD3, several Actor networks and a centralized dual Critic network are established. Each Actor network generates adjustment actions according to the source of permissions, and they are merged to form a joint candidate solution. The adjustment actions are each formed into a single-source solution. The same business process is executed on the single-source solution and the joint candidate solution within the permission business copy. Based on the execution differences, combined difference data is generated and fed back to the corresponding Actor network. Execute the joint candidate solution within the authorized service copy to form a positive execution trajectory; revoke the joint candidate solution and restore the authorized service copy; perform the revocation replay to form a recovery execution trajectory. The positive execution trajectory is input into the first Critic network, and the recovered execution trajectory is input into the second Critic network. The centralized dual Critic network is updated based on the two evaluation results, and several Actor networks are updated according to the delay period based on the combined difference data. Based on the updated target permission scheme output by several Actor networks, the target permission scheme is submitted to the permission approval process and the execution result is recorded.

[0008] Optionally, obtaining human resources business authorization information and task information specifically includes: Obtain human resources business authorization information and task information, and break down the human resources business authorization information according to the source identifier of the permission to form the authorization scope corresponding to each source of permission; Extract the scope of task permissions required to execute the target business process based on the task information; The authorization scopes corresponding to each permission source are superimposed to generate a combined permission scope; The combined permission scope is compared with the task permission scope item by item, and the permission items in the combined permission scope that exceed the task permission scope are extracted. Determine the source of permissions to be overlaid based on the permission items that exceed the scope of the task permissions, and establish the association between the permission items that exceed the scope of the task permissions and the permission sources. Convert the combined permission scope, task permission scope, permission items exceeding the task permission scope, and their associated relationships into a conflict state, and input the conflict state into the permission business copy.

[0009] Optionally, the establishment of isolated permission service copies based on the business environment specifically includes: Based on the conflict status, locate the task data and authorization configuration corresponding to the target business process, copy the task data and authorization configuration to an independent replica space, and establish a data identification mapping between the business environment and the independent replica space. Based on the data identifier mapping, the access requests generated by the target business process are redirected to an independent replica space. The conflict state is loaded in the independent replica space and the permission source superposition state is restored to block the access requests from reading and writing data in the business environment. Record the initial replica state of the independent replica space before loading the candidate permission scheme, and limit the data changes generated by the execution of the candidate permission scheme to the independent replica space; After completing the execution rehearsal, the task data and authorization configuration are restored based on the initial copy state to form a permission business copy.

[0010] Optionally, the establishment of several Actor networks and a centralized dual-Critic network based on MATD3 specifically includes: Extract the source of permissions involved in the permission stacking from the conflict state, establish Actor network instances according to the source of permissions, and separate the source conflict state associated with each source of permissions from the conflict state. Input the conflict status of each source into the corresponding Actor network instance to generate an adjustment action carrying the permission source identifier and permission item identifier; Multiple adjustment actions are merged according to the permission item identifier. During the merging process, the correspondence between the permission source identifier and the adjustment action is preserved to form a joint candidate solution. The joint candidate solution is then input into a centralized dual-Critic network.

[0011] Optionally, the step of executing the same business process for single-source solutions and joint candidate solutions within the authorized service copy specifically includes: Extract the corresponding adjustment actions from the joint candidate scheme according to the permission source identifier, write the adjustment actions associated with the same permission source identifier into the authorization configuration in the initial copy state, keep the authorization configuration corresponding to other permission sources unchanged, and form a single source scheme; Each single-source solution is loaded into the permission business copy in sequence. An access request is initiated from the starting node of the target business process. The execution is completed in the order of the nodes of the target business process and the single-source execution trajectory is recorded. After each single-source solution is completed, the permission business copy is restored according to the initial copy state. After all single-source solutions have been executed, the combined candidate solution is loaded into the restored permission business copy, and the target business process is executed again in the same node order, recording the combined execution trajectory.

[0012] Optionally, the step of generating combined difference data based on execution differences and feeding it back to the corresponding Actor network specifically includes: Align the joint execution trajectory with the individual execution trajectories according to the node order of the target business process, and extract the trajectory segments where the permission verification results change under the same business node; Based on the permission item identifier corresponding to the trajectory segment, the corresponding adjustment action and permission source identifier are determined from the joint candidate scheme, and the trajectory segment, adjustment action and permission source identifier are associated to generate combined difference data; According to the source of permission, the combined difference data is written into the training sample area of ​​the corresponding Actor network instance. When the delayed update period is reached, the combined difference data in the training sample area is read, backpropagation is performed on the corresponding Actor network instance, and the network parameters are updated.

[0013] Optionally, the formation of a positive execution trajectory and the formation of a recovery execution trajectory specifically include: Write the joint candidate solution into the authorization configuration of the permission business copy, execute the access request according to the target business process, and record the permission verification results and execution status of each business node in sequence to form a positive execution trajectory; Terminate the access session corresponding to the forward execution, undo the adjustment actions written to the federated candidate scheme, and restore the permission service copy according to the initial copy status. The target business process is re-executed using the same task information as the forward execution, and the permission verification results and execution status of each business node are recorded in sequence to form a recovery execution trajectory.

[0014] Optionally, updating the centralized dual-critic network based on the two evaluation results specifically includes: The conflict state, joint candidate solutions, and positive execution trajectory are input into the first Critic network, and the first evaluation result is generated based on the execution results of the target business process under the joint candidate solutions. The recovered execution trajectory is compared with the initial copy state, and the execution records that were not recovered after the joint candidate scheme was withdrawn are extracted. The conflict state, joint candidate scheme and unrecovered execution records are input into the second Critic network to generate the second evaluation result. The evaluation deviations between the first and second Critic networks are calculated based on the first and second evaluation results, respectively, and the centralized dual Critic network is updated through backpropagation. When the cumulative number of updates to the centralized dual Critic network reaches the delay period, the combined difference data is read according to the permission source identifier. The combined difference data and the second evaluation result are associated with the corresponding Actor network instance. The update gradient of the corresponding Actor network instance is calculated through the first Critic network and the network parameters are updated.

[0015] Optionally, the step of outputting the target permission scheme based on the updated Actor network specifically includes: The adjustment actions output by the updated Actor networks are merged according to the permission source identifier to generate the target permission scheme; Submit the target permission scheme and its corresponding conflict status to the permission approval process, and receive the approval result returned by the permission approval process; When the approval result is passed, the target permission scheme is written into the authorization configuration in the business environment and the target business process is executed, and the permission verification results and permission usage records during the execution of the target business process are recorded. When the approval result is "modification", the target permission scheme is modified according to the approval result and written into the authorization configuration. The execution result, conflict status and target permission scheme are associated and saved.

[0016] According to an embodiment of the present invention, a human resource management access control system based on reinforcement learning includes the following modules: The permission conflict identification module is used to obtain human resources business authorization information and task information, identify permission items that exceed the scope of task permissions after the permission sources are superimposed, establish the association between permission items and permission sources, and generate conflict status. The permission copy control module is used to copy the task data and authorization configuration corresponding to the target business process according to the conflict state, establish a permission business copy isolated from the business environment, and perform permission business copy restoration according to the initial copy state. The multi-agent decision-making module is used to establish an Actor network and a centralized dual-Critic network that correspond one-to-one with the source of authority based on MATD3. The Actor network generates adjustment actions and merges them to form a joint candidate solution. The scheme pre-simulation module is used to form a single-source scheme from the joint candidate schemes, execute the single-source scheme and the joint candidate scheme within the authorized business copy, generate combination difference data based on the execution trajectory, and perform the reversal re-simulation of the joint candidate scheme to form a recovery execution trajectory. The network training module is used to input the forward execution trajectory into the first Critic network, input the recovered execution trajectory into the second Critic network, update the centralized dual Critic network, and update the corresponding Actor network according to the delay period based on the combination difference data. The permission approval and execution module is used to generate a target permission scheme based on the updated Actor network, submit the target permission scheme to the permission approval process, update the authorization configuration in the business environment and record the execution results according to the approval results.

[0017] The beneficial effects of this invention are: (1) By superimposing and analyzing the authorization ranges corresponding to different permission sources and comparing the combined permission range with the task permission range, this invention can identify permission conflicts formed by the combined effect of multiple permission sources, clarify the permission items that exceed the task permission range and their corresponding permission sources, and improve the accuracy of permission conflict location.

[0018] (2) By establishing a permission business copy isolated from the business environment, the present invention performs the same business process on single-source schemes and joint candidate schemes, and performs forward execution and revocation reenactment on joint candidate schemes. It can verify candidate permission schemes and the recovery status after revocation without modifying the formal authorization configuration, thereby reducing the impact of permission adjustment on actual business.

[0019] (3) The present invention sets up an Actor network corresponding to the source of permissions based on MATD3, and uses a centralized dual Critic network to evaluate the forward execution trajectory and the recovery execution trajectory respectively. The corresponding Actor network is updated in a targeted manner according to the combined difference data, so that the generated target permission scheme takes into account the business process execution requirements, the difference of permission source and the integrity of permission revocation. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a human resource management access control method based on reinforcement learning proposed in this invention; Figure 2 This is a schematic diagram of a module of a human resource management access control system based on reinforcement learning proposed in this invention; Figure 3 This is a schematic diagram illustrating the forward execution and revocation replay of a human resource management access control method based on reinforcement learning proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A human resource management access control system and method based on reinforcement learning, comprising the following steps: Obtain human resources business authorization information and task information, identify permission conflicts caused by overlapping permission sources, and generate conflict status; Based on the business environment, establish isolated permission business copies, write conflict states into the permission business copies, and form an execution rehearsal environment for candidate permission schemes; Based on MATD3, several Actor networks and a centralized dual Critic network are established. Each Actor network generates adjustment actions according to the source of permissions, and they are merged to form a joint candidate solution. The adjustment actions are each formed into a single-source solution. The same business process is executed on the single-source solution and the joint candidate solution within the permission business copy. Based on the execution differences, combined difference data is generated and fed back to the corresponding Actor network. Execute the joint candidate solution within the authorized service copy to form a positive execution trajectory; revoke the joint candidate solution and restore the authorized service copy; perform the revocation replay to form a recovery execution trajectory. The positive execution trajectory is input into the first Critic network, and the recovered execution trajectory is input into the second Critic network. The centralized dual Critic network is updated based on the two evaluation results, and several Actor networks are updated according to the delay period based on the combined difference data. Based on the updated target permission scheme output by several Actor networks, the target permission scheme is submitted to the permission approval process and the execution result is recorded.

[0023] In this embodiment, obtaining human resources business authorization information and task information specifically includes: The system acquires human resources business authorization information and task information, and then breaks down the human resources business authorization information according to the permission source identifier to form the authorization scope corresponding to each permission source. The human resources business authorization information is read from the permission configuration record, role assignment record, and task authorization record. The permission source identifier is used to distinguish between job role authorization, business task authorization, and approval authorization. The authorization records are grouped according to the permission source identifier, and the business object, allowed operation, applicable data scope, and effective time corresponding to the same permission source are merged into one authorization scope. In one embodiment, a user has both HRBP role authorization and payroll review task authorization. The HRBP role authorization forms the first authorization scope, and the payroll review task authorization forms the second authorization scope. The task permission scope required to execute the target business process is extracted based on the task information. The task information uses the business task records that have been approved. The business objects and allowed operations called by each process node are read according to the target business process associated with the task record. The permissions required to complete the target business process are merged into the task permission scope. The task permission scope only retains the permissions actually called by the target business process. Permissions that users already hold based on their job roles but have not been called by the target business process are not written into the task permission scope. In one embodiment, the payroll review task calls the abnormal salary record query and abnormal marking operations. The task permission scope does not include the permissions for employee performance query and salary result export. The authorization scopes corresponding to each permission source are superimposed to generate a combined permission scope; the authorization scopes corresponding to each permission source are aligned according to the business object, and the allowed operations, applicable data scopes, and effective time that apply to the same business object are merged. The authorization scopes that apply to different business objects but can be associated through a common business identifier are recorded as associated authorization scopes to form a combined permission scope; in one embodiment, the first authorization scope allows querying employee name and performance results based on employee identifier, and the second authorization scope allows querying cross-department salary records based on employee identifier. After superimposition, a combined permission scope that associates name, performance results, and salary records based on employee identifier is formed. The combined permission scope is compared item by item with the task permission scope to extract permission items in the combined permission scope that exceed the task permission scope. The combined permission scope and task permission scope are then compared according to business object, allowed operation, applicable data range, and valid time. Business objects, operations, data ranges, and valid times that are not recorded in the task permission scope but are accessible by the combined permission scope are recorded as permission items exceeding the task permission scope. When the combined permission scope can form a data association not recorded in the task permission scope through a common business identifier, the corresponding data association is recorded as a permission item exceeding the task permission scope. In one embodiment, if the task permission scope only allows querying abnormal salary records, and the combined permission scope can simultaneously associate employee names and performance results, then the employee name query, performance result query, and associated access relationship are recorded as permission items exceeding the task permission scope. Based on the permission items exceeding the task's permission scope, determine the permission sources involved in the overlay, and establish an association between the permission items exceeding the task's permission scope and the permission sources; read the authorization record corresponding to each permission item exceeding the task's permission scope; when a permission item exceeding the task's permission scope is generated by a single permission source, establish a single-source association between the permission item exceeding the task's permission scope and the corresponding permission source; when a permission item exceeding the task's permission scope is generated by multiple permission sources, record the authorization content provided by each permission source, and establish a combined-source association between the permission item exceeding the task's permission scope and multiple permission sources; in one embodiment, employee name and performance results are derived from HRBP role authorization, cross-departmental salary records are derived from salary review task authorization, and the associated access capabilities of name, performance results, and salary records correspond to a combined-source association between HRBP role authorization and salary review task authorization; Convert the combined permission scope, task permission scope, permission items exceeding the task permission scope, and their associated relationships into a conflict state, and input the conflict state into the permission business copy.

[0024] In this embodiment, establishing isolated permission service copies based on the business environment specifically includes: Based on the conflict status, locate the task data and authorization configuration corresponding to the target business process, copy the task data and authorization configuration to an independent replica space, and establish a data identifier mapping between the business environment and the independent replica space; retrieve the task data called by the target business process based on the permission items exceeding the task permission scope recorded in the conflict status, and read the authorization configuration corresponding to the permission source association; create an independent database mode as an independent replica space with the conflict number, and copy the retrieved task data and authorization configuration to the independent database mode; during the copying process, generate a replica data identifier for each piece of task data, and establish a data identifier mapping table composed of the official data identifier, the replica data identifier, and the conflict number, so that the target business process can locate the corresponding replica data based on the official data identifier; Based on the data identifier mapping, access requests generated by the target business process are redirected to an independent replica space. Within this space, conflict states are loaded and the permission source overlay state is restored, blocking data read / write operations from the business environment. An independent access session is created for the pre-execution simulation, and a conflict number is written into it. When the target business process initiates an access request, the data identifier mapping table is queried based on the conflict number, and the official data identifier in the access request is replaced with the replica data identifier. The access request connection address is switched to independent database mode. The corresponding authorization configuration is written to the independent replica space based on the combined permission scope in the conflict state, enabling the independent access session to execute the target business process according to the permissions overlay from the permission source. The independent access session uses a database account with only independent database mode access permissions; the business environment rejects read and write requests initiated by this database account. Before loading the candidate permission scheme, the initial replica state of the independent replica space is recorded, and the data changes generated by the execution of the candidate permission scheme are limited to the independent replica space. Before writing the candidate permission scheme, a database snapshot is generated for the task data and authorization configuration in the independent replica space, and the database snapshot number is associated with the conflict number and saved to form the initial replica state. The candidate permission scheme is written into the authorization configuration in the independent replica space, and the target business process is executed through an independent access session. During the execution, all authorization configuration changes, task data changes, and access records generated are written to the independent replica space and associated according to the conflict number. The business environment does not accept data submission requests generated by the independent access session. After completing the execution rehearsal, the task data and authorization configuration are restored based on the initial replica state to form a permission business replica. After completing one execution rehearsal, the corresponding independent access session is terminated, the database snapshot number associated with the conflict number is read, the independent replica space is rolled back to the initial replica state, and the access record index generated in this execution rehearsal is deleted. After the rollback is completed, consistency checks are performed on the task data and authorization configuration respectively. If the check results are consistent, the independent access session is re-established. Single-source schemes, joint candidate schemes, and reversal rehearsals all start from the same initial replica state, and each execution rehearsal generates an independent rehearsal number.

[0025] In this embodiment, establishing several Actor networks and a centralized dual-Critic network based on MATD3 specifically includes: Extract the permission sources participating in permission stacking from the conflict state, establish Actor network instances according to the permission sources, and separate the source conflict states associated with each permission source from the conflict state; based on the association between permission items exceeding the task permission scope and permission sources in the conflict state, count the number of permission sources participating in permission stacking, and establish one Actor network instance for each permission source; each Actor network instance adopts the same network structure and saves independent network parameters and training states; in one embodiment, the conflict state records two permission sources: HRBP role authorization and salary review task authorization, and establishes two Actor network instances; for each permission source, extract the corresponding authorization scope, the corresponding permission items exceeding the task permission scope, and the association relationships formed with other permission sources from the conflict state, and combine them to form a source conflict state; when the same permission item exceeding the task permission scope is associated with multiple permission sources, the permission items exceeding the task permission scope are written into multiple source conflict states, retaining the same permission item identifier; Each source conflict state is input into the corresponding Actor network instance to generate an adjustment action carrying the permission source identifier and permission item identifier. The category information in the source conflict state is converted into encoded values, and the data range ratio, validity time, and historical usage count are converted into normalized values ​​and input into the corresponding Actor network instance. The Actor network instance outputs adjustment parameters corresponding to the permission items that exceed the task permission range. The adjustment parameters are used to determine the data open ratio, validity time, access count, and field visibility of the permission item. When generating the adjustment action, the permission source identifier corresponding to the Actor network instance, the identifier of the permission item being adjusted, and the adjustment parameters are encapsulated into the same action record. In one embodiment, the Actor network instance corresponding to the HRBP role authorization outputs a data open ratio of 0 and a validity time of 0 for the performance query permission, and the Actor network instance corresponding to the payroll review task authorization outputs a data open ratio of 0.3 and a validity time of 72 hours for the cross-departmental salary query permission. Multiple adjustment actions are merged according to the permission item identifier, preserving the correspondence between the permission source identifier and the adjustment action during the merging process to form a joint candidate scheme. The joint candidate scheme is then input into a centralized dual-Critic network. All adjustment actions are grouped according to the permission item identifier, and adjustment actions with different permission item identifiers are written into the joint candidate schemes respectively. Adjustment actions with the same permission item identifier but different permission source identifiers are not overwritten and are saved separately according to the permission source identifier. The joint adjustment result of the permission item is generated based on the data openness ratio, effective time, access frequency, and field visibility corresponding to each adjustment action. The joint candidate scheme saves the permission item identifier, the joint adjustment result, and the permission source identifiers and adjustment parameters participating in the joint adjustment. The conflict state is combined with the joint candidate scheme and then input into the centralized dual-Critic network. The centralized dual-Critic network evaluates the joint candidate scheme after the adjustment actions of each permission source have worked together.

[0026] In this embodiment, executing the same business process for single-source solutions and joint candidate solutions within the authorized business copy specifically includes: According to the permission source identifier, the corresponding adjustment action is extracted from the joint candidate scheme. The adjustment actions associated with the same permission source identifier are written into the authorization configuration in the initial copy state, while the authorization configuration corresponding to other permission sources remains unchanged, forming a single-source scheme. The joint candidate scheme saves the correspondence between adjustment actions, permission source identifiers, and permission item identifiers. The adjustment actions are grouped according to the permission source identifier, and each group of adjustment actions corresponds to a single-source scheme. When generating a single-source scheme, the permission business copy is first restored to the initial copy state, and then the adjustment actions in the current group are written into the authorization configuration of the corresponding permission source. Permission sources that do not belong to the current group continue to use the authorization configuration in the initial copy state. In one embodiment, the joint candidate scheme includes the field visibility adjustment action corresponding to the HRBP role authorization and the data open ratio adjustment action corresponding to the payroll review task authorization. When generating the HRBP single-source scheme, only the field visibility adjustment action is written, and the payroll review task authorization remains in the initial copy state. Each single-source solution is loaded sequentially into the permission business copy. An access request is initiated from the starting node of the target business process. Execution is completed according to the node order of the target business process, and the single-source execution trajectory is recorded. After each single-source solution is completed, the permission business copy is restored based on the initial copy state. Each single-source solution is executed sequentially according to the sorting result of the permission source identifier, using the same task information and the same target business process input data for each execution. After a single-source solution is loaded, an independent access session is established from the starting node of the target business process. The corresponding business nodes of the target business process are called sequentially, and the access request, permission verification result, returned data identifier, execution status, and execution time of each business node are recorded. These are combined according to the order of the business nodes to form a single-source execution trajectory. After the single-source solution is executed, the independent access session is terminated, and the initial copy state associated with the conflict number is called to restore the task data and authorization configuration. After restoration is completed, the next single-source solution is loaded. After all single-source solutions have been executed, the joint candidate solution is loaded into the restored permission business copy, and the target business process is executed again in the same node order, recording the joint execution trajectory. After all single-source solutions have been executed, the permission business copy is restored to its initial copy state, and all adjustment actions in the joint candidate solution are written into the authorization configuration of the corresponding permission source. The target business process is executed using the same task information, target business process input data, and business node order as in the single-source solution execution phase. During execution, the execution data of each business node is collected in the same recording format as the single-source execution trajectory, and the joint execution trajectory is generated in the order of the business nodes.

[0027] In this embodiment, generating combined difference data based on execution differences and feeding it back to the corresponding Actor network specifically includes: Align the joint execution trajectory with each individual execution trajectory according to the node order of the target business process, and extract trajectory segments where the permission verification results change under the same business node; configure a unique node identifier for each business node in the target business process, and record the permission verification results according to the node identifier in both the individual execution trajectory and the joint execution trajectory; when aligning execution trajectories, use the node identifier in the joint execution trajectory as a benchmark to retrieve execution records with the same node identifier in each individual execution trajectory, and compare the permission verification results of the corresponding execution records; when the values ​​corresponding to the same business node in the joint execution trajectory and the individual execution trajectory are inconsistent, extract the execution records from the generation of the permission verification request to the end of the business node execution to form a trajectory segment; Based on the permission item identifier corresponding to the trajectory segment, the corresponding adjustment action and permission source identifier are determined from the joint candidate scheme. The trajectory segment, adjustment action, and permission source identifier are associated to generate combined difference data. The permission item identifier carried by the permission verification request in the trajectory segment is read. The joint candidate scheme is retrieved by the permission item identifier to obtain the adjustment action generated for the same permission item and the permission source identifier that generated the adjustment action. When the same permission item is associated with multiple adjustment actions, the association record between the trajectory segment and each adjustment action is established respectively. The association record is assigned to the same joint candidate scheme execution process by the pre-rehearsal number. The difference in permission verification result, adjustment action output value, permission source identifier, and permission item identifier corresponding to the joint execution trajectory and the single source execution trajectory are written into the same data record to generate combined difference data. In one embodiment, the adjustment action corresponding to the HRBP role authorization and the adjustment action corresponding to the salary review task authorization together cause the salary export node to change from refuse execution to allow execution. Then, combined difference data corresponding to the two permission sources are generated respectively. According to the permission source identifier, the combined difference data is written into the training sample area of ​​the corresponding Actor network instance. When the delayed update period is reached, the combined difference data in the training sample area is read, backpropagation is performed on the corresponding Actor network instance, and the network parameters are updated. A training sample area is established for each Actor network instance. According to the permission source identifier, the combined difference data is written into the corresponding training sample area. The source conflict state used when generating the adjustment action is associated with the combined difference data and saved. After the centralized dual-Critic network completes two network parameter updates, it is determined that one Actor network delayed update period has been reached. The source conflict state and combined difference data are read from the corresponding training sample area. The source conflict state is input into the corresponding Actor network instance to generate the current adjustment action. Then, the current adjustment action is combined with the adjustment actions generated by other Actor network instances to form a joint action and input into the centralized dual-Critic network. The Actor loss is calculated based on the action value output by the first Critic network. When backpropagating the gradient obtained by the Actor loss, the gradient at the output end of the corresponding adjustment action is retained according to the permission item identifier in the combined difference data. The gradient is passed to the corresponding Actor network instance and the network parameters are updated. After the update is completed, the combined difference data markers that participated in this update are cleared.

[0028] In this embodiment, forming a forward execution trajectory and forming a recovery execution trajectory specifically include: The joint candidate solution is written into the authorization configuration of the permission business copy. Access requests are executed according to the target business process, and the permission verification results and execution status of each business node are recorded sequentially to form a forward execution trajectory. According to the correspondence between the permission source identifier, permission item identifier and adjustment action in the joint candidate solution, the corresponding authorization configuration in the permission business copy is modified, and a rehearsal number is assigned to this execution. Using the task information as the input of the target business process, access requests are initiated sequentially from the starting business node. The node identifier, access request, permission verification result and execution status of each business node are written under the same rehearsal number according to the execution time. The permission verification result includes allow and deny, and the execution status is executed as complete, failed, or not executed due to the failure of the preceding business node. The records are arranged according to the execution order of the business nodes to form a forward execution trajectory. Terminate the access session corresponding to the forward execution, undo the adjustment actions written by the joint candidate scheme, and restore the permission business copy according to the initial copy state; after the target business process is completed, close the access session associated with the rehearsal number, locate the modified authorization configuration according to the permission item identifier saved by the joint candidate scheme, delete the newly added authorization record and restore the modified authorization parameters; read the initial copy state saved before the candidate permission scheme is loaded, roll back the authorization configuration and task data in the permission business copy to the initial copy state, and perform consistency verification on the number of records, data identifiers and authorization configuration versions after restoration; after the consistency verification passes, mark the permission business copy as restored. The target business process is re-executed using the same task information as the forward execution, and the permission verification results and execution status of each business node are recorded sequentially to form a recovery execution trajectory. After the permission business copy is restored, the access session is re-established using the task information and process input corresponding to the forward execution phase, and the target business process is executed from the same starting business node in the same node order. The permission verification results and execution status of each business node are collected using the same recording method as the forward execution trajectory, and the recorded results are associated with the original rehearsal number. The recovery execution trajectory is generated according to the order of business nodes, and the forward execution trajectory and the recovery execution trajectory can be matched according to the node identifier.

[0029] In this embodiment, updating the centralized dual-Critic network based on the two evaluation results specifically includes: The conflict state, joint candidate solutions, and forward execution trajectory are input into the first Critic network. Based on the execution results of the target business process under the joint candidate solutions, a first evaluation result is generated. The forward execution trajectory is converted into a trajectory sequence according to the order of business nodes. Let the total number of business nodes in the target business process be N, the number of business nodes that have been completed be Nc, the number of business nodes that have failed to be executed be Nf, and the number of business nodes that are allowed to be executed under the joint candidate solutions but are not allowed to be executed within the scope of task permissions be No. The forward execution reward is calculated according to r1=(Nc-Nf-2No) / N. The conflict state, joint candidate solutions, trajectory sequence, and forward execution reward are input into the first Critic network, and the first Critic network outputs the first evaluation result Q1. The restored execution trajectory is compared with the initial replica state. Execution records that were not restored after the joint candidate solution was withdrawn are extracted. The conflict state, joint candidate solution, and unrestored execution records are input into the second Critic network to generate a second evaluation result. The restored execution trajectory is matched with the execution records corresponding to the initial replica state according to the business node identifier. Business nodes with consistent permission verification results and execution states are recorded as restored nodes, and business nodes with inconsistent results are recorded as unrestored nodes. Let the number of unrestored nodes be Nr. The withdrawal and restoration reward is calculated according to r2 = 1 - 2Nr / N. When Nr is 0, r2 is 1, and when Nr equals N, r2 is -1. The conflict state, joint candidate solution, execution records corresponding to unrestored nodes, and withdrawal and restoration reward are input into the second Critic network, and the second Critic network outputs the second evaluation result Q2. The evaluation deviations between the first and second Critic networks are calculated based on the first and second evaluation results, respectively. The centralized dual-Critic network is then updated via backpropagation. The target Actor network is used to generate the target joint action corresponding to the next conflict state. The target joint action is input into the target first Critic network and the target second Critic network, respectively. The first target evaluation value and the second target evaluation value are calculated according to y1=r1+0.99Q′1 and y2=r2+0.99Q′2. The first Critic network loss and the second Critic network loss are calculated according to L1=(Q1-y1)² and L2=(Q2-y2)². Backpropagation is performed with a learning rate of 0.001 to complete the parameter update of the centralized dual-Critic network. When the cumulative number of updates of the centralized dual-Critic network reaches the delay period, the combined difference data is read according to the permission source identifier. The combined difference data and the second evaluation result are associated with the corresponding Actor network instance. The update gradient of the corresponding Actor network instance is calculated through the first Critic network and the network parameters are updated. The delay period is set to complete two parameter updates for the centralized dual-Critic network. The corresponding Actor network instance is determined according to the permission source identifier in the combined difference data, and the corresponding adjustment action output position is determined according to the permission item identifier. The corresponding position is recorded as 1, and the other adjustment action output positions are recorded as 0 to form an action position mask. When the second evaluation result Q2 is less than 0, the action output value corresponding to the unrecovered node is replaced with the action value before the joint candidate scheme is loaded. When the second evaluation result Q2 is not less than 0, the original action output value is retained. The processed joint action is input into the first Critic network, the gradient of the first evaluation result relative to the output action of the corresponding Actor network instance is calculated, the gradient is multiplied by the action position mask item by item, and backpropagation is performed to update the network parameters of the corresponding Actor network instance with a learning rate of 0.0001.

[0030] In this embodiment, the updated target permission scheme for several Actor networks specifically includes: The adjustment actions output by the updated Actor networks are merged according to the permission source identifier to generate the target permission scheme; the adjustment actions are collected according to the permission source identifier and written to the corresponding authorization configuration location according to the permission item identifier; when the same permission item corresponds to multiple adjustment actions, the correspondence between the adjustment actions and the permission source identifier is retained to form the target permission scheme. Submit the target permission scheme and its corresponding conflict status to the permission approval process, and receive the approval result returned by the permission approval process; generate a scheme number for the target permission scheme, and write the scheme number, conflict status and target permission scheme together into the approval record; the approval result has two statuses: approved and modified, and the approver's identifier and approval time are recorded; When the approval result is passed, the target permission scheme is written into the authorization configuration in the business environment and the target business process is executed. The permission verification results and permission usage records during the execution of the target business process are recorded. The authorization configuration in the business environment is located according to the permission item identifier in the target permission scheme and the update is completed. The target business process is started with the task information, and the permission verification results, actual called permission items and execution time of each business node are associated with the scheme number. When the approval result is "modification", the target permission scheme is modified according to the approval result and written into the authorization configuration. The execution result, conflict status, and target permission scheme are associated and saved. The modification location and modification value recorded in the approval result are read, and the corresponding adjustment action in the target permission scheme is replaced to generate the post-approval permission scheme. The post-approval permission scheme is written into the authorization configuration, and the execution result, post-approval permission scheme, original target permission scheme, and conflict status are associated with the same scheme number.

[0031] According to an embodiment of the present invention, a human resource management access control system based on reinforcement learning includes the following modules: The permission conflict identification module is used to obtain human resources business authorization information and task information, identify permission items that exceed the scope of task permissions after the permission sources are superimposed, establish the association between permission items and permission sources, and generate conflict status. The permission copy control module is used to copy the task data and authorization configuration corresponding to the target business process according to the conflict state, establish a permission business copy isolated from the business environment, and perform permission business copy restoration according to the initial copy state. The multi-agent decision-making module is used to establish an Actor network and a centralized dual-Critic network that correspond one-to-one with the source of authority based on MATD3. The Actor network generates adjustment actions and merges them to form a joint candidate solution. The scheme pre-simulation module is used to form a single-source scheme from the joint candidate schemes, execute the single-source scheme and the joint candidate scheme within the authorized business copy, generate combination difference data based on the execution trajectory, and perform the reversal re-simulation of the joint candidate scheme to form a recovery execution trajectory. The network training module is used to input the forward execution trajectory into the first Critic network, input the recovered execution trajectory into the second Critic network, update the centralized dual Critic network, and update the corresponding Actor network according to the delay period based on the combination difference data. The permission approval and execution module is used to generate a target permission scheme based on the updated Actor network, submit the target permission scheme to the permission approval process, update the authorization configuration in the business environment and record the execution results according to the approval results.

[0032] Example 1: To verify the feasibility of this invention in practice, it was applied to a human resources business platform. The platform includes job authorization, temporary task authorization, and approval authorization. Some personnel simultaneously handle personnel collaboration, payroll review, and file verification. While individual access permissions meet business requirements, their combined use may grant the same person access to a combination of identity, performance, and payroll information. Traditional permission consolidation expands the scope of access, while permission overlap can prevent necessary business processes from executing, and manual verification makes it difficult to determine the source of conflict.

[0033] During implementation, business authorization information and task information are read, and the authorization scope is split according to the permission source identifier. The task permission scope required for the target business process is extracted from the task information. The various authorization scopes are superimposed to form a combined permission scope. After comparing it with the task permission scope, permission items that exceed the task's requirements are extracted, and the association between permission items and permission sources is established to generate conflict states. Taking an employee with both HRBP authorization and payroll review authorization as an example, if the task only requires querying abnormal salary records and submitting the review results, but the combined permission scope can associate the employee's name, performance results, and specific salary through the employee identifier, then the name query, performance query, and associated access relationship are identified as permission conflicts to be processed.

[0034] Based on the conflict status, the task data and authorization configuration required for the target business process are copied from the business environment, and a permission business copy is established in an isolated copy space. Access requests generated by the target business process are redirected to the copy space through data identifier mapping, and copy access sessions cannot write to formal business data. The initial copy state is saved before loading candidate permission schemes, so that each rehearsal starts with the same task data and authorization configuration. The permission business copy is restored after the rehearsal ends.

[0035] Based on MATD3, Actor network instances are established according to the source of permissions. The Actor network corresponding to HRBP authorization generates adjustment actions for identity and performance information, while the Actor network corresponding to payroll review authorization generates adjustment actions for salary data range and access operations. Each adjustment action retains the permission source identifier and permission item identifier, and they are merged to form a joint candidate solution. Adjustment actions from a single permission source are extracted to form a single-source solution, and each single-source solution and the joint candidate solution execute the same target business process in the permission business copy. The system aligns the single-source execution trajectory and the joint execution trajectory according to business nodes, traces the permission verification results that have changed in the joint state to the corresponding adjustment action, forms combined difference data, and writes it into the training sample area of ​​the corresponding Actor network.

[0036] After the joint candidate solution completes its forward execution, the permission verification results and execution status of each business node are recorded, forming a forward execution trajectory. Subsequently, the access session is terminated, the adjustments written by the joint candidate solution are revoked, the authorization configuration and task data are restored based on the initial replica state, and the target business process is re-executed using the same task information, forming a recovery execution trajectory. The first Critic network evaluates the business adaptability of the candidate solution based on the forward execution trajectory, and the second Critic network evaluates the state recovery degree after permission revocation based on the recovery execution trajectory. After the centralized dual Critic network update, the combined difference data is read according to the latency period, and the corresponding Actor network from which the permission originates is updated accordingly.

[0037] After training, the adjustment actions output by each Actor network are merged into the target permission scheme. The target permission scheme changes the display of names across ranges to employee IDs, disables performance result-related queries, retains the submission of abnormal salary record queries and review results, and restricts unnecessary export operations. The target permission scheme is submitted to the permission approval process, and after approval, it is written into the formal authorization configuration. When approvers modify the scheme, the system saves the permission scheme before and after modification, as well as the execution results, forming subsequent training data. Through this process, candidate schemes undergo business verification before formal implementation, conflicting differences can be traced back to the specific source of permissions, and formal authorization is still confirmed by the approval process.

[0038] Table 1: Comparison of the implementation effects of different access control methods

[0039] Comparative results show that the method of this invention outperforms fixed-rule RBAC, TD3, and conventional MATD3 in terms of conflict identification accuracy, conflict identification recall, and business process completion rate. Single-source schemes and joint candidate schemes are pre-simulated using the same process, enabling execution changes resulting from combined permissions to be traced back to the corresponding permission source; permission business copies allow candidate schemes to be validated in an isolated environment, reducing insufficient permissions after formal execution. Forward execution trajectories and recovery execution trajectories are used for training the dual-Critic network, allowing the model to simultaneously consider business execution and undo / recovery, reducing the rate of unauthorized access. Combined difference data is used to directionally update the corresponding Actor network, reducing irrelevant adjustments, thus resulting in a lower average processing time than other methods.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A human resource management access control method based on reinforcement learning, characterized in that, Includes the following steps: Obtain human resources business authorization information and task information, identify permission conflicts caused by overlapping permission sources, and generate conflict status; Based on the business environment, establish isolated permission business copies, write conflict states into the permission business copies, and form an execution rehearsal environment for candidate permission schemes; Based on MATD3, several Actor networks and a centralized dual Critic network are established. Each Actor network generates adjustment actions according to the source of permissions, and they are merged to form a joint candidate solution. The adjustment actions are each formed into a single-source solution. The same business process is executed on the single-source solution and the joint candidate solution within the permission business copy. Based on the execution differences, combined difference data is generated and fed back to the corresponding Actor network. Execute the joint candidate solution within the authorized service copy to form a positive execution trajectory; revoke the joint candidate solution and restore the authorized service copy; perform the revocation replay to form a recovery execution trajectory. The positive execution trajectory is input into the first Critic network, and the recovered execution trajectory is input into the second Critic network. The centralized dual Critic network is updated based on the two evaluation results, and several Actor networks are updated according to the delay period based on the combined difference data. Based on the updated target permission scheme output by several Actor networks, the target permission scheme is submitted to the permission approval process and the execution result is recorded.

2. The human resource management access control method based on reinforcement learning according to claim 1, characterized in that, The acquisition of human resources business authorization information and task information specifically includes: Obtain human resources business authorization information and task information, and break down the human resources business authorization information according to the source identifier of the permission to form the authorization scope corresponding to each source of permission; Extract the scope of task permissions required to execute the target business process based on the task information; The authorization scopes corresponding to each permission source are superimposed to generate a combined permission scope; The combined permission scope is compared with the task permission scope item by item, and the permission items in the combined permission scope that exceed the task permission scope are extracted. Determine the source of permissions to be overlaid based on the permission items that exceed the scope of the task permissions, and establish the association between the permission items that exceed the scope of the task permissions and the permission sources. Convert the combined permission scope, task permission scope, permission items exceeding the task permission scope, and their associated relationships into a conflict state, and input the conflict state into the permission business copy.

3. The human resource management access control method based on reinforcement learning according to claim 2, characterized in that, The establishment of isolated permission-based business copies based on the business environment specifically includes: Based on the conflict status, locate the task data and authorization configuration corresponding to the target business process, copy the task data and authorization configuration to an independent replica space, and establish a data identification mapping between the business environment and the independent replica space. Based on the data identifier mapping, the access requests generated by the target business process are redirected to an independent replica space. The conflict state is loaded in the independent replica space and the permission source superposition state is restored to block the access requests from reading and writing data in the business environment. Record the initial replica state of the independent replica space before loading the candidate permission scheme, and limit the data changes generated by the execution of the candidate permission scheme to the independent replica space; After completing the execution rehearsal, the task data and authorization configuration are restored based on the initial copy state to form a permission business copy.

4. The human resource management access control method based on reinforcement learning according to claim 3, characterized in that, The establishment of several Actor networks and a centralized dual-Critic network based on MATD3 specifically includes: Extract the source of permissions involved in the permission stacking from the conflict state, establish Actor network instances according to the source of permissions, and separate the source conflict state associated with each source of permissions from the conflict state. Input the conflict status of each source into the corresponding Actor network instance to generate an adjustment action carrying the permission source identifier and permission item identifier; Multiple adjustment actions are merged according to the permission item identifier. During the merging process, the correspondence between the permission source identifier and the adjustment action is preserved to form a joint candidate solution. The joint candidate solution is then input into a centralized dual-Critic network.

5. The human resource management access control method based on reinforcement learning according to claim 4, characterized in that, The specific steps of executing the same business process for single-source solutions and joint candidate solutions within the authorized business instance include: Extract the corresponding adjustment actions from the joint candidate scheme according to the permission source identifier, write the adjustment actions associated with the same permission source identifier into the authorization configuration in the initial copy state, keep the authorization configuration corresponding to other permission sources unchanged, and form a single source scheme; Each single-source solution is loaded into the permission business copy in sequence. An access request is initiated from the starting node of the target business process. The execution is completed in the order of the nodes of the target business process and the single-source execution trajectory is recorded. After each single-source solution is completed, the permission business copy is restored according to the initial copy state. After all single-source solutions have been executed, the combined candidate solution is loaded into the restored permission business copy, and the target business process is executed again in the same node order, recording the combined execution trajectory.

6. The human resource management access control method based on reinforcement learning according to claim 5, characterized in that, The step of generating combined difference data based on execution differences and feeding it back to the corresponding Actor network specifically includes: Align the joint execution trajectory with the individual execution trajectories according to the node order of the target business process, and extract the trajectory segments where the permission verification results change under the same business node; Based on the permission item identifier corresponding to the trajectory segment, the corresponding adjustment action and permission source identifier are determined from the joint candidate scheme, and the trajectory segment, adjustment action and permission source identifier are associated to generate combined difference data; According to the source of permission, the combined difference data is written into the training sample area of ​​the corresponding Actor network instance. When the delayed update period is reached, the combined difference data in the training sample area is read, backpropagation is performed on the corresponding Actor network instance, and the network parameters are updated.

7. The human resource management access control method based on reinforcement learning according to claim 6, characterized in that, The formation of a positive execution trajectory and the formation of a recovery execution trajectory specifically include: Write the joint candidate solution into the authorization configuration of the permission business copy, execute the access request according to the target business process, and record the permission verification results and execution status of each business node in sequence to form a positive execution trajectory; Terminate the access session corresponding to the forward execution, undo the adjustment actions written to the federated candidate scheme, and restore the permission service copy according to the initial copy status. The target business process is re-executed using the same task information as the forward execution, and the permission verification results and execution status of each business node are recorded in sequence to form a recovery execution trajectory.

8. The human resource management access control method based on reinforcement learning according to claim 7, characterized in that, The process of updating the centralized dual-Critic network based on the two evaluation results specifically includes: The conflict state, joint candidate solutions, and positive execution trajectory are input into the first Critic network, and the first evaluation result is generated based on the execution results of the target business process under the joint candidate solutions. The recovered execution trajectory is compared with the initial copy state, and the execution records that were not recovered after the joint candidate scheme was withdrawn are extracted. The conflict state, joint candidate scheme and unrecovered execution records are input into the second Critic network to generate the second evaluation result. The evaluation deviations between the first and second Critic networks are calculated based on the first and second evaluation results, respectively, and the centralized dual Critic network is updated through backpropagation. When the cumulative number of updates to the centralized dual Critic network reaches the delay period, the combined difference data is read according to the permission source identifier. The combined difference data and the second evaluation result are associated with the corresponding Actor network instance. The update gradient of the corresponding Actor network instance is calculated through the first Critic network and the network parameters are updated.

9. A human resource management access control method based on reinforcement learning according to claim 8, characterized in that, The updated target permission scheme based on the several Actor networks specifically includes: The adjustment actions output by the updated Actor networks are merged according to the permission source identifier to generate the target permission scheme; Submit the target permission scheme and its corresponding conflict status to the permission approval process, and receive the approval result returned by the permission approval process; When the approval result is passed, the target permission scheme is written into the authorization configuration in the business environment and the target business process is executed, and the permission verification results and permission usage records during the execution of the target business process are recorded. When the approval result is "modification", the target permission scheme is modified according to the approval result and written into the authorization configuration. The execution result, conflict status and target permission scheme are associated and saved.

10. A reinforcement learning-based human resource management access control system, applied to the reinforcement learning-based human resource management access control method described in any one of claims 1 to 9, characterized in that, Includes the following modules: The permission conflict identification module is used to obtain human resources business authorization information and task information, identify permission items that exceed the scope of task permissions after the permission sources are superimposed, establish the association between permission items and permission sources, and generate conflict status. The permission copy control module is used to copy the task data and authorization configuration corresponding to the target business process according to the conflict state, establish a permission business copy isolated from the business environment, and perform permission business copy restoration according to the initial copy state. The multi-agent decision-making module is used to establish an Actor network and a centralized dual-Critic network that correspond one-to-one with the source of authority based on MATD3. The Actor network generates adjustment actions and merges them to form a joint candidate solution. The scheme pre-simulation module is used to form a single-source scheme from the joint candidate schemes, execute the single-source scheme and the joint candidate scheme within the authorized business copy, generate combination difference data based on the execution trajectory, and perform the reversal re-simulation of the joint candidate scheme to form a recovery execution trajectory. The network training module is used to input the forward execution trajectory into the first Critic network, input the recovered execution trajectory into the second Critic network, update the centralized dual Critic network, and update the corresponding Actor network according to the delay period based on the combination difference data. The permission approval and execution module is used to generate a target permission scheme based on the updated Actor network, submit the target permission scheme to the permission approval process, update the authorization configuration in the business environment and record the execution results according to the approval results.

Images