Method and apparatus for determining task allocation strategy, electronic device, and program product

Abstract

The application discloses a task allocation strategy determination method and device, electronic equipment and program product, and relates to the technical field of artificial intelligence, wherein the determination method comprises: in the case that it is detected that an approval task in an approval process needs to be allocated, obtaining the features of the approval process and the features of the approval task, and determining the approval task as a to-be-allocated task; updating a preset knowledge graph based on all the features to obtain an updated knowledge graph; extracting target features associated with the to-be-allocated task from the updated knowledge graph, and constructing a state vector based on all the target features; and processing the state vector by using a reinforcement learning engine to obtain a task allocation strategy. The application solves the technical problem that the task allocation strategy cannot be flexibly determined in the related art, and reduces the execution efficiency of the approval process.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, and program product for determining a task allocation strategy. Background Technology

[0002] Currently, workflow engines generally use predefined static rules (e.g., if amount > X, then route to director approval, else route to manager approval) to determine task allocation (who to route to) and execution path (which branch to follow next). These rules are usually fixed during process design and rely on the experience of the process designer and pre-defined business logic. This approach cannot dynamically adapt to changes in the business environment (e.g., approver's temporary leave, sudden load changes, urgent business insertion) or complex contexts. Furthermore, task allocation and path selection are usually independent decision points, lacking collaborative optimization and making it difficult to flexibly determine task allocation strategies, thus reducing the execution efficiency of the approval process.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and program product for determining a task allocation strategy, to at least solve the technical problem in related technologies where the inability to flexibly determine a task allocation strategy reduces the execution efficiency of the approval process.

[0005] According to one aspect of the embodiments of this application, a method for determining a task allocation strategy is provided, comprising: when it is detected that an approval task in an approval process needs to be allocated, obtaining the features of the approval process and the features of the approval task, and determining the approval task as a task to be allocated; updating a preset knowledge graph based on all features to obtain an updated knowledge graph; extracting target features associated with the task to be allocated from the updated knowledge graph, and constructing a state vector based on all target features; and processing the state vector using a reinforcement learning engine to obtain a task allocation strategy.

[0006] Furthermore, before obtaining the characteristics of the approval process and the approval task, the process also includes: obtaining multiple elements in the approval workflow and identifying each element as an entity, determining whether there are dependencies between each entity, where each entity has an attribute value; and constructing a pre-defined knowledge graph based on all entities, all attribute values, and all dependencies.

[0007] Furthermore, the steps of obtaining the characteristics of the approval process and the characteristics of the approval tasks include: obtaining the status characteristics of the approval process and obtaining the characteristics of each task approval node in the approval process, the characteristics of which include at least: node load characteristics, node status characteristics and processing characteristics for historical approval tasks; and obtaining the task characteristics of the approval tasks.

[0008] Furthermore, the step of updating the preset knowledge graph based on all features to obtain the updated knowledge graph includes: updating each entity in the preset knowledge graph to obtain the updated entity, and updating the attribute values ​​corresponding to each entity to obtain the updated attribute values; updating the dependencies of the preset knowledge graph to obtain the updated dependencies; and determining the updated knowledge graph based on all updated entities, all updated attribute values, and all updated dependencies.

[0009] Furthermore, before processing the state vector using a reinforcement learning engine to obtain the task allocation policy, the process includes: determining multiple reward objectives, where each reward objective corresponds to a reward metric; constructing positive and negative rewards based on each reward metric; assigning preset weights to each positive and negative reward; and constructing a reward function based on all positive rewards, all negative rewards, and all preset weights, whereby the reward function is used to update the policy model in the reinforcement learning engine.

[0010] Furthermore, after processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, the process further includes: allocating tasks to be assigned based on the task allocation strategy to obtain allocation results; executing the tasks to be assigned according to the allocation results and obtaining the execution results of the tasks to be assigned; calculating the reward value of the task allocation strategy based on the execution results and the reward function; constructing a quadruple based on the state vector, task allocation strategy, reward value, and target state vector, and storing the quadruple in the replay pool. The target state vector is constructed by extracting features from the target knowledge graph, which is obtained by updating the updated knowledge graph. Sampled data is obtained by sampling the data in the replay pool at a preset period, and the strategy model is updated based on the sampled data.

[0011] Furthermore, before executing the assigned tasks based on the allocation results, the process also includes: triggering a review process when the task allocation strategy is prohibited from execution, wherein the review process is used to review the task allocation strategy and obtain the review results; and redetermining the task allocation strategy based on the review results.

[0012] According to another aspect of the embodiments of this application, a task allocation strategy determination apparatus is also provided, comprising: an acquisition unit, configured to acquire features of the approval process and features of the approval tasks when it is detected that an approval task in the approval process needs to be allocated, and determine the approval task as a task to be allocated; an update unit, configured to update a preset knowledge graph based on all features to obtain an updated knowledge graph; a construction unit, configured to extract target features associated with the task to be allocated from the updated knowledge graph, and construct a state vector based on all target features; and a processing unit, configured to process the state vector using a reinforcement learning engine to obtain a task allocation strategy.

[0013] Furthermore, the device for determining the task allocation strategy also includes: a first acquisition module, used to acquire multiple elements in the approval workflow before acquiring the features of the approval process and the features of the approval task, and to identify each element as an entity and determine whether there is a dependency relationship between each entity, wherein each entity has an attribute value; and a first construction module, used to construct a preset knowledge graph based on all entities, all attribute values ​​and all dependencies.

[0014] Furthermore, the acquisition unit includes: a second acquisition module, used to acquire the status characteristics of the approval process and the characteristics of each task approval node in the approval process, the characteristics including at least: node load characteristics, node status characteristics and processing characteristics for historical approval tasks; and a third acquisition module, used to acquire the task characteristics of the approval task.

[0015] Furthermore, the update unit includes: a first update module, used to update each entity of the preset knowledge graph to obtain the updated entity, and update the attribute values ​​corresponding to each entity to obtain the updated attribute values; a second update module, used to update the dependencies of the preset knowledge graph to obtain the updated dependencies; and a first determination module, used to determine the updated knowledge graph based on all updated entities, all updated attribute values, and all updated dependencies.

[0016] Furthermore, the device for determining the task allocation strategy also includes: a second determining module, used to determine multiple reward targets before processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, wherein each reward target corresponds to a reward index; a second constructing module, used to construct positive rewards and negative rewards based on each reward index; a first allocation module, used to assign preset weights to each positive reward and each negative reward; and a third constructing module, used to construct a reward function based on all positive rewards, all negative rewards, and all preset weights, wherein the reward function is used to update the policy model in the reinforcement learning engine.

[0017] Furthermore, the device for determining the task allocation strategy also includes: a second allocation module, used to allocate tasks to be allocated based on the task allocation strategy after processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, and obtain the allocation result; a first execution module, used to execute the tasks to be allocated according to the allocation result and obtain the execution result of the tasks to be allocated; a first calculation module, used to calculate the reward value of the task allocation strategy based on the execution result and the reward function; and a fourth construction module, used to construct a quadruple based on the state vector, the task allocation strategy, the reward value, and the target state vector, and store the quadruple in the replay pool, wherein the target state vector is constructed by extracting features from the target knowledge graph, the target knowledge graph is obtained by updating the updated knowledge graph, and the sampled data is obtained by sampling the data in the replay pool at a preset period, and the strategy model is updated based on the sampled data.

[0018] Furthermore, the task allocation strategy determination device also includes: a first triggering module, used to trigger a review process when the task allocation strategy is prohibited from execution before executing the task to be allocated according to the allocation result, wherein the review process is used to review the task allocation strategy and obtain the review result; and a third determination module, used to redetermine the task allocation strategy based on the review result.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method for determining any of the above-described task allocation strategies.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method for determining any of the above-described task allocation strategies.

[0021] In this invention, when it is detected that approval tasks in the approval process need to be assigned, the features of the approval process and the features of the approval tasks are obtained, and the approval tasks are identified as tasks to be assigned. Based on all features, a preset knowledge graph is updated to obtain an updated knowledge graph. Target features associated with the tasks to be assigned are extracted from the updated knowledge graph, and a state vector is constructed based on all target features. A reinforcement learning engine is used to process the state vector to obtain a task assignment strategy. This solves the technical problem in related technologies where the task assignment strategy cannot be flexibly determined, thus reducing the execution efficiency of the approval process.

[0022] In this invention, when the allocation requirement for approval tasks is detected, multi-dimensional features of the approval process and approval tasks can be collected, and the approval tasks can be identified as tasks to be allocated. Then, these features are integrated into a pre-built knowledge graph to obtain an updated knowledge graph. Through dynamic updates of the knowledge graph, the dynamic changes of the approval process can be reflected in real time. By extracting graph nodes and dependencies closely related to the tasks to be allocated, a state vector is constructed. Subsequently, the reinforcement learning engine receives this state vector and outputs the optimal task allocation strategy, thereby improving the execution efficiency of the approval process. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0024] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining a task allocation strategy is shown.

[0025] Figure 2 This is a flowchart of the method for determining the task allocation strategy according to Embodiment 1 of this application;

[0026] Figure 3 This is a schematic diagram of an apparatus for determining an optional task allocation strategy according to an embodiment of this application;

[0027] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that all relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected and involved in this invention are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data comply with the relevant laws, regulations, and standards of the relevant regions, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface. After receiving consent from the aforementioned user or organization, the relevant information is obtained. If the user chooses to refuse, the process enters the expert decision-making process.

[0031] Current decision-making is primarily based on local information about the current task (such as the application amount), making it difficult to effectively utilize global real-time state information (such as the current queue length of all approvers, historical approval efficiency, and collaboration relationships). Furthermore, rule-based decisions typically consider only a single objective (such as compliance), making it difficult to dynamically balance multiple objectives at runtime (such as minimizing total approval time, maximizing resource utilization, and reducing risk). In this invention, during workflow engine runtime, by dynamically sensing the workflow engine's execution state, constructing and updating a dynamic knowledge graph containing rich contextual information in real time, and utilizing reinforcement learning techniques to learn the optimal joint strategy for task allocation and path selection online, the shortcomings of static rule-based methods—ignoring global state and having a single optimization objective—are overcome, achieving a dynamic improvement in global execution efficiency.

[0032] The present invention will now be described in detail with reference to various embodiments.

[0033] Example 1

[0034] According to an embodiment of this application, an embodiment of a method for determining a task allocation strategy is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for determining a task allocation strategy is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1 The processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions may also be included. In addition, it may include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera, wherein the network interface can be connected to wired and / or wireless networks. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the task allocation strategy determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned task allocation strategy determination method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for determining the task allocation strategy is shown. Figure 2 This is a flowchart of the method for determining the task allocation strategy according to Embodiment 1 of this application, as follows: Figure 2 As shown, the method includes the following steps:

[0041] Step S201: When it is detected that the approval tasks in the approval process need to be assigned, the characteristics of the approval process and the characteristics of the approval tasks are obtained, and the approval tasks are identified as tasks to be assigned.

[0042] In this embodiment of the invention, the execution status of the workflow engine (i.e., BPMN (Business Process Model and Notation, a software component for executing business processes)) can be monitored. When it is identified that an approval task in the approval process needs to be assigned (i.e., a decision point has been reached, such as when a new approval task is created and it is necessary to decide which approver to assign the task to, or after the current approval task is completed, how to proceed with the next step or which approver to assign it to), all relevant characteristics of the approval process and tasks can be collected, including the current task node position, process variables (such as amount, task type, risk level, etc.), and the remaining time and deadline of the process. For example, in the process of approving a loan application, the current task node is the initial review, and the next task is risk assessment. At this time, the characteristics of the risk assessment task (such as the application amount, the risk level of the customer account, etc.) and the characteristics of the overall approval process (such as the current stage of the process, the remaining time requirement of the process, etc.) can be collected, and the risk assessment can be identified as a task to be assigned.

[0043] Step S202: Based on all features, update the preset knowledge graph to obtain the updated knowledge graph.

[0044] In this embodiment of the invention, the preset knowledge graph is a graph constructed during initialization, including information such as organizational structure and approval process. It can collect information such as current process instance, tasks to be assigned, relevant approvers, and global queue length (i.e., the current queue length of all approvers). The nodes, node attributes (such as the current load of approvers), and relationships (such as the relationship of newly assigned tasks) in the preset knowledge graph can be adjusted in real time. Based on all features, the preset knowledge graph can be updated to obtain the updated knowledge graph.

[0045] Step S203: Extract target features associated with the task to be assigned from the updated knowledge graph, and construct a state vector based on all target features.

[0046] In this embodiment of the invention, the target features are the features of nodes and edges related to the task to be assigned, including but not limited to the attributes of the task itself, the status of relevant approvers, and the status of the current process instance. Target features related to the task to be assigned can be extracted from the current knowledge graph, i.e., the updated knowledge graph, to construct the state representation (i.e., state vector) required for reinforcement learning. For example, if a risk assessment task is the task to be assigned, the type, amount, risk level, and features of all possible approvers (including approver expertise, current workload, and relationships with business entities associated with the task) can be extracted from the updated knowledge graph and encoded into a state vector.

[0047] Step S204: The state vector is processed using a reinforcement learning engine to obtain a task allocation strategy.

[0048] In this embodiment of the invention, the reinforcement learning engine is a module that uses the principles of reinforcement learning for decision optimization. It can learn and iteratively improve task allocation strategies through interaction with the environment. The state vector can be used as input, processed by the policy network within the reinforcement learning engine to obtain the optimal task allocation strategy (e.g., assigning the task to which approver and selecting subsequent process paths). The policy network is a mapping function from state to action, representing the probability distribution of choosing action a (i.e., task allocation strategy) given a state s (i.e., the state vector). By performing action a in the environment, a reward r is calculated using the reward function, and the system enters the next state s', forming an experience sample (s, a, r, s'), i.e., a quadruple. The expected value in state s is calculated using a value model, and the expected value in the next state s' is predicted. Therefore, using the reward r, the expected value in state s, and the expected value in state s', the dominance function value of action a (used to measure the performance of an action relative to average) is calculated. Based on the dominance function value, the policy network can be updated.

[0049] Optionally, the reinforcement learning engine can send task allocation strategy instructions to the BPMN engine. After receiving the instructions, the BPMN engine dynamically adjusts the subsequent paths of the current process according to the instructions. The BPMN engine can safely perform hot updates at runtime, that is, it can update and modify the state of the process or the subsequent paths without stopping or restarting the system.

[0050] In this embodiment of the invention, all the above steps can be continuously repeated to continuously learn and optimize the task allocation and path selection strategies of the BPMN engine during operation, so as to adapt to the dynamically changing business environment.

[0051] In summary, when an approval task in the approval process needs to be assigned, the multi-dimensional features of the approval process and the approval task can be obtained and used to update the knowledge graph to reflect the latest situation. Then, target features closely related to the approval task are extracted from the updated graph to construct a state vector, which is used as input to the reinforcement learning engine. Based on this state vector, the reinforcement learning engine learns autonomously and outputs the optimal task assignment strategy and path selection, effectively improving processing efficiency and balancing resource utilization. This solves the technical problem in related technologies where the task assignment strategy cannot be flexibly determined, which reduces the execution efficiency of the approval process.

[0052] In order to accurately construct a preset knowledge graph, in the method for determining the task allocation strategy provided in Embodiment 1 of this application, multiple elements in the approval workflow are obtained, and each element is identified as an entity. It is determined whether there is a dependency relationship between each entity, wherein each entity has an attribute value; based on all entities, all attribute values ​​and all dependencies, a preset knowledge graph is constructed.

[0053] In this embodiment of the invention, elements may include: employees (approvers), departments, roles, process definitions, process instances, tasks, business objects (customers, projects, products), etc. Dependencies exist between these elements: organizational relationships, process relationships, business relationships, and dynamic relationships. Organizational relationships include the approver's department, reporting line, and collaboration group; process relationships include the tasks the approver can approve and the optional paths following those tasks; business relationships include the customers and projects the employee is responsible for, and the projects and customers associated with the task; dynamic relationships include the current task allocation relationship and the current collaboration relationship (e.g., jointly processing the approval of a project). For example, an approval task may depend on a fixed approver, and that approver's decisions may affect the choice of subsequent paths. By determining these dependencies, the relationships between each element can be clearly understood, and each element can be identified as an entity. Each entity also corresponds to attribute values ​​(e.g., the approver's current number of pending tasks and historical average approval time). A pre-defined knowledge graph can be constructed based on all entities, all attribute values, and all dependencies. This knowledge graph can dynamically reflect the status and relationships of each element in the approval process, providing a rich contextual environment for subsequent reinforcement learning engines.

[0054] In order to accurately update the preset knowledge graph, it is necessary to first obtain the characteristics of the approval process and the characteristics of the approval task. In the method for determining the task allocation strategy provided in Embodiment 1 of this application, the status characteristics of the approval process are obtained, and the characteristics of each task approval node in the approval process are obtained. The characteristics include at least: node load characteristics, node status characteristics and processing characteristics for historical approval tasks; and the task characteristics of the approval task are obtained.

[0055] In this embodiment of the invention, the latest status characteristics of the approval process are acquired in real time, including but not limited to the current task node, process variables (such as application amount, risk level, etc.), elapsed time, remaining time, and deadline. It can also acquire the characteristics of each task approval node in the approval process, such as the node's real-time load (i.e., the task queue length of the task approval node at the current moment), status (whether the approver is online, whether there are leave arrangements, etc.), and past task processing performance (e.g., the average processing time, pass rate, and rejection rate of the approver in the past for similar tasks). Furthermore, it can acquire the task characteristics of the approval tasks, such as the type of approval task and the urgency of the approval task. It can also determine the matching degree between the approver's expertise and each approval task type. For example, if the task is a preliminary review of loan credit, and specialist A's expertise is in approving loan credit, then the matching degree is 99%. By collecting the status characteristics of the approval process, the real-time and historical characteristics of each approval node, and the characteristics of the approval tasks themselves, more accurate task allocation and path optimization decisions can be made based on a thorough understanding of the business environment.

[0056] In order to accurately determine the updated knowledge graph, in the method for determining the task allocation strategy provided in Embodiment 1 of this application, each entity of the preset knowledge graph is updated to obtain the updated entity, and the attribute values ​​corresponding to each entity are updated to obtain the updated attribute values; the dependencies of the preset knowledge graph are updated to obtain the updated dependencies; and the updated knowledge graph is determined based on all updated entities, all updated attribute values, and all updated dependencies.

[0057] In this embodiment of the invention, each entity in the preset knowledge graph can be updated based on real-time acquired features to obtain updated entities. For example, if approver A completes an approval task, then the entity of approver A will be marked as having reduced load, and its node attributes in the graph will be updated accordingly. Furthermore, the attribute values ​​corresponding to each entity can be updated to obtain updated attribute values, including but not limited to the approver's real-time load (such as the number of tasks to be processed), skill matching degree, and current busy / idle status. For example, in addition to reducing the task load attribute of approver A, the "estimated processing time" attribute value can also be updated, for example, from 2 hours to 1.5 hours, reflecting its enhanced processing capacity. The dependencies in the preset knowledge graph can be updated to obtain updated dependencies. Based on all updated entities, all updated attribute values, and all updated dependencies, the updated knowledge graph is determined, ensuring the continuous dynamism and timeliness of the knowledge graph, enabling it to reflect the latest dynamics in the approval process and changes in the approval environment in real time.

[0058] In order to accurately construct the reward function, in the method for determining the task allocation strategy provided in Embodiment 1 of this application, multiple reward targets are determined, wherein each reward target corresponds to a reward index; based on each reward index, positive rewards and negative rewards are constructed; preset weights are assigned to each positive reward and each negative reward; and a reward function is constructed based on all positive rewards, all negative rewards and all preset weights, wherein the reward function is used to update the policy model in the reinforcement learning engine.

[0059] In this embodiment of the invention, firstly, multiple reward objectives can be set, each corresponding to a reward metric, such as minimizing total approval time, maximizing throughput, and balancing load. For each reward objective, a set of positive rewards is defined to recognize behaviors that move towards the objective, and a set of negative rewards (penalties) is defined to punish behaviors that deviate from the objective. Preset weights are assigned to each positive and negative reward. Through weight allocation, an optimal balance point can be found among multiple reward metrics, based on all positive rewards (such as...). , , ), all negative rewards ( , , ) and all preset weights ( , , , , , ), construct the reward function (i.e. After each strategic decision, the model parameters are updated based on the reward calculated by this function. By finding the optimal solution in a multi-dimensional objective space, multiple objectives such as approval efficiency and rational resource allocation can be dynamically balanced, thereby optimizing the approval process and achieving refined management of resource scheduling.

[0060] To accurately update the strategy model, the task allocation strategy determination method provided in Embodiment 1 of this application involves: allocating tasks to be allocated based on the task allocation strategy to obtain allocation results; executing the tasks to be allocated according to the allocation results and obtaining the execution results of the tasks to be allocated; calculating the reward value of the task allocation strategy based on the execution results and the reward function; constructing a quadruple based on the state vector, task allocation strategy, reward value, and target state vector, and storing the quadruple in the replay pool. The target state vector is constructed using features extracted from the target knowledge graph, which is obtained by updating the updated knowledge graph. Sampled data is obtained by sampling the data in the replay pool at a preset period, and the strategy model is updated based on the sampled data.

[0061] In this embodiment of the invention, the task allocation strategy includes assigning the task to a specific approver and suggesting an approval path (such as an expedited approval path, a standard path, or an additional review path). Based on the task allocation strategy, the task to be assigned is allocated. For example, the task is dynamically assigned to a specified approver's to-do list, the execution path of the current process instance is modified, and for predefined optional branches, a specified path branch is selected to obtain the allocation result. The task to be assigned is executed according to the allocation result, and the execution result of the task to be assigned is obtained. The execution result includes, but is not limited to, the time it takes for the approver to complete the task, the approval result (approved, rejected, or requiring supplementary materials), and any abnormal states encountered during the approval process (such as the approver taking temporary leave or system failure). Based on the execution result and a reward function, the reward value of the task allocation strategy is calculated. For example, if the approval task is completed within the recommended time, a positive reward, such as +100 points, can be calculated based on the reward function; conversely, if the approval is delayed, a negative reward, such as -50 points, will be calculated.

[0062] After completing the assigned tasks, the updated knowledge graph can be updated to obtain the target knowledge graph. Features can then be extracted from the target knowledge graph to construct the target state vector. Subsequently, quadruples can be constructed based on the state vector, task allocation strategy, reward value, and target state vector, and stored in the replay pool. New quadruples can be continuously collected to periodically or incrementally update the policy network parameters (i.e., by sampling the data in the replay pool at a preset period, obtaining sampled data, and updating the policy model based on the sampled data and the exploration strategy (such as ε-greedy, which sets a probability threshold ε to randomly decide whether to take the known optimal action or try a randomly selected action in order to discover a better strategy). This enhances the adaptability and learning ability of the policy and realizes continuous iteration for the optimization of the approval process.

[0063] In order to accurately redetermine the task allocation strategy, in the task allocation strategy determination method provided in Embodiment 1 of this application, when the task allocation strategy is prohibited from execution, a review process is triggered. The review process is used to review the task allocation strategy and obtain the review result; based on the review result, the task allocation strategy is redetermined.

[0064] In this embodiment of the invention, a user appeal portal can be provided. If a user does not approve of the decision (task allocation strategy), they can refuse to execute the task allocation strategy and trigger a review process, effectively ensuring the rationality of the decision and the compliance of the process. Subsequently, the task allocation strategy can be re-checked through manual review or a decision-making mechanism to ensure the rationality and security of the strategy. For example, the reviewer (such as a department head or senior approver) can be automatically notified to conduct a manual review, or another layer of decision-making model (such as a rule-based expert system) can be automatically invoked for a second decision. The final review result of the reviewer or mechanism on the task allocation strategy is recorded, and the task allocation strategy is re-determined based on the review result.

[0065] The task allocation strategy determination method provided in this application solves the shortcomings of existing static rules and predictive routing methods in terms of flexibility, collaboration, dynamic adaptability and global optimization by dynamically perceiving the global state, using graph representation to represent complex relationships, and learning the optimal task allocation strategy online through reinforcement learning. This improves the execution efficiency and resource utilization efficiency of business approval processes.

[0066] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0067] Example 2

[0068] This application also provides a task allocation strategy determination apparatus. It should be noted that the task allocation strategy determination apparatus of this application can be used to execute the task allocation strategy determination method provided in this application. The task allocation strategy determination apparatus provided in this application will be described below.

[0069] According to an embodiment of this application, an apparatus for implementing the above-described method for determining the task allocation strategy is also provided. Figure 3 This is a schematic diagram of an apparatus for determining an optional task allocation strategy according to an embodiment of this application, such as... Figure 3 As shown, the device for determining the task allocation strategy may include: an acquisition unit 30, an update unit 31, a construction unit 32, and a processing unit 33.

[0070] The acquisition unit 30 is used to acquire the characteristics of the approval process and the characteristics of the approval tasks when it is detected that the approval tasks in the approval process need to be assigned, and to determine the approval tasks as tasks to be assigned.

[0071] Update unit 31 is used to update the preset knowledge graph based on all features to obtain the updated knowledge graph;

[0072] Construction unit 32 is used to extract target features associated with the task to be assigned from the updated knowledge graph, and to construct a state vector based on all target features;

[0073] Processing unit 33 is used to process the state vector using a reinforcement learning engine to obtain a task allocation strategy.

[0074] The task allocation strategy determination device provided in this application embodiment can, through the acquisition unit 30, acquire the features of the approval process and the features of the approval tasks when it is detected that the approval tasks in the approval process need to be allocated, and determine the approval tasks as tasks to be allocated. The update unit 31 can update the preset knowledge graph based on all features to obtain the updated knowledge graph. The construction unit 32 can extract the target features associated with the tasks to be allocated from the updated knowledge graph, and construct a state vector based on all target features. The processing unit 33 can process the state vector using a reinforcement learning engine to obtain the task allocation strategy.

[0075] Optionally, the device for determining the task allocation strategy further includes: a first acquisition module, used to acquire multiple elements in the approval workflow before acquiring the features of the approval process and the features of the approval task, and to determine each element as an entity and to determine whether there is a dependency relationship between each entity, wherein each entity has an attribute value; and a first construction module, used to construct a preset knowledge graph based on all entities, all attribute values ​​and all dependencies.

[0076] Optionally, the acquisition unit 30 includes: a second acquisition module, used to acquire the status characteristics of the approval process and the characteristics of each task approval node in the approval process, the characteristics including at least: node load characteristics, node status characteristics and processing characteristics for historical approval tasks; and a third acquisition module, used to acquire the task characteristics of the approval task.

[0077] Optionally, the update unit 31 includes: a first update module, used to update each entity of the preset knowledge graph to obtain the updated entity, and update the attribute values ​​corresponding to each entity to obtain the updated attribute values; a second update module, used to update the dependencies of the preset knowledge graph to obtain the updated dependencies; and a first determination module, used to determine the updated knowledge graph based on all updated entities, all updated attribute values, and all updated dependencies.

[0078] Optionally, the task allocation strategy determination device further includes: a second determination module, used to determine multiple reward targets before processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, wherein each reward target corresponds to a reward index; a second construction module, used to construct positive rewards and negative rewards based on each reward index; a first allocation module, used to assign preset weights to each positive reward and each negative reward; and a third construction module, used to construct a reward function based on all positive rewards, all negative rewards, and all preset weights, wherein the reward function is used to update the policy model in the reinforcement learning engine.

[0079] Optionally, the task allocation strategy determination device further includes: a second allocation module, used to allocate tasks to be allocated based on the task allocation strategy after processing the state vector with a reinforcement learning engine to obtain the task allocation strategy, and obtain the allocation result; a first execution module, used to execute the tasks to be allocated according to the allocation result and obtain the execution result of the tasks to be allocated; a first calculation module, used to calculate the reward value of the task allocation strategy based on the execution result and the reward function; and a fourth construction module, used to construct a quadruple based on the state vector, the task allocation strategy, the reward value and the target state vector, and store the quadruple in the replay pool, wherein the target state vector is constructed by extracting features from the target knowledge graph, the target knowledge graph is obtained by updating the updated knowledge graph, and the sampled data is obtained by sampling the data in the replay pool at a preset period, and the strategy model is updated based on the sampled data.

[0080] Optionally, the task allocation strategy determination device further includes: a first triggering module, used to trigger a review process when the task allocation strategy is prohibited from execution before executing the task to be allocated according to the allocation result, wherein the review process is used to review the task allocation strategy and obtain a review result; and a third determination module, used to redetermine the task allocation strategy based on the review result.

[0081] The aforementioned task allocation strategy determination device may further include a processor and a memory. The aforementioned acquisition unit 30, update unit 31, construction unit 32, processing unit 33, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0082] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, a reinforcement learning engine can be used to process the state vectors to obtain a task allocation strategy.

[0083] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0084] It should be noted that the acquisition unit 30, update unit 31, construction unit 32, and processing unit 33 mentioned above correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0085] Example 3

[0086] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced with a mobile terminal or an electronic device, etc.

[0087] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0088] In this embodiment, the computer terminal described above can execute the program code for the following steps in the method for determining the task allocation strategy: when it is detected that an approval task in the approval process needs to be allocated, the characteristics of the approval process and the characteristics of the approval task are obtained, and the approval task is determined as the task to be allocated; based on all the characteristics, the preset knowledge graph is updated to obtain the updated knowledge graph; target features associated with the task to be allocated are extracted from the updated knowledge graph, and a state vector is constructed based on all the target features; the state vector is processed using a reinforcement learning engine to obtain the task allocation strategy.

[0089] Optionally, the computer terminal described above can execute the program code for the following steps in the method for determining the task allocation strategy: obtaining multiple elements in the approval workflow and identifying each element as an entity, determining whether there is a dependency relationship between each entity, wherein each entity has an attribute value; and constructing a preset knowledge graph based on all entities, all attribute values, and all dependencies.

[0090] Optionally, the computer terminal described above can execute the program code for the following steps in the method for determining the task allocation strategy: obtaining the status characteristics of the approval process and obtaining the characteristics of each task approval node in the approval process, the characteristics including at least: node load characteristics, node status characteristics and processing characteristics for historical approval tasks; obtaining the task characteristics of the approval task.

[0091] Optionally, the aforementioned computer terminal may execute the program code for the following steps in the method for determining the task allocation strategy: updating each entity in the preset knowledge graph to obtain the updated entity, and updating the attribute values ​​corresponding to each entity to obtain the updated attribute values; updating the dependencies of the preset knowledge graph to obtain the updated dependencies; and determining the updated knowledge graph based on all updated entities, all updated attribute values, and all updated dependencies.

[0092] Optionally, the aforementioned computer terminal can execute program code for the following steps in the method for determining the task allocation strategy: determining multiple reward objectives, wherein each reward objective corresponds to a reward indicator; constructing positive rewards and negative rewards based on each reward indicator; assigning preset weights to each positive reward and each negative reward; and constructing a reward function based on all positive rewards, all negative rewards, and all preset weights, wherein the reward function is used to update the policy model in the reinforcement learning engine.

[0093] Optionally, the aforementioned computer terminal can execute the program code for the following steps in the method for determining the task allocation strategy: allocating tasks to be allocated based on the task allocation strategy to obtain allocation results; executing tasks to be allocated according to the allocation results and obtaining the execution results of the tasks to be allocated; calculating the reward value of the task allocation strategy based on the execution results and the reward function; constructing a quadruple based on the state vector, task allocation strategy, reward value, and target state vector, and storing the quadruple in the replay pool, wherein the target state vector is constructed by extracting features from the target knowledge graph, the target knowledge graph is obtained by updating the updated knowledge graph, and the strategy model is updated based on the sampled data obtained by sampling the data in the replay pool at a preset period.

[0094] Optionally, the computer terminal described above can execute the program code for the following steps in the method for determining the task allocation strategy: when the task allocation strategy is prohibited from execution, trigger a review process, wherein the review process is used to review the task allocation strategy and obtain a review result; based on the review result, redetermine the task allocation strategy.

[0095] Optionally, Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 4As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0096] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the task allocation strategy determination method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned task allocation strategy determination method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The processor can access the information and application programs stored in the memory via the transmission device to execute the steps described above in the method for determining the task allocation strategy.

[0098] The embodiments of this application provide a scheme for determining a task allocation strategy. By integrating reinforcement learning and dynamic knowledge graph technologies, it can intelligently analyze the real-time status and global resource distribution of the approval process, and realize collaborative optimization decision-making from task to approver and path branch. Through continuous learning and dynamic adjustment, this scheme improves the adaptability of the process and the utilization rate of resources, ensuring that the approval is both efficient and compliant. This solves the technical problem in related technologies that the task allocation strategy cannot be flexibly determined, which reduces the execution efficiency of the approval process.

[0099] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be terminal devices such as smartphones, tablets, PDAs, and mobile internet devices (MIDs). Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0100] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0101] Example 4

[0102] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the task allocation strategy determination method provided in Embodiment 1.

[0103] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0104] This application also provides a computer program product, which, when executed on a data processing device, is suitable for performing steps of a method for determining a task allocation strategy.

[0105] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0106] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0111] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining a task allocation strategy, characterized in that, include: When it is detected that approval tasks in the approval process need to be assigned, the characteristics of the approval process and the characteristics of the approval tasks are obtained, and the approval tasks are identified as tasks to be assigned. Based on all the aforementioned features, the preset knowledge graph is updated to obtain the updated knowledge graph; Extract target features associated with the task to be assigned from the updated knowledge graph, and construct a state vector based on all the target features; The state vector is processed using a reinforcement learning engine to obtain a task allocation strategy.

2. The method for determining the task allocation strategy according to claim 1, characterized in that, Before obtaining the characteristics of the approval process and the characteristics of the approval task, the process further includes: Multiple elements in the approval workflow are obtained, and each element is identified as an entity. It is determined whether there is a dependency relationship between each entity, wherein each entity has a corresponding attribute value. The preset knowledge graph is constructed based on all the entities, all the attribute values, and all the dependencies.

3. The method for determining the task allocation strategy according to claim 1, characterized in that, The steps for obtaining the characteristics of the approval process and the characteristics of the approval task include: Obtain the status characteristics of the approval process and the characteristics of each task approval node in the approval process. The characteristics include at least: node load characteristics, node status characteristics, and processing characteristics for historical approval tasks. Obtain the task characteristics of the approval task.

4. The method for determining the task allocation strategy according to claim 1, characterized in that, The steps for updating the preset knowledge graph based on all the aforementioned features to obtain the updated knowledge graph include: Each entity in the preset knowledge graph is updated to obtain the updated entity, and the attribute values ​​corresponding to each entity are updated to obtain the updated attribute values. The dependencies of the preset knowledge graph are updated to obtain the updated dependencies; The updated knowledge graph is determined based on all the updated entities, all the updated attribute values, and all the updated dependencies.

5. The method for determining the task allocation strategy according to claim 1, characterized in that, Before processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, the process also includes: Multiple reward objectives are defined, wherein each reward objective corresponds to a reward metric; Based on each of the aforementioned reward metrics, construct positive and negative rewards; Assign a preset weight to each positive reward and each negative reward; A reward function is constructed based on all the positive rewards, all the negative rewards, and all the preset weights, wherein the reward function is used to update the policy model in the reinforcement learning engine.

6. The method for determining the task allocation strategy according to claim 1, characterized in that, After processing the state vector using a reinforcement learning engine to obtain the task allocation strategy, the process further includes: Based on the task allocation strategy, the tasks to be allocated are allocated to obtain the allocation results; Execute the task to be assigned based on the allocation result, and obtain the execution result of the task to be assigned; Based on the execution result and the reward function, calculate the reward value of the task allocation strategy; Based on the state vector, the task allocation strategy, the reward value, and the target state vector, a four-tuple is constructed and stored in the replay pool. The target state vector is constructed by extracting features from the target knowledge graph, which is obtained by updating the updated knowledge graph. Sampled data is obtained by sampling the data in the replay pool at a preset period, and the strategy model is updated based on the sampled data.

7. The method for determining the task allocation strategy according to claim 6, characterized in that, Before executing the task to be assigned based on the allocation result, the process also includes: If the task allocation strategy is prohibited from execution, a review process is triggered, wherein the review process is used to review the task allocation strategy and obtain a review result; Based on the review results, the task allocation strategy is redefined.

8. A device for determining a task allocation strategy, characterized in that, include: The acquisition unit is used to acquire the characteristics of the approval process and the characteristics of the approval task when it is detected that the approval task in the approval process needs to be assigned, and to determine the approval task as a task to be assigned. The update unit is used to update the preset knowledge graph based on all the features to obtain the updated knowledge graph. A construction unit is used to extract target features associated with the task to be assigned from the updated knowledge graph, and to construct a state vector based on all the target features; The processing unit is used to process the state vector using a reinforcement learning engine to obtain a task allocation strategy.

9. A computer program product, characterized in that, The method includes a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for determining a task allocation strategy as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for determining the task allocation strategy according to any one of claims 1 to 7.

Images