Modeling methods, scheduling methods, smart devices and storage media for government affairs processes

Abstract

The present invention provides a modeling method, a scheduling method, an intelligent device and a storage medium for a government affairs process, and relates to the technical field of process scheduling management. The method divides the government affairs process into structures and obtains the correlation between the structures, so that a Petri net process model corresponding to the government affairs process can be established according to the mapping relationship and the correlation between the structures, thereby converting the process information of the non-standard and abstract government affairs process into a computer-recognizable model. At the same time, the status of various resources of the government affairs process is mapped by the library, the activities of the government affairs process are mapped by the transition, and the specific resource objects of the government affairs process are mapped by the token, so that the resource factors are focused on and the resource limitations of the government affairs process scheduling problem are reflected. After the above-mentioned mapping relationship is established, the model established by the method realizes the modeling description of the scheduling problem and formally reflects the status of the scheduling process, thereby providing a possibility for the subsequent optimal solution of the government affairs scheduling process.

Description

Technical Field

[0001] The present invention relates to the technical field of process scheduling management, and in particular to a modeling method, a scheduling method, an intelligent device and a storage medium for a government process. Background Art

[0002] At present, government affairs still use the resource preemptive scheduling method used in traditional work processes. For example, when the government receives a job, it will evaluate whether there are enough resources to perform the job. If there are, it will immediately preempt the corresponding resources to start the job. If not, it will need to wait until there are resources to perform the job. If multiple jobs require a certain resource at the same time, random preemption will be performed, and one party will occupy it while the other party waits.

[0003] Since the resource preemptive scheduling method only makes judgments based on local information and does not effectively utilize global information, it is difficult for the resource preemptive scheduling method to obtain a global optimal solution, that is, it is difficult to obtain an optimal scheduling plan.

[0004] The following example illustrates the defects of resource preemptive scheduling:

[0005] The Housing Security Department of a certain Ministry of Housing and Urban-Rural Development has five sub-institutions, namely, the Business Acceptance Department, the Comprehensive Business Department, the Security Housing Management Department, the Finance Department, and the Management Committee, which provide government services related to housing security to the public. The Business Acceptance Department is directly open to the public for business acceptance. The remaining four departments handle business. The security department adopts a traditional working model, and each department handles business one by one in the order of priority, that is, what comes first is done, and what comes later is done later. In the process of handling business, due to limited human resources, problems like this often arise (assuming that each department can only undertake one task at a certain moment):

[0006] 1) If Figure 3 As shown in the figure, the Comprehensive Business Department has two tasks to be processed. After the first task is processed, it will be sent to the Finance Department, and after the second task is processed, it will be sent to the Affordable Housing Management Department. At this time, the Finance Department is continuously busy and the tasks need to wait, while the Affordable Housing Management Department is continuously idle and the tasks can be processed directly. If the Comprehensive Processing Department processes the second task first and then the first task, the time for the second task will be effectively shortened compared to the original order.

[0007] 2) If Figure 4As shown in the figure, the Housing Security Department has a 10-hour business to be processed, and the General Business Department has another business that will be sent to the Housing Security Department in one hour and processed in one hour. Both businesses will be sent to the Finance Department, each of which will take 1 hour to process. If the first business is processed first, the first business will take 11 hours, and the second business will take 11 hours. If the second business is processed first, the first business will take 12 hours, and the first business will take 2 hours. At the same time, the Finance Department can complete the work 1 hour ahead of schedule.

[0008] From the above cases, we can see that due to limited resources, unnecessary waiting may occur during the business processing, delaying the processing time. Therefore, it is necessary to optimize the scheduling method of government affairs process, control the status of each department at a macro level, and reasonably arrange the work of each department according to the needs, so as to shorten or even eliminate this unnecessary waiting, improve the utilization rate of government resources, and thus improve government work efficiency and public satisfaction. Summary of the invention

[0009] The problem solved by the present invention is that most of the existing process modeling solutions are aimed at process reengineering or auxiliary process design, and ignore the limitations and key factors of process scheduling problems.

[0010] To solve the above problems, a first aspect of an embodiment of the present invention provides a modeling method for government affairs processes based on Petri nets, comprising:

[0011] Structuralize government affairs processes and obtain the correlation between structures;

[0012] A Petri net process model of the government affairs process is established according to the mapping relationship between the structures of the government affairs process and the objects of the Petri net and the association between the structures; wherein the objects include places, transitions and tokens, the places map the states of the resources of the government affairs process, the transitions map the activities of the government affairs process, and the tokens map the specific resource objects of the government affairs process;

[0013] A token that maps a resource object that is globally shared and cyclically used is obtained, the token that maps the resource object that is globally shared and cyclically used is defined as a resource token, and a place corresponding to the resource token is defined as a resource place.

[0014] Furthermore, the establishing of the Petri net process model of the government affairs process according to the mapping relationship between each structure of the government affairs process and the Petri net object and the association between each structure includes:

[0015] Establishing an initial Petri net model of the government affairs process in a top-down manner according to the mapping relationship and the association between the structures;

[0016] Fill in specific attribute values ​​for each object in the Petri net initial model according to the mapping relationship.

[0017] Furthermore, the Petri net is a timed transition Petri net, and the timed transition Petri net is defined as:

[0018] TTPN=(P, T, I, O, M0, D);

[0019] Where P is the set of places, T is the set of transitions, I is the input function, O is the output function, M0 is the initial identifier, D = (d1, d2, d3...d n ), Q is the delay set of all transitions; the delay of transitions maps the time consumption of the activities of the government process.

[0020] Furthermore, the transition includes an initial transition and an end transition, the delay between the initial transition and the end transition is 0, the initial transition maps the start of the process instance of the government process, and the end transition maps the end of the process instance of the government process.

[0021] A second aspect of the present invention provides a method for scheduling government affairs processes, comprising:

[0022] Defining states of the scheduling process of the government affairs process, wherein the set of states is the state set;

[0023] Modeling the scheduling process of the government affairs process as a Markov decision process, wherein the scheduling process modeled as a Markov decision process includes an agent, and the agent observes the Petri net process model established by the modeling method of the government affairs process based on Petri nets as described above to obtain the state;

[0024] The optimal solution for the scheduling process of the government affairs process is solved through a deep reinforcement learning algorithm to obtain the optimal scheduling plan.

[0025] Furthermore, modeling the scheduling process of the government affairs process as a Markov decision process includes:

[0026] Defining the state of the scheduling process, the state at least includes the state of resources of the Petri net process model, the state of the process, the state of the process instance, the time state and the scheduling state, and the set of the states is the state set of the scheduling process;

[0027] Constructing an action set of the scheduling process, wherein the action set consists of two actions: scheduling and not scheduling;

[0028] A reward function and a decay factor for the scheduling process are determined.

[0029] Furthermore, modeling the scheduling process of the government affairs process as a Markov decision process also includes:

[0030] Obtaining the resource status and process instance status of the Petri net process model at time t, and obtaining a list of schedulable activity instances at time t according to the resource status and process instance status;

[0031] Traversing the list of schedulable activity instances through the preset decision activity instance selection mechanism, determining the activity to be decided, and taking the activity to be decided as a component of the state at time t;

[0032] Get the action output by the agent, execute the action output by the agent to generate the state at time t+1, and send a reward to the agent at the same time.

[0033] Furthermore, the method of solving the optimal solution of the scheduling process of the government affairs process by using a deep reinforcement learning algorithm includes:

[0034] Solve the optimal solution of the scheduling process of the government affairs process through DDQN;

[0035] The optimal solution of the scheduling process of solving the government affairs process by DDQN includes:

[0036] Determine the input parameters of the DDQN, the input parameters include the number of iterations T, the state feature dimension N, the action set A, the step size α, the decay factor γ, the exploration rate ∈, the current Q network Q, the target Q network Q′, the number of samples of batch gradient descent m, and the target Q network parameter update frequency C;

[0037] Randomly initialize the values ​​Q corresponding to all states and actions, randomly initialize all parameters w of the current Q network, initialize the parameters w′=w of the target Q network Q′, and clear the set D of experience playback;

[0038] Perform iteration and output the iteration result after the iteration is terminated;

[0039] Obtaining Q network parameters output by the DDQN;

[0040] The iterative process includes:

[0041] Step 1: Initialize the state s of the scheduling process to the first state of the current state sequence, and obtain its feature vector φ(s);

[0042] Step 2: Use φ(s) as input in the Q network to obtain the Q value output corresponding to all actions of the Q network, and use the ∈-greedy method to select the corresponding action a in the current Q value output s ;

[0043] Step 3: Execute the current action a in state ss , get the feature vector φ(s′) and reward r corresponding to the new state s′, whether the state is terminated is_end;

[0044] Step 4: Substitute {φ(s), a s , r, φ(s′), is_end} This five-tuple is stored in the experience replay set D;

[0045] Step 5: s = s′;

[0046] Step 6: Sample m samples from the experience playback set D Calculate the current target Q value yj:

[0047]

[0048] in, The output value of is a′(a′∈A), and a′ is equal to the action set A that makes Q(φ(s′ j ), a, w) value is the largest action a max ;

[0049] Step 7: Use mean square error loss function Update all parameters W of the Q network through gradient backpropagation of the neural network;

[0050] Step 8: If T%C=1, update the target Q network parameter w′=w;

[0051] Step 9: If s′ is a terminal state, the current round of iteration is completed, otherwise go to step 2.

[0052] A third aspect of the present invention provides an intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the Petri net-based government process modeling method or the government process scheduling method as described above is implemented.

[0053] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the modeling method of government affairs processes based on Petri nets as described above or the scheduling method of government affairs processes as described above.

[0054] The beneficial effects of the present invention are as follows: by structurally dividing the government affairs process and obtaining the correlation between the structures, after determining the mapping relationship between the government affairs process and the objects of the Petri net, a Petri net process model corresponding to the government affairs process can be established according to the mapping relationship and the correlation between the structures, thereby converting the process information of the non-standard and abstract government affairs process into a computer-recognizable model. At the same time, the status of various resources of the government affairs process is mapped by the library, the activities of the government affairs process are mapped by the transition, and the specific resource objects of the government affairs process are mapped by the token, thereby focusing on the resource factors and reflecting the resource limitations of the government affairs process scheduling problem. Since the government affairs process scheduling problem focuses on the resource objects that are globally shared and cyclically used, the resource objects that are globally shared and cyclically used are also mapped by resource tokens. After the above-mentioned mapping relationship is established, the model established by this method realizes the modeling description of the scheduling problem and formally reflects the status of the scheduling process, thereby providing a possibility for the subsequent optimal solution of the government affairs scheduling process. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of a method for modeling government affairs processes based on Petri nets according to an embodiment of the present invention;

[0056] Figure 2 A flowchart of a method for scheduling a government affairs process according to an embodiment of the present invention;

[0057] Figure 3 A business processing sequence diagram of the integrated business department of an embodiment of the present invention;

[0058] Figure 4 This is a business processing sequence diagram of the affordable housing management department of an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of an object of a Petri net according to an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of a model of approval activities in an embodiment of the present invention;

[0061] Figure 7 A schematic diagram of a sub-process of an embodiment of the present invention;

[0062] Figure 8 A schematic diagram of a sequential activity model according to an embodiment of the present invention;

[0063] Fig. 9 A schematic diagram of a model of concurrent activities and forking according to an embodiment of the present invention;

[0064] Fig.10 A schematic diagram of a concurrent activity and merging model according to an embodiment of the present invention;

[0065] Fig.11 A schematic diagram of a selection activity or bifurcation model according to an embodiment of the present invention;

[0066] Fig.12 A schematic diagram of a selection activity or a merging model according to an embodiment of the present invention;

[0067] Fig.13 A schematic diagram of a cyclic activity model according to an embodiment of the present invention;

[0068] Fig.14 This is a deadlock example diagram of an embodiment of the present invention;

[0069] Fig.15 A schematic diagram of a reinforcement learning framework of a Markov decision process according to an embodiment of the present invention;

[0070] Fig.16 4 is a structural diagram of a smart device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0072] The term "comprising" and any variations thereof in the specification and claims of the present invention and the above drawings are intended to cover non-exclusive inclusions. For example, a process, method or system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0073] The terms "first", "second" and "third" described in this specification are only used to distinguish devices / components / sub-components / parts, etc., and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features such as "first", "second" and "third" may explicitly or implicitly indicate that at least one of the features is included, unless otherwise clearly and specifically defined, and the meaning of "plurality" is at least two, for example two, three, etc. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0074] Since the amount of calculation required to find the optimal solution to the scheduling problem is large, the optimal solution to the scheduling problem of government affairs processes must be solved by computer. Therefore, it is necessary to establish a corresponding computer model for the government affairs process to facilitate computer recognition. However, most of the existing process modeling solutions are aimed at process reengineering or auxiliary process design. These solutions are mostly used for theoretical research and ignore the limitations and key factors of process scheduling problems, such as resource constraints and global loop shared resource objects.

[0075] like Figure 1 As shown, the first aspect of the embodiment of the present invention provides a modeling method of government affairs process based on Petri net, including:

[0076] S101: Structurally divide the government affairs process and obtain the correlation between the structures.

[0077] The basis for establishing a process model based on Petri nets is to have a comprehensive understanding of the government affairs process itself. The understanding of the government affairs process should start from the five basic elements of the government affairs process, understand the operations that need to be performed to implement the government affairs process, the conditions that each operation needs to meet, the subject of the operation execution, the logical relationship between the operations, and the rules that control each operation, such as the time constraints for each operation.

[0078] Therefore, after having sufficient understanding of the government affairs process, the government affairs process can be structurally divided. By structurally dividing the government affairs process, the non-standard and abstract government affairs process information can be preliminarily standardized. Therefore, when modeling the government affairs process, the various structures of the process can be mapped to the objects of the Petri net, and then the corresponding Petri net process model can be constructed through the association between the structures.

[0079] In this embodiment, the model can be established in two ways. The first is to directly divide the government affairs process into the activity level, and then determine the relevance of each activity, and establish the initial Petri net model according to the mapping relationship between each structure of the government affairs process and the object of the Petri net; Figure 7 As shown, the second is to first establish a global model based on the overall government affairs process, clarify the overall goal and main structure of the government affairs process, and on the basis of this model, analyze and refine the lower network one by one, and for each Petri net transition, determine whether it needs to be split and refined. If necessary, establish a corresponding sub-process model to replace the original transition. It can be understood that the government affairs process is divided into multiple sub-processes, and then determine whether the sub-processes can continue to be divided, analyze and refine the lower network activity layer one by one, and establish the final Petri net initial model. Among them, this embodiment preferably adopts the second method to establish the model.

[0080] When the initial Petri net model is established, it is necessary to fill specific attribute values ​​for each object in the initial Petri net model according to the mapping relationship, such as approval activities and approval personnel, so as to establish a Petri net process model corresponding to the specific process.

[0081] S102: Establishing a Petri net process model of the government process based on the mapping relationship between the structures of the government process and the objects of the Petri net and the association between the structures; wherein the objects include places, transitions and tokens, the places map the states of the resources of the government process, the transitions map the activities of the government process, and the tokens map the specific resource objects of the government process.

[0082] In this embodiment, the mapping relationship between the various structures of the government affairs process and the objects of the Petri net is as follows: Place maps the status of various resources in the government affairs process, which can also be understood as the status of each stage of the government affairs process; Token in the place represents a specific resource object, such as staff and processing equipment; Transition is used to represent various activities of the process, such as approval activities in the approval process.

[0083] like Figure 5 As shown, Petri net is a process model. In this embodiment, the objects of Petri net include: place, transition, logical transition, and token.

[0084] Logical transition is a special form of transition, which is divided into logical AND transition and logical OR transition, including four types of transitions: AND fork, AND merge, OR fork, OR merge. AND fork transition means that after the transition is completed, two or more transition activities will be triggered at the same time. AND merge transition means that the triggering of this transition is based on the completion of two or more transitions. OR fork transition means that after the transition is completed, two or more transition activities may be triggered, but only one will be selected in the end. OR merge transition corresponds to it, indicating that the transition is triggered by one transition in a set of two or more transitions. In real-world process instances, logical transitions often exist in pairs.

[0085] After determining the above Petri net pairs, various types of activities in the process can be modeled and analyzed. Activity types include: sequential activities, concurrent activities, and selection activities.

[0086] like Figure 8 As shown, a sequential activity, an activity T is triggered after a preceding activity P1(f) (or start node), and a succeeding activity B(P3) (or end node) is triggered when the activity ends. Such an activity is called a sequential activity. Figure 6 This is a typical sequential activity model.

[0087] Concurrent activities refer to situations where two or more tasks occur at the same time, and there is a logical AND relationship between the tasks. The following two models are used to describe concurrent activities:

[0088] like Fig. 9As shown in the figure, the bifurcation model refers to a condition that triggers two or more transitions.

[0089] like Fig.10 As shown in the figure, the combined model means that the occurrence of one transition requires the completion of two or more transitions.

[0090] like Fig.11 and Fig.12 As shown, selection activity refers to selecting one of two or more tasks. There is a logical OR relationship between the tasks. Similar to concurrent activities, the OR fork model and the OR merge model are also used to describe the selection activity.

[0091] like Fig.13 As shown in the figure, a cyclic activity refers to an activity that needs to be performed repeatedly in a process. For example, the approval of various materials often involves cyclic activities, and unqualified materials need to be returned for modification and re-examination. In fact, a cyclic activity can be regarded as a modified selection activity. There is a logical or relationship between entering the cycle and taking the next step.

[0092] Optionally, establishing the Petri net process model of the government affairs process according to the mapping relationship between each structure of the government affairs process and the Petri net object and the association between each structure includes:

[0093] Establishing an initial Petri net model of the government affairs process in a top-down manner according to the mapping relationship and the association between the structures;

[0094] Fill in specific attribute values ​​for each object in the Petri net initial model according to the mapping relationship.

[0095] In this embodiment, the model can be established in two ways. The first is to directly divide the government affairs process into the activity level, and then determine the relevance of each activity, and establish the initial Petri net model according to the mapping relationship between each structure of the government affairs process and the object of the Petri net; Figure 7 As shown, the second is to first establish a global model based on the overall government affairs process, clarify the overall goal and main structure of the government affairs process, and on the basis of this model, analyze and refine the lower network one by one, and for each Petri net transition, determine whether it needs to be split and refined. If necessary, establish a corresponding sub-process model to replace the original transition. It can be understood that the government affairs process is divided into multiple sub-processes, and then determine whether the sub-processes can continue to be divided, analyze and refine the lower network activity layer one by one, and establish the final Petri net initial model. Among them, this embodiment preferably adopts the second method to establish the model.

[0096] When the initial Petri net model is established, it is necessary to fill specific attribute values ​​for each object in the initial Petri net model according to the mapping relationship, such as approval activities and approval personnel, so as to establish a Petri net process model corresponding to the specific process.

[0097] Optionally, the Petri net is a timed transition Petri net, and the timed transition Petri net (hereinafter referred to as TTPN for ease of description) is defined as:

[0098] TTPN=(P, T, I, O, M0, D);

[0099] Where P is the set of places, T is the set of transitions, I is the input function, O is the output function, M0 is the initial identifier, D = (d1, d2, d3...d n ), d n is the delay of transition, and D is the delay set of all transitions; the delay of transition maps the time consumption of the activities of the government process.

[0100] The operation rules of TTPN (Timed Transition Petri Net) are similar to those of ordinary Petri Nets. However, the start of transition requires that all inputs meet the conditions, that is, there are tokens in the input Place. Once the transition starts, the corresponding token of the input Place is removed. After the corresponding delay, the transition ends and the corresponding token is added to the output Place.

[0101] Optionally, the transition includes an initial transition and an end transition, the delay between the initial transition and the end transition is 0, the initial transition maps the start of the process instance of the government process, and the end transition maps the end of the process instance of the government process.

[0102] Since the government process scheduling system schedules activity instances in government process instances and needs to process multiple government process instances, in order to better define the start and end of the process instance, an initial transition and an end transition are added to the government process model. The delay of both transitions is 0, the input and output are both empty, and the transition does not change the state of the resources. After adding the initial transition, the end of the initial transition marks the start of the process instance, and the end of the end transition marks the end of the process instance. Before the end of the initial transition, all activity instances in the process instance cannot be started. After the end of the end transition, all activity instances in the process instance must have been completed.

[0103] Among them, process instances and activity instances represent specific activities or processes that include specific objects. A process includes multiple process instances, and a process instance includes multiple activity instances. Taking the reissuance of an ID card as an example, the process for Zhang San to reissue his ID card is a process instance. Assuming that the process for Zhang San to reissue his ID card includes activity one and activity two, then activity one and activity two are activity instances.

[0104] By limiting the Petri net to a timed transition Petri net, possible deadlock problems in multi-project scheduling can be avoided.

[0105] like Fig.14 As shown, deadlock is a special type of resource conflict, which is manifested in that the activities in process one and the activities in process two jointly occupy the resources required by each other, restrict each other, and ultimately cannot proceed.

[0106] However, the resource deadlock problem between multiple projects does not exist in the Petri net process model based on the timed transition Petri net modeling. When using the timed transition Petri net modeling, the occupation period of the same transition for different input resources is the same.

[0107] by Fig.14 For example, the occurrence of transition T1 can only start when both the common place P1 and the resource places W1 and W2 contain tokens. When the transition starts, all the tokens in the input places are removed until the transition ends and all the tokens are output. Therefore, in this example, there is no deadlock situation where T1 occupies the token of W1, T2 occupies the token of W2, and then T1 waits for the token of W2, and T2 waits for the token of W1. There is no situation where transitions occupy some resources and wait for some resources, thus indirectly avoiding the deadlock problem.

[0108] S103: Acquire a token that maps a resource object that is globally shared and cyclically used, define the token that maps the resource object that is globally shared and cyclically used as a resource token, and define a place corresponding to the resource token as a resource place.

[0109] Among them, since the research object of this application is the government process scheduling problem, the focus is on the globally shared and cyclically used resource objects, such as computers and people, rather than the specific resources involved in a specific process, such as approval documents. Figure 6As shown in the figure, this diagram describes an approval activity. The approval T can only be carried out when the approval document f arrives (Token f in P1) and the approver m is free (Token m in P2). When the approval is executed, P1 and P2 lose the corresponding Token at the same time. After the approval is completed, the output is generated (Token o appears in P3), and Token m in P2 appears again. In this approval process, resource m, that is, the approver participates in the activity but does not change, is a resource object that is globally shared and cyclically used, and is also an object that generates resource constraints for process scheduling. In order to distinguish this special resource from ordinary input and output resources, the corresponding tokens are named resource tokens and ordinary tokens respectively, and the corresponding places are divided into resource places and ordinary places.

[0110] In the application, after defining the library corresponding to the resource token as a resource library, the following steps are further included:

[0111] The self-loop structure formed by the transition and the resource library is replaced with a structural block with new attributes.

[0112] In the sequential activities, the transition forms a sequential structure of A→T→B with the ordinary place, and a self-circulating structure of C→T→C with the resource place. Figure 5 This is a typical sequential activity model. In fact, the loop structure will terminate after one loop, because the generation of transition T requires both resource tokens and ordinary tokens. Based on the purpose of easy understanding and analysis of the Petri net model, according to the equivalent replacement property of the Petri net, we construct a new Petri net structure block T′=C→T→C to replace the original transition process C→T→C, and obtain a new sequential activity model. Figure 5 After replacing the self-loop structure in the code with a structure block with a new attribute, Figure 6 The same operation is also performed for the subsequent activity types, which can simplify the Petri net.

[0113] By dividing the government affairs process into structures and obtaining the correlation between each structure, after determining the mapping relationship between the government affairs process and the objects of the Petri net, the Petri net process model corresponding to the government affairs process can be established according to the mapping relationship and the correlation between each structure, thereby converting the process information of the non-standard and abstract government affairs process into a computer-recognizable model. At the same time, the status of each resource of the government affairs process is mapped by the library, the activity of the government affairs process is mapped by the transition, and the specific resource object of the government affairs process is mapped by the token, thereby focusing on the resource factor and reflecting the resource limitation of the government affairs process scheduling problem. Since the government affairs process scheduling problem focuses on the resource objects that are globally shared and cyclically used, the resource token is also used to map the resource objects that are globally shared and cyclically used. After the above mapping relationship is established, the model established by this method realizes the modeling description of the scheduling problem and formally reflects the status of the scheduling process, thereby providing the possibility for the subsequent optimal solution of the scheduling process of the government affairs process.

[0114] like Figure 2 As shown, another embodiment of the present invention provides a method for scheduling a government affairs process, including:

[0115] S201: defining the status of the scheduling process of the government affairs process;

[0116] The status of the scheduling process includes any status information related to scheduling, such as the total amount of available resources, resource attributes, and information about the process instance being executed.

[0117] S202: Modeling the scheduling process of the government affairs process as a Markov decision process, wherein the scheduling process modeled as a Markov decision process includes an agent, and the agent observes the Petri net process model established by the modeling method of the government affairs process based on Petri net as described above to obtain the state.

[0118] Among them, the Markov decision process is a random control process, that is, for each time node, when the process is in a certain state s, the decision maker can take any action a allowed in this state, and then the next system state s′ will be randomly generated, and the corresponding reward will be fed back to the decision maker. Fig.15As shown, in this embodiment, the Markov decision process is a direct framework for a reinforcement learning problem that achieves a goal from interaction. The learner and the decision maker are called agents. Everything else that the agent interacts with is called the environment. The agent continuously selects actions, and the environment responds accordingly and shows the new state to the agent. The environment also gives a numerical value as feedback or reward. The agent's goal is to maximize this feedback value by selecting different actions.

[0119] It can be represented by a five-tuple (S, A, P, R, γ):

[0120] S represents the set of states (of finite dimension).

[0121] A represents the set of actions in the decision-making process (of finite dimensions).

[0122] P describes the state transition matrix

[0123] R represents the reward function, R(s) describes the expected (immediate) reward for performing an action in state s, R(s, a) = E[R t+1 |S t =s,A t =a].

[0124] γ represents the discounted factor, γ∈[0,1].

[0125] Optionally, modeling the scheduling process of the government affairs process as a Markov decision process includes:

[0126] Defining the state of the scheduling process, the state at least includes the state of resources of the Petri net process model, the state of the process, the state of the process instance, the time state and the scheduling state, and the set of the states is the state set of the scheduling process;

[0127] Constructing an action set of the scheduling process, wherein the action set consists of two actions: scheduling and not scheduling;

[0128] A reward function and a decay factor for the scheduling process are determined.

[0129] In the process of scheduling government affairs processes, the status mainly includes the following parts:

[0130] The status of government resources includes at least the total amount of various types of government resources, various attributes of resources, the available time period of each resource, whether it is idle at time t, and the activity instance information occupying the resource at time t.

[0131] The status of the government affairs process includes at least the government affairs process information run by the government, the structure of the government affairs process (i.e., the logical association information between government affairs process activities), the list of resources required by government affairs process activities, and the probability distribution of the running time of government affairs activities.

[0132] The status of the government process instance, which at least includes the information of the government process instance being executed by the government at time t, a list of the number of various types of government process instances, the execution status of each process instance, the status of the government process activity instance of the government process instance sub-unit, whether it is executable, and whether it has been executed.

[0133] Time status, which includes the time information corresponding to the government time t, whether time t is working time, and the natural time period of time t;

[0134] The scheduling status includes at least the status of the scheduling process, the scheduling stage at time t, and the scheduling decision flags of each activity instance at time t.

[0135] Optionally, modeling the scheduling process of the government affairs process as a Markov decision process further includes:

[0136] Obtaining the resource status and process instance status of the Petri net process model at time t, and obtaining a list of schedulable activity instances at time t according to the resource status and process instance status;

[0137] Traversing the list of schedulable activity instances through the preset decision activity instance selection mechanism, determining the activity to be decided, and taking the activity to be decided as a component of the state at time t;

[0138] Obtain the action output by the agent, execute the action output by the agent to generate the state at time t+1, and send a reward to the agent at the same time.

[0139] In this embodiment, due to the uncertainty of the process, the state of the scheduling process is changing, and the state set of the scheduling process is an infinite set. The states at different times are different, the idle resources in different states are different, and the activity instances to be decided are unique. Therefore, the activity instances to be decided in the historical state are not learnable, so it is necessary to pre-set a decision activity instance selection mechanism, through which the list of schedulable activity instances is traversed, the activity to be decided is selected, and the activity to be decided is used as a component of the corresponding state.

[0140] In the process of government process scheduling, the action set contains only two actions, scheduling and not scheduling. The scheduling action is to schedule the corresponding pending decision activity. The pending decision activity will be executed after scheduling and immediately occupy the corresponding resources. Not scheduling means not scheduling the corresponding pending decision activity and directly entering the next state. The decision activity instance selection mechanism will continuously remove the pending decision activities that are not scheduled, that is, not scheduling in the next state until the government's resource status changes, ensuring that the decision process will traverse all lists of schedulable activity instances.

[0141] In this embodiment, the state transition matrix of the process is complex and cannot be specifically expressed. Similar to many Markov decision processes, the next state s′ that state s enters after executing action a is given by the environment.

[0142] The reward function of the government process is set according to different optimization goals, which can be set as sparse rewards or formal rewards. The total reward of the entire scheduling process should be positively correlated with the degree of target optimization. The more total rewards a scheduling process obtains, the higher the degree of target optimization.

[0143] The decay factor expresses the importance attached to the future. The smaller the decay factor, the less importance is attached to the future, and the closer it is to 1, the more importance is attached to the future. In the government process scheduling problem, the decay factor is expressed as the importance attached to the activity instances that have not yet entered the schedulable state during scheduling.

[0144] So far, we have modeled the government process scheduling problem as a Markov decision problem.

[0145] The process of solving the Markov decision process is to determine the optimal strategy, which enables the agent to obtain the maximum total reward G by executing the strategy. t A simple definition is:

[0146]

[0147] But if the above formula is to be defined, the return sequence must be finite, that is

[0148]

[0149] Where T is the end time point.

[0150] If there is no end point, the return may be infinite. In order to obtain a limited return, future benefits can be discounted, which is why the decay factor exists. Define discounted return:

[0151]

[0152] We generally use π to represent a strategy, and π(a|s) to represent the probability of taking action a in a state s. The public representation is:

[0153] π(a|s)=P(At=a|St=s)

[0154] The policy completely defines all the actions of the agent in all states and their probabilities.

[0155] When the agent adopts strategy π, the cumulative return follows a distribution, and the expected value of the cumulative return at state s is defined as the state value function:

[0156]

[0157] Accordingly, the state-action value function is:

[0158]

[0159] The state-action value function represents the expected return of performing action a in state s when executing policy π, also known as Q-value.

[0160] The purpose of calculating the state value function is to build a learning algorithm to obtain the optimal strategy from the data. Each strategy corresponds to a state value function, and the optimal strategy naturally corresponds to the optimal state value function.

[0161] S203: Solve the optimal solution of the scheduling process of the government affairs process through a deep reinforcement learning algorithm to obtain the optimal scheduling plan.

[0162] The deep reinforcement learning algorithm uses a deep neural network to fit the value function in reinforcement learning end-to-end, which is equivalent to abstracting useful features from high-dimensional data as states and then iterating with reinforcement learning. However, the state of the scheduling process of government affairs processes is relatively complex. Theoretically, the state of the scheduling process of government affairs processes is infinite. Therefore, in this embodiment, a deep reinforcement learning algorithm is used to solve the optimal strategy for the scheduling process of government affairs processes.

[0163] Optionally, solving the optimal solution of the scheduling process of the government affairs process by using a deep reinforcement learning algorithm includes:

[0164] The optimal solution for the scheduling process of the government affairs process is solved by DDQN (Double Deep Q Network);

[0165] The optimal solution of the scheduling process of solving the government affairs process by DDQN includes:

[0166] Determine the input parameters of the DDQN, the input parameters include the number of iterations T, the state feature dimension N, the action set A, the step size α, the decay factor γ, the exploration rate ∈, the current Q network Q, the target Q network Q′, the number of samples of batch gradient descent m, and the target Q network parameter update frequency C; wherein the state is ultimately converted into a vector, and the state feature dimension is the dimension of the vector into which the state is converted;

[0167] Randomly initialize the values ​​Q corresponding to all states and actions, randomly initialize all parameters w of the current Q network, initialize the parameters w′=w of the target Q network Q′, and clear the set D of experience playback;

[0168] Perform iteration and output the iteration result after the iteration is terminated;

[0169] Obtaining Q network parameters output by the DDQN;

[0170] The iterative process includes:

[0171] Step 1: Initialize the state s′ of the scheduling process as the first state of the current state sequence, and obtain its feature vector φ(s);

[0172] Step 2: Use φ(s) as input in the Q network to obtain the Q value output corresponding to all actions of the Q network, and use the ∈-greedy method to select the corresponding action a in the current Q value output s ;

[0173] Step 3: Execute the current action a in state s s , get the feature vector φ(s′) and reward r corresponding to the new state s, whether the state is terminated is_end;

[0174] Step 4: Substitute {φ(s), a s , r, φ(s′), is_end} This five-tuple is stored in the experience replay set D;

[0175] Step 5: s = s′;

[0176] Step 6: Sample m samples from the experience playback set D Calculate the current target Q value yj:

[0177]

[0178] in, The output value of is a′(a′∈A), and a′ is equal to the action set A that makes Q(φ(s′ j ), a, w) value is the largest action a max ;

[0179] Step 7: Use mean square error loss function Update all parameters w of the Q network through gradient backpropagation of the neural network;

[0180] Step 8: If T%C=1, update the target Q network parameter w′=w;

[0181] Step 9: If s′ is a terminal state, the current round of iteration is completed, otherwise go to step 2.

[0182] By dividing the process into structures and obtaining the correlation between the structures, after determining the correspondence between each structure and the object of the Petri net, a Petri net process model corresponding to the process is established, so as to convert the process information of the non-standard and abstract government process into a computer-recognizable model. On the basis of computer recognition of the process, the scheduling process of the Petri net process model is modeled as a Markov decision process, and the states, actions and rewards of the scheduling process are formally defined. Then, the optimal solution of the Markov decision process is solved through a deep reinforcement learning algorithm, thereby generating an optimal scheduling plan.

[0183] In the application, the intelligent agent of the model can make intelligent scheduling decisions after training, and output a scheduling solution that can achieve better management goals than resource preemptive scheduling. In this process, a simulation environment that simulates the operation of government affairs processes can be built. The intelligent agent first interacts with the simulation environment to learn, and then is placed in the actual environment to interact with the actual environment and learn deeper knowledge that is more in line with the actual environment.

[0184] like Fig.16 As shown, another embodiment of the present invention provides an intelligent device 16, including a memory 161, a processor 160, and a computer program 162 stored in the memory 161 and executable on the processor 160, wherein the processor 160 implements the steps of the modeling method of the government affairs process based on Petri nets or the steps of the scheduling method of the government affairs process as described above when executing the computer program 162. Figure 1 S101 to S103 shown.

[0185] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electric carrier signals and telecommunication signals.

[0188] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0189] Although the disclosure is disclosed as above, the protection scope of the disclosure is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for scheduling government affairs processes, characterized in that: include: Define the status of the scheduling process of the government affairs process; Modeling the scheduling process of the government affairs process as a Markov decision process, wherein the intelligent agent observes the Petri net process model established by the modeling method of the government affairs process based on the Petri net to obtain the state; Solve the optimal solution of the scheduling process of the government affairs process through a deep reinforcement learning algorithm to obtain the optimal scheduling plan; The modeling method of the government affairs process based on Petri net includes: Structuralize government affairs processes and obtain the correlation between structures; A Petri net process model of the government affairs process is established according to the mapping relationship between the structures of the government affairs process and the objects of the Petri net and the association between the structures, wherein the objects include places, transitions and tokens, the places map the states of the resources of the government affairs process, the transitions map the activities of the government affairs process, and the tokens map the specific resource objects of the government affairs process; Acquire a token that maps a resource object that is globally shared and cyclically used, define the token that maps the resource object that is globally shared and cyclically used as a resource token, and define a place corresponding to the resource token as a resource place; The Petri net is a timed transition Petri net, which is defined as: TTPN=(P,T,I,O,M0,D); Where P is the set of places, T is the set of transitions, I is the input function, O is the output function, M0 is the initial identifier, D = (d1, d2, d3...d n ), D is the set of time delays of all transitions; the time delay of transitions maps the time consumption of the activities of the government affairs process; The transition includes an initial transition and an end transition, the delay of the initial transition and the end transition is 0, the initial transition maps the start of the process instance of the government process, and the end transition maps the end of the process instance of the government process.

2. The method for scheduling government affairs processes according to claim 1, characterized in that: The step of establishing the Petri net process model of the government affairs process according to the mapping relationship between the structures of the government affairs process and the objects of the Petri net and the association between the structures includes: Establishing an initial Petri net model of the government affairs process in a top-down manner according to the mapping relationship and the association between the structures; Fill the attribute value of each object in the Petri net initial model according to the mapping relationship.

3. The method for scheduling government affairs processes according to claim 1, characterized in that: Modeling the scheduling process of the government affairs process as a Markov decision process includes: Defining the state of the scheduling process, the state at least includes the state of resources of the Petri net process model, the state of the process, the state of the process instance, the time state and the scheduling state, and the set of the states is the state set of the scheduling process; Constructing an action set of the scheduling process, wherein the action set consists of two actions: scheduling and not scheduling; A reward function and a decay factor for the scheduling process are determined.

4. The method for scheduling government affairs processes according to claim 3, characterized in that: The step of modeling the scheduling process of the government affairs process as a Markov decision process further includes: Obtaining the resource status and process instance status of the Petri net process model at time t, and obtaining a list of schedulable activity instances at time t according to the resource status and process instance status; Traversing the list of schedulable activity instances through a preset decision activity instance selection mechanism, determining the activity to be decided, and taking the activity to be decided as a component of the state at time t; Obtain the action output by the agent, execute the action output by the agent to generate the state at time t+1, and send a reward to the agent at the same time.

5. The method for scheduling government affairs processes according to claim 3, characterized in that: The optimal solution for the scheduling process of the government affairs process by using a deep reinforcement learning algorithm includes: Solve the optimal solution of the scheduling process of the government affairs process through DDQN; The optimal solution of the scheduling process of solving the government affairs process by DDQN includes: Determine the input parameters of the DDQN, the input parameters include the number of iterations T, the state feature dimension N, the action set A, the step size α, the decay factor γ, the exploration rate ∈, the current Q network Q, the target Q network Q′, the number of samples of batch gradient descent m, and the target Q network parameter update frequency C; Randomly initialize the values ​​Q corresponding to all states and actions, randomly initialize all parameters w of the current Q network, initialize the parameters w′=w of the target Q network Q′, and clear the set D of experience playback; Perform iteration and output the iteration result after the iteration is terminated; Obtaining Q network parameters output by the DDQN; The iterative process includes: Step 1: Initialize the state s of the scheduling process to the first state of the current state sequence, and obtain its feature vector φ(s); Step 2: Use φ(s) as input in the Q network to obtain the Q value output corresponding to all actions of the Q network, and use the ∈-greedy method to select the corresponding action a in the current Q value output s ; Step 3: Execute the current action a in state s s , get the feature vector φ(s′) and reward r corresponding to the new state s′, whether the state is terminated is_end; Step 4: Substitute {φ(s), a s , r, φ(s′), is_end} This five-tuple is stored in the experience replay set D; Step 5: s = s′; Step 6: Sample m samples from the experience playback set D j=1,2.,,,m,calculate the current target Q value y j : in, The output value of is a′(a′∈A), and a′ is equal to the action set A that makes Q(φ(s′ i ), a, w) value is the largest action a max ; Step 7: Use mean square error loss function Update all parameters w of the Q network through gradient backpropagation of the neural network; Step 8: If T%C=1, update the target Q network parameter w′=w; Step 9: If s′ is a terminal state, the current round of iteration is completed, otherwise go to step 2.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the method for scheduling government affairs processes as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the scheduling method for government affairs processes as described in any one of claims 1 to 5 is implemented.

Images