A Multi-Task User Interaction Log Segmentation Method Based on Graph Embedding

Abstract

The present invention discloses a multi-task user interaction log segmentation method based on graph embedding, including: log preprocessing, behavior vector learning, log segmentation, and log slice clustering. First, the present invention extracts the behavioral attribute context and content context in the log and represents the user interaction log as a direct following graph; secondly, it enhances the weight of the direct following graph using the similarity of the behavioral context and learns the vector representation of the behavior based on the graph embedding algorithm; then, it calculates the similarity of its predecessors and successors using the behavior vector and segments the user interaction log according to the change of the similarity of predecessors and successors; finally, it clusters the segmented log slices, thereby improving the log segmentation effect and enhancing the segmentation accuracy.

Description

Technical Field

[0001] The present invention belongs to the fields of robotic process automation and process mining, and particularly relates to a multi-task user interaction log segmentation method based on graph embedding. Background Art

[0002] Robotic process automation technology can replace humans to complete some tasks with high repetition, large workload, and strong regularity. Usually, robotic process automation needs to go through the following three stages: (1) process task analysis and design, (2) process program implementation and deployment, and (3) process program monitoring and maintenance. Among them, in the process model analysis and design stage, it is necessary to analyze the operation behavior of users on the software interface with the help of user interaction logs, mine the process model of tasks from them, and determine the automatable behavior sequences.

[0003] User interaction logs are sequential logs that record the interaction behavior between users and application programs, and are used to describe the execution process of one or more tasks. According to the number of tasks and execution methods, user interaction logs can be roughly divided into three types: single-task execution logs, multi-task serial execution logs, and multi-task concurrent execution logs. The main difference between serial and concurrent execution lies in whether the execution process of one task is interfered by other tasks, which is manifested in the log as whether the behavior sequences of different tasks are interleaved. User interaction log segmentation aims to distinguish the tasks to which different behaviors belong, so as to extract the behavior sequences of the same task and reorganize them into logs with tasks as units (abbreviated as task logs), which helps the subsequent process modeling work.

[0004] Currently, researchers at home and abroad have done a lot of valuable research work on user interaction log segmentation. The existing methods are mainly based on the frequent-pattern identification technology, which can mine the frequently occurring behavior sequences from user interaction logs and use them as the basis for log segmentation. The existing methods have good effects on single-task execution logs and multi-task serial execution logs, but have poor effects on multi-task concurrent execution logs. This is because there are complex interleaving situations in the behavior sequences of multi-task concurrent execution logs, and there are multiple variants of the behavior sequences obtained by the frequent-pattern identification technology, which cannot accurately describe the execution process of tasks. In addition, the existing methods only solve the problem from the perspective of control flow and do not make full use of the content information of behaviors in user interaction logs, making it difficult to accurately describe the relationship between behaviors. Summary of the Invention

[0005] In view of the problem that the interleaving of user interaction logs caused by concurrent execution of tasks in a multi-task scenario leads to low accuracy of log segmentation, the present invention proposes a method for segmenting user interaction logs in a multi-task scenario based on graph embedding, which can improve the segmentation effect of logs with concurrent execution of multiple tasks.

[0006] The present invention provides a method for segmenting multi-task user interaction logs based on graph embedding, and the method includes the following steps:

[0007] (1) Collect user interaction logs generated by the interaction between users and application programs in a multi-task scenario

[0008] (2) On the basis of (1), perform data preprocessing work, including removing noise behaviors, and extracting behavior attribute context AC and behavior content context CC from the behavior context;

[0009] (3) Represent the user interaction logs as a direct following graph according to the direct following relationship between behaviors;

[0010] (4) On the basis of (3), perform a weight enhancement operation on the direct following graph using the behavior attribute context and content context;

[0011] (5) On the direct following graph with enhanced weights, use the graph embedding algorithm node2vec to learn the vector representation of behavior nodes;

[0012] (6) Calculate the precursor similarity of behaviors according to the behavior node vectors and the successor similarity

[0013] (7) On the basis of (6), determine the log segmentation points according to the change of behavior precursor and successor similarities, and segment the user interaction logs;

[0014] (8) Given the number of task categories, cluster the log slices obtained in step (7) using the K-means algorithm, and sort the clustered results according to the timestamp size to obtain the complete task logs, so as to achieve the purpose of task segmentation.

[0015] Preferably, the user interaction logs are an ordered set obtained by sorting behavior a i =(t, τ, C) in ascending order of timestamp, where t represents the timestamp, τ represents the behavior type, and C represents the behavior context, including software type, file name, form name, URL, component label, and component content.

[0016] Preferably, the removal of noise behaviors includes removing redundant behaviors and removing irrelevant behaviors.

[0017] Preferably, in the step (2), the behavior attribute context is mainly used to describe the location information where the behavior occurs, including software type, file name, form name, URL, and component label; the behavior content context is mainly used to describe the changes in the content on the application when the behavior occurs.

[0018] Preferably, the implementation method of the step (3) is as follows: taking the combination of the behavior type and the behavior attribute context as the node identifier, taking the direct following relationship between behaviors as the edge, and taking the direct following relationship frequency as the weight, to construct a direct following graph G=(V, E, W), where: V={v1, v2, …, v n} represents the node set, E={e1, e2, …, e m} represents the edge set, and W={w1, w2, …, w m} represents the weight set.

[0019] Preferably, in the step (4), the weight enhancement formula is:

[0020]

[0021] where: i represents the i-th occurrence of the direct following relationship between nodes v x , v y . The weight enhancement factor is defined as follows:

[0022]

[0023] where: represents the behavior corresponding to node v x , represents the behavior corresponding to node v y ; sim AC (,) and sim CC (,) respectively represent the Levenshtein distance similarity of the behavior attribute context and the behavior content context.

[0024] Preferably, the step (5) is specifically the following sub-steps:

[0025] (5.1) Calculate the transition probability between graph nodes. The transition probability calculation formula is as follows:

[0026]

[0027] where α is a hyperparameter that controls the sampling strategy, represents the enhanced weight of the edge connecting nodes v x and v y ;

[0028] (5.2) Randomly select several nodes as starting points and perform random walk sampling in the graph to obtain sampling paths;

[0029] (5.3) Use the sampling paths as learning samples and learn the vector representations of the nodes using the stochastic gradient descent algorithm.

[0030] Preferably, the formulas for calculating the similarity between the predecessors and successors of the behaviors in step (6) are:

[0031]

[0032]

[0033] where M represents the window radius for similarity calculation, and sim vec (,) represents the cosine similarity of the behavior vectors.

[0034] Preferably, the condition for log splitting in step (7) is: given a similarity threshold T, when and only when the conditions and are satisfied, using behaviors a i and a i+1 as the splitting points to split the log into two to generate new log slices.

[0035] Advantages of the present invention

[0036] The present invention combines a graph embedding algorithm to obtain vector representations of behaviors, and performs operations of first splitting and then clustering the logs based on the similarity of the vectors, providing a feasible method for solving the problem of poor segmentation effect of user interaction logs generated in the scenario of multi-task concurrent execution; when learning the vector representations of behaviors, the present invention comprehensively considers the context information of behavior attributes and content context, and the obtained vector representations can more accurately describe the relationships between behaviors; the present invention uses the similarity between predecessors and successors of behaviors to split the logs and cluster the log slices, which can improve the accuracy of log segmentation. Description of the drawings

[0037] Figure 1 Flowchart of the method of the present invention

[0038] Figure 2 Schematic diagram of user interaction logs

[0039] Figure 3 Schematic diagram of the direct following graph

[0040] Figure 4 Schematic diagram of log splitting Detailed implementation manners

[0041] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0042] The following will specifically describe the present invention in conjunction with the accompanying drawings.

[0043] This embodiment provides a multi - task user interaction log segmentation method based on graph embedding, as Figure 1 shown, and includes the following steps:

[0044] (1) Collect user interaction logs generated by the interaction between users and application programs As Figure 2 shown, the user interaction log is an ordered set sorted in ascending order of timestamps by user behavior a i =(t, τ, C), where t represents the timestamp, τ represents the behavior type, and C represents the behavior context, including software type, file name, form name, URL, component label, and component content.

[0045] (2) On the basis of (1), perform data pre - processing work, including the following sub - steps:

[0046] (2.1) Considering that a large number of noise behaviors unrelated to the task process will be generated during the interaction between users and application programs, it is necessary to clean the data. This mainly includes: a. Removing redundant behaviors, including continuous copy behaviors and behaviors of not pasting after copying; b. Removing irrelevant behaviors, including moving window behaviors and window scaling behaviors.

[0047] (2.2) Based on the behavior context information, extract the behavior attribute context AC and the behavior content context CC. The behavior attribute context is mainly used to describe the location information where the behavior occurs. Taking the Figure 2 shown log as an example, the behavior attribute context includes software type, file name, form name, URL, and component label; while the behavior content context is mainly used to describe the change situation of the content on the application program when the behavior occurs. Taking the copyCell behavior in Figure 2 as an example, the content context is the content being copied.

[0048] (3) Given the user interaction log For any two behaviors When and only when it satisfies a x <a y And At this time, there is a direct following relationship between a x , a y , denoted as r x→y .

[0049] According to the direct following relationship between behaviors, the user interaction log is represented as a direct following graph G=(V, E, W), where: V={v1, v2, …, v n} represents the set of behavior nodes, and the behavior node v i =(τ, AC) is uniquely determined by the behavior type and the behavior attribute context; E={e1, e2, …, e m} represents the set of edges, and each edge e i :(v x , v y ) corresponds to a direct following relationship r x→y ; W={w1, w2, …, w m} represents the set of weights, and the weight value w i :#r x→y is used to describe the frequency of occurrence of each direct following relationship. Figure 3 Shown in Figure 2 is the direct following graph representation of the user interaction log in

[0050] (4) Use the behavior context to perform an enhancement operation on the weights of the direct following graph. The weight enhancement formula for the directed edges connecting nodes v x , v y is as follows:

[0051]

[0052] where: i represents the i-th occurrence of the direct following relationship between nodes v x , v y . The weight enhancement factor is defined as follows:

[0053]

[0054] where: represents the behavior corresponding to node v x , represents the behavior corresponding to node v y ; sim AC (,) and sim CC (,) respectively represent the Levenshtein Distance similarity of the behavior attribute context and the behavior content context.

[0055] (5) Use the graph embedding algorithm node2vec on the direct following graph to learn the vector representation of the behavior nodes. The specific steps of the algorithm are as follows: (5.1) Calculate the transition probability between graph nodes. The transition probability calculation formula is as follows:

[0056]

[0057] Where: α is a hyperparameter for controlling the sampling strategy, represents the node v x and v y The enhanced weight of the connected edge; (5.2) Randomly select several nodes as starting points and perform random walk sampling in the graph to obtain a sampling path; (5.3) Use the sampling path as a learning sample and use the stochastic gradient descent algorithm to learn the vector representation of the nodes.

[0058] (6) Calculate the precursor similarity of the behavior according to the behavior node vector and the successor similarity The calculation formula is as follows:

[0059]

[0060]

[0061] Where: M represents the window radius of similarity calculation, sim vec (,) represents the cosine similarity of two behavior vectors. Let a i be the behavior that occurs for the i-th time in the user interaction log. Take M behaviors forward from a i and calculate the average value of the cosine similarities of the vector of pairwise behavior nodes as the precursor similarity of a i ; Take M behaviors backward from a i and calculate the average value of the cosine similarities of the vector of pairwise behavior nodes as the successor similarity of a i .

[0062] (7) Determine the log segmentation point according to the change of the behavior precursor and successor similarities. Given a similarity threshold T, when and only when the condition and are satisfied, use the behaviors a i and a i+1 as the segmentation points to generate new log slices. As Figure 4 shown, set the similarity threshold T = 0.2. Among the 100 behaviors from a1 to a 100 , there are three groups of behaviors that meet the segmentation point conditions, namely the 28th - 29th behavior points, the 42nd - 43rd behavior points, and the 63rd - 64th behavior points. Therefore, four log slices can be obtained.

[0063] (8) Given the number of task categories, use the K-means algorithm to cluster the log slices obtained in step (7). Use the central vector of all behavior vectors in the log slice as the class center vector, aggregate the similar log slices, and sort them according to the timestamp size to obtain the complete task log, so as to achieve the purpose of task segmentation.

[0064] The above description of the embodiments is to enable those of ordinary skill in the art to understand and apply the present invention. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative efforts. Therefore, the present invention is not limited to the above embodiments, and all improvements and modifications made by those skilled in the art based on the disclosure of the present invention should fall within the protection scope of the present invention.

Claims

1. A multi-task user interaction log segmentation method based on graph embedding, characterized in that, The method includes the following steps: (1)Collect user interaction logs generated by the interaction between users and applications in multi-task scenarios (2) Based on (1), perform data preprocessing work, including removing noise behaviors, extracting the behavior attribute context AC and the behavior content context CC from the behavior context; The behavior attribute context is mainly used to describe the location information where the behavior occurs. The behavior attribute context includes software type, file name, form name, URL, and component label; the behavior content context is used to describe the change situation of the content on the application when the behavior occurs; (3) According to the direct following relationship between behaviors, represent the user interaction log as a direct following graph; The implementation method of the step (3) is as follows: taking the combination of the behavior type and the behavior attribute context as the node identifier, taking the direct following relationship between behaviors as the edge, and taking the direct following relationship frequency as the weight, constructing a direct following graph G=(V, E, W), where: V={v1, v2, …, v n} represents the node set, E={e1, e2, …, e m} represents the edge set, W={w1, w2, …, w m} represents the weight set; (4) Based on (3), perform a weight enhancement operation on the direct following graph using the behavior attribute context and the content context; (5) On the direct following graph with enhanced weights, use the graph embedding algorithm node2vec to learn the vector representation of behavior nodes; (6) Calculate the predecessor similarity and successor similarity of the behavior based on the behavior node vector and successor similarity (7) Based on (6), determine the log segmentation point according to the change situation of the similarity between behavior predecessors and successors, and segment the user interaction log; (8) Given the number of task categories, use the K-means algorithm to cluster the log slices obtained in step (7), and sort the clustered results according to the timestamp size to obtain the complete task log, so as to achieve the purpose of task segmentation.

2. The multi-task user interaction log segmentation method based on graph embedding according to claim 1, characterized in that, The user interaction log is an ordered set obtained by sorting behavior a i =(t, τ, C) in ascending order of timestamps, where t represents the timestamp, τ represents the behavior type, and C represents the behavior context. The user interaction log includes software type, file name, form name, URL, component label, and component content.

3. The multi-task user interaction log segmentation method based on graph embedding according to claim 1, characterized in that, The removal of noise behaviors includes removing redundant behaviors and removing irrelevant behaviors.

4. The multi-task user interaction log segmentation method based on graph embedding according to claim 1, characterized in that, In step (4), the weight enhancement formula is: where: i represents the i-th occurrence of the direct following relationship between nodes v x , v y , and the weight enhancement factor is defined as follows: Wherein: represents the behavior of node v x corresponding behavior represents the behavior of node v y corresponding behavior; sim AC (,) and sim CC (,) respectively represent the Levenshtein distance similarities of the behavior attribute context and the behavior content context.

5. The multi-task user interaction log segmentation method based on graph embedding according to claim 1, characterized in that, Step (5) is specifically the following sub-steps: (5.1) Calculate the transition probability between graph nodes. The transition probability calculation formula is as follows: where α is a hyperparameter for controlling the sampling strategy, represents the enhanced weight of the edge connecting node v x and v y ; (5.2) Randomly select several nodes as starting points, and perform random walk sampling in the graph to obtain a sampling path; (5.3) Use the sampling path as a learning sample, and use the stochastic gradient descent algorithm to learn the vector representation of the nodes.

6. The multi-task user interaction log segmentation method based on graph embedding according to claim 2, characterized in that, In step (6), the similarity calculation formula between behavior predecessors and successors is: where M represents the window radius for similarity calculation, and sim vec (,) represents the cosine similarity of the behavior vectors.

7. The multi-task user interaction log segmentation method based on graph embedding according to claim 6, characterized in that, The condition for log splitting in step (7) is: given a similarity threshold T, when and only when the condition and is satisfied, using a i and a i+1 as the split points, the log is split into two to generate new log slices.

Images