Information processing device, information processing method, and information processing program

Abstract

An information processing device (10) acquires an operation log relating to operation information, analyzes the operation log, and identifies attribute information of the operation log. Then on the basis of the identified attribute information, the information processing device (10) visualizes an image including an object that represents a log of a prescribed unit of operation and that can be selected by a user operation. Then when the object included in the visualized image is selected by a user operation, the information processing device (10) generates an executable automatic operation program on the basis of the selected object.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] In business activities, it is important to carry out daily operations efficiently and smoothly in order to continue business activities. Since the business environment changes due to external and internal factors, it is necessary to continuously implement business improvement. To carry out effective business improvement, it is necessary to carry out the steps of grasping the actual situation, finding and analyzing areas for improvement, proposing improvement plans, and implementing the improvement plans in that order.

[0003] Robotic Process Automation (RPA) is becoming increasingly popular as a powerful tool for improving business efficiency. However, in order to achieve the desired results from the introduction of RPA, there are challenges such as the need for analysts' know-how and skills to find tasks suitable for RPA and the need to comprehensively analyze the entire business process, and as a result, it is often not used effectively.

[0004] One method known to solve this problem is to use operation logs, which are records of user device operations, as input and perform statistical analysis to mechanically discover repetitive tasks. By utilizing such methods, it becomes possible to easily discover repetitive tasks that are candidates for RPA without relying on know-how or skills.

[0005] To actually automate business processes based on these results, the output results of these technologies cannot be directly used as input for the RPA tool, so an automated operation program must be created on the RPA tool while manually viewing the output results. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Fumitaka Yokose and 5 others, "Business visualization technology that contributes to the promotion of DX," NTT Technical Journal, February 2020, pp.72-75 Summary of the Invention [Problem to be solved by the invention]

[0007] However, with conventional technology, output results could not be directly input to an RPA tool, making it difficult to create automated operation programs.For example, with conventional process mining technology, it was necessary to manually launch a separate RPA tool and create operation scenarios with the RPA tool while viewing the analysis results, which was time-consuming.

[0008] The present invention has been made in view of the above, and has an object to provide an information processing device, an information processing method, and an information processing program that are capable of creating an automatic operation program. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the object, the information processing device of the present invention is characterized in having an analysis unit that acquires an operation log related to operation information, analyzes the operation log, and identifies attribute information of the operation log; a visualization unit that visualizes an image including an object that is an object of a predetermined unit of the operation log and that is selectable by a user operation based on the attribute information identified by the analysis unit; and a collaboration unit that, when an object included in the image visualized by the visualization unit is selected by a user operation, generates an executable automatic operation program based on the selected object. Effect of the Invention

[0010] According to the present invention, an automatic operation program can be easily created. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a block diagram showing an example of the configuration of an information processing device. [Diagram 2] FIG. 2 is a diagram illustrating an example of an operation log stored in the operation log storage unit. [Diagram 3] FIG. 3 is a diagram showing the relationship between business, work, case, and operation type. [Figure 4] FIG. 4 is a diagram illustrating an example of a data configuration of the operation log. [Diagram 5] FIG. 5 is a diagram showing an example of a classification result. [Figure 6] FIG. 6 is a diagram showing an example of the data configuration of an operation log to which labels have been added. [Figure 7] FIG. 7 is a diagram showing an example of an operation log to which a column for operation details has been added. [Figure 8] FIG. 8 is a diagram showing an example of an operation log in chronological order. [Figure 9] FIG. 9 is a diagram illustrating an example of a co-occurrence matrix. [Figure 10] FIG. 10 is a diagram for explaining the similarity calculation process. [Figure 11] FIG. 11 is a diagram illustrating the division process of the operation log. [Figure 12] FIG. 12 is a diagram showing an example of a classification result. [Figure 13] FIG. 13 is a diagram showing an example of the number of operation types in each operation set. [Figure 14] FIG. 14 is a diagram showing an example of an operation log in chronological order. [Figure 15] FIG. 15 is a diagram illustrating an example of a co-occurrence matrix. [Figure 16] FIG. 16 is a diagram illustrating an example of how to weight a co-occurrence matrix according to distance. [Figure 17] FIG. 17 is a diagram for explaining an example of how to weight a co-occurrence matrix depending on whether or not the events occurred within the same window. [Figure 18] FIG. 18 is a diagram illustrating the calculation process of the center of gravity vector. [Figure 19] FIG. 19 is a diagram showing the transition of the similarity. [Figure 20] FIG. 20 is a diagram illustrating an example of a divided operation set. [Figure 21] FIG. 21 is a diagram illustrating the classification process of the operation sequence. [Figure 22] FIG. 22 is a diagram illustrating the classification process of the operation sequence. [Diagram 23] FIG. 23 is a diagram illustrating an example of the operation log. [Figure 24] FIG. 24 is a diagram illustrating an example of work model information. [Diagram 25] FIG. 25 is a diagram showing an example of a working model creation screen. [Figure 26] FIG. 26 is a diagram for explaining the procedure for providing information. [Figure 27] FIG. 27 is a diagram for explaining the procedure for adding information. [Figure 28] FIG. 28 is a diagram showing an example of a screen on which the operation log is visualized in a timeline format. [Figure 29] FIG. 29 is a diagram showing an example of a screen that visualizes an operation process. [Diagram 30] FIG. 30 is a diagram showing an example of a screen in which operations are visualized in a predetermined unit. [Diagram 31] FIG. 31 is a diagram showing an example of a screen on which a user creates a desired automated operation flow. [Diagram 32] FIG. 32 is a flowchart illustrating an example of a processing procedure for the operation log acquisition processing according to the embodiment. [Diagram 33] FIG. 33 is a diagram illustrating an example of a computer that implements an information processing device by executing a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an information processing device, an information processing method, and an information processing program according to the present application will be described in detail with reference to the accompanying drawings. The present invention is not limited to the embodiments described below.

[0013] [Configuration of information processing device] Fig. 1 is a block diagram showing an example of the configuration of an information processing device. As shown in Fig. 1, an information processing device 10 of this embodiment is connected to a terminal device 20 via a network (not shown). The information processing device 10 is realized by, for example, a general-purpose computer such as a personal computer, and is an information processing device that executes a process of visualizing the operation process for each task by arranging an operation log based on a series of operation events (task model) of a task presented by a user and automatically adding task information and case information. Note that the information processing device 10 may be any type of information processing device including a server.

[0014] The terminal device 20 is an information processing device used by a user. The user is, for example, a business person. The business person uses various software such as a business system and a general-purpose application on the terminal device 20. The terminal device 200 may be any type of information processing device including a client device such as a smartphone, a desktop PC, a notebook PC, or a tablet PC. In the example of FIG. 1, the information processing device 10 and the terminal device 20 are separate devices, but the terminal device 20 may have some or all of the functions of the information processing device 10.

[0015] The terminal device 20 has an operation log acquisition unit 21. The operation log acquisition unit 21 acquires a user's operation log. For example, the operation log acquisition unit 21 acquires an operation log including an operation time, an operation location, an operation position, and the like at the timing of an operation event. Here, the operation log acquisition unit 21 generates an operation log that is the basis of analysis, including information on an operation type and an operation location, which are the smallest unit of an RPA scenario, by detecting an event that occurs due to an operation from an OS or a GUI framework and acquiring information on GUI components, in order to record the operation content on the terminal device 20 as a log. Then, the terminal device 20 transmits the operation log to the information processing device 10.

[0016] The information processing device 10 receives the operation log acquired by the operation log acquisition unit 21 of the terminal device 20, and stores it in the operation log storage unit 13a of the storage unit 13. Note that the timing at which the information processing device 10 and the terminal device 20 transmit and receive the operation log may be any timing, and for example, the transmission and reception may be performed at a predetermined time interval, or the transmission and reception may be performed every time the operation log acquisition unit 21 acquires an operation log.

[0017] The information processing device 10 includes a communication unit 11, a control unit 12, and a storage unit 13. The processing of each unit included in the information processing device 10 will be described below.

[0018] The communication unit 11 is realized by a NIC (Network Interface Card) or the like, and controls communication between an external device and the control unit 12 via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 11 receives an operation log from the terminal device 20.

[0019] The storage unit 13 stores data and programs necessary for various processes by the control unit 12, and has an operation log storage unit 13a. For example, the storage unit 13 is a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0020] The operation log storage unit 13a stores the operation log acquired from the terminal device 20. For example, as illustrated in Fig. 2, the operation log storage unit 13a stores a table including columns of "operation time" indicating the time when the operation was performed, "user" indicating the user who performed the operation, "application information" indicating the application to be operated, "window information" indicating the window title of the window to be operated, "operation location" indicating the location of the operation, "capture image" which is an image at the time of the operation, and "operation location" indicating the location where the operation was performed, as the operation log. Fig. 2 is a diagram showing an example of the operation log stored in the operation log storage unit.

[0021] The control unit 12 has an internal memory for storing programs that define various processing procedures and required data, and executes various processes using these. For example, the control unit 12 has an analysis unit 12a, a visualization unit 12b, and an RPA linkage unit 12c. Here, the control unit 12 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0022] The analysis unit 12a acquires an operation log related to operation information, analyzes the operation log, and identifies attribute information of the operation log. For example, the analysis unit 12a identifies task information indicating the task content of the operation log, identifies work information indicating the work content of the operation log included in each identified task information, and identifies case information of the operation log included in each identified work content. In addition, the operation log may identify the operation type of the operation log included in each identified work content.

[0023] Here, the relationship between business, work, case, and operation type will be described with reference to FIG. 3. FIG. 3 is a diagram showing the relationship between business, work, case, and operation type. The analysis unit 12a analyzes the operation log acquired from the terminal device 20, and identifies and assigns attribute information. The analysis unit 12a creates a hierarchical structure of "business," "work," and "case / operation type," as exemplified in FIG. 3. The analysis unit 12a identifies and assigns a business ID, a task ID, a case ID, and an operation type ID as each piece of attribute information.

[0024] For example, the analysis unit 12a mechanically analyzes the operation log to extract task classifications, task classifications, and main flows that are useful for task analysis. The analysis unit 12a focuses on task groups as task classifications, and tags the operation logs with tasks by grouping the tasks according to their co-occurrence. The analysis unit 12a also focuses on the association between business documents and operation screens as task classifications, and tags the operation logs with tasks. The analysis unit 12a may also extract main flows by arranging the operation logs and extracting operations that appear in common in many operation sequences.

[0025] In the following, specific processes are described in the order of a process for identifying a business ID, a process for identifying a task ID, and a process for identifying a case ID. Note that the processes for identifying and assigning a business ID, a task ID, and a case ID do not all need to be automatic, and may be manual.

[0026] [Business ID identification process] First, the process of identifying a task ID will be described. An example of an operation log used in this description will be described. FIG. 4 is a diagram showing an example of the data configuration of an operation log. As shown in table T1 in FIG. 4, an operation log has items of operation log ID, operation time, information identifying the operation target window, and description content. The operation log ID is a value unique to each operation log that is assigned when the operation log is accumulated. The operation time is the time when the operation log is acquired. The information identifying the operation target is the window title, URL, file path, application name, etc. of the operation target window. The description content is text information in the operation target window.

[0027] For example, the operation log with operation log ID "1" is an operation log acquired at "2019 / 03 / 27 13:15:32". And, the operation log with operation log ID "1" has the operation target "https: / / www.shuccho_shinsei.html" and the contents are "Business trip application system, department name, person in charge name, business trip destination, business trip route, please enter the cheapest route."

[0028] The analysis unit 12a first divides the text information of the "description contents" corresponding to each ID of the operation log to be processed into words, and creates a vector (hereinafter, referred to as an operation log vector) using a set of words in the description contents of the operation log. The analysis unit 12a creates a word vector for each ID of the operation log using a method such as Bag-of-Words, TF-IDF, or Word2Vec.

[0029] The analysis unit 12a can also employ a method of creating a vector using only common words from among the word sets in the contents of the operation logs. In this case, the following procedure is added instead of the above procedure. First, the analysis unit 12a classifies the operation logs by the operation target window. The analysis unit 12a compares the same operation screens from the identification information of the operation screens in the operation logs. In this way, the analysis unit 12a compares all the operation logs of the operation target windows and extracts a common word set, i.e., a word set that appears in each operation log. Then, the analysis unit 12a creates an operation log vector from the extracted word set. By adopting this method, the analysis unit 12a is able to perform classification with high accuracy without being influenced by words that appear in each case.

[0030] Next, the analysis unit 12a classifies the operation logs into a plurality of groups using a document classification method based on the operation log vector. The analysis unit 12a uses a clustering method such as a topic model, X-means, or K-means as the document classification method. The number of groups to be classified can be set by automatic estimation or can be set manually.

[0031] Then, the analysis unit 12a determines a label for each group. The analysis unit 12a extracts characteristic words for each group, determines the extracted words as labels, and assigns them to the operation logs corresponding to each group. Characteristic words are words that appear frequently in the distribution of word appearance frequency (hereinafter referred to as "word distribution"). Alternatively, characteristic words are words that appear frequently only in a specific group and do not exist or appear infrequently in other groups.

[0032] For example, a case where a topic model is used will be described. When a plurality of operation logs are classified into a plurality of groups using the topic model, the analysis unit 12a obtains a distribution of the appearance frequency of words for each topic. Then, the analysis unit 12a extracts words having a high distribution ratio for each topic as labels of the groups. Then, the classification unit 142 assigns the extracted labels to the corresponding groups.

[0033] A case where a clustering method such as X-means is used will be described. The analysis unit 12a classifies a plurality of operation logs into groups using the clustering method. After that, the analysis unit 12a extracts, as group labels, words that appear frequently only in a specific group and do not appear or appear less frequently in other groups. Then, the analysis unit 12a assigns the extracted labels to the corresponding groups.

[0034] Fig. 5 is a diagram showing an example of a classification result. In Fig. 5, as a specific example, it is assumed that a person in charge is performing a business trip application task and a purchase application task, and the operation log shown in the upper part of Fig. 5 is acquired. Also, a set of words in each operation log is shown as an example. And, a frame F1 in the lower part of Fig. 5 shows the labels assigned to each group of the operation log by the analysis unit 12a.

[0035] Here, operation logs that correspond to the same task are expected to have many common words, whereas operation logs for different tasks are expected to have different words that appear frequently in each operation log.

[0036] In the example of Figure 5, in the "Travel application task" "Travel system input task," many words related to the travel system appear, such as the underlined "Travel, route, departure, ...." In the "Route confirmation task," many words related to the route appear, such as the bolded "Route, via, time, ...."

[0037] From the above, by the analysis unit 12a classifying the operation logs based on a word set, it is possible to classify the operations into a plurality of groups at the granularity of the tasks.

[0038] Also, it is desirable that the labels given to each group are names that are only used for that task, so that people can understand which task it is. For this reason, a possible method is to use words that appear frequently in a specific group but do not appear or appear less frequently in other groups as labels. In the example of Figure 5, in the business trip application system and the purchasing application system, "name", "department", and "system" appear frequently and in both groups, so they are not suitable as labels.

[0039] On the other hand, words such as underlined "business trip, route, departure,...", bold "route, via, time,...", and bold italic "purchase, product, amount,..." appear frequently in each group and rarely in other groups. Therefore, by the analysis unit 12a treating these words as words representing the tasks, i.e., as labels, the business analyst can easily grasp what tasks the operation logs of each group correspond to, without the business analyst having to process them himself.

[0040] Fig. 6 is a diagram showing an example of the data configuration of operation logs to which labels have been added. As shown in table T2 in Fig. 6, after classification by the analysis unit 12a, the label of the group to which the operation log belongs is added as a task ID in the label column of each operation log. For example, the operation log with operation log ID "1" is labeled with "business trip."

[0041] [Processing to identify the job ID] Next, the process of identifying the task ID will be described. In the process of identifying the task ID, for example, the operation log is classified by the task ID (label) assigned in the process of identifying the task ID described above, and the process of identifying the task ID is performed for each task.

[0042] The analysis unit 12a acquires the operation log and identifies each operation performed by the user using the operation log. For example, as illustrated in Fig. 7, the analysis unit 12a reads out a table of the operation log from the operation log storage unit 13a, adds a column for "operation content", adds window information and information on the operation location to the "operation content" column, and stores the column in the operation log storage unit 13a. Fig. 7 is a diagram showing an example of an operation log to which a column for operation content has been added. Then, the analysis unit 12a identifies operations with the same "operation content" as being of the same type.

[0043] To explain this with a specific example, when there are multiple operations whose operation content is "web page 1 + button a", the analysis unit 12a identifies these operations as operations of the same type. Note that, in order to identify operations of the same type, the analysis unit 12a may add a column such as an operation ID that uniquely identifies the type of operation, together with or instead of the column of "operation content".

[0044] The analysis unit 12a creates a vector for each operation based on the co-occurrence relationship between the identified operations. For example, the analysis unit 12a counts the number of other operations performed within a predetermined number of times before and after the operation sequence performed in chronological order for each identified operation, creates a co-occurrence matrix showing the co-occurrence relationship between the operations using the counted number, and creates a vector for each operation based on the co-occurrence matrix.

[0045] Here, the process of creating a co-occurrence matrix will be described with reference to the examples of Figs. 8 and 9. Fig. 8 is a diagram showing an example of an operation log in chronological order. Fig. 9 is a diagram showing an example of a co-occurrence matrix created by the operation vector creation unit. For example, the analysis unit 12a reads the "operation contents" added to the operation log in chronological order as shown in Fig. 8. Then, the analysis unit 12a creates a co-occurrence matrix by counting n operations before and after each operation as shown in Fig. 9. In the example of Fig. 9, the analysis unit 12a creates a co-occurrence matrix for n=2.

[0046] Then, the analysis unit 12a generates a vector for each operation from the co-occurrence matrix. The size of the generated vector is the number of types of operations, and each element includes the count number of n operations before and after the target operation. For example, using the example of FIG. 9, the analysis unit 12a creates a vector of the operation content "webpage 1 + button a" as webpage 1 + button a = [0, 3, 3, 0, 1, 0, 0...], and a vector of the operation content "webpage 1 + text box b" as webpage 1 + text box b = [3, 1, 3, 1, 0, 0, 0...].

[0047] Furthermore, when the type of the identified operation is equal to or greater than a predetermined threshold, the analysis unit 12a may create a vector for each operation using a predetermined dimension reduction method. That is, when creating a vector for each operation from a co-occurrence matrix for each operation, the length of the vector is the number of operation types, and the greater the number of operation types, the higher the calculation cost. For this reason, for example, the analysis unit 12a may use a dimension reduction method such as SVD (Single Value Decomposition) to perform dimensional reduction of each operation vector. This makes it possible to reduce the calculation cost.

[0048] The analysis unit 12a uses the created vectors of each operation to calculate the similarity between a predetermined number of operations adjacent in the time series. For example, the analysis unit 12a uses the created vectors of each operation to calculate a center of gravity vector or a sum vector for a predetermined number of operation series adjacent in the time series, and calculates the similarity between the vectors.

[0049] Here, the similarity calculation process will be described with reference to Fig. 10. Fig. 10 is a diagram for explaining the similarity calculation process. For example, as illustrated in Fig. 10, the analysis unit 12a reads the operation log in chronological order and calculates the operation log to be divided. i The sequence of previous m operations (operations i―m ,···,operation i ) and operations i The sequence of m operations (operations i+1 ,···,operation i+1+m, the center of gravity vector or the sum vector is calculated using each of the operation vectors.

[0050] Then, the analysis unit 12a calculates the similarity between the operation sequences by using a vector similarity calculation method such as cosine similarity or Euclidean distance. After that, the analysis unit 12a stores the operation order and the similarity in the storage unit 13. For example, the operation order "1" refers to the interval between the first operation and the second operation in the time series.

[0051] The analysis unit 12a uses the similarity to determine a division point for each operation, and divides the time series of operations into operation sets based on the division point. FIG. 11 is a diagram for explaining the division process of the operation log by the determination unit. For example, as illustrated in FIG. 11, the analysis unit 12a reads out the "order of operations" and "similarity" from the analysis unit 12a, and if the similarity is calculated by cosine similarity, finds the length (d1) from the start of the decrease in similarity to the minimum point of similarity, and the length (d2) from the minimum point to the end of the increase in similarity, and calculates the depth (d=d1+d2) using them. On the other hand, if the similarity is calculated by Euclidean distance, the analysis unit 12a finds the length from the start of the increase in similarity to the maximum point of similarity, and the length from the maximum point to the end of the decrease in similarity, and calculates the depth of them.

[0052] If the depth is equal to or greater than a threshold, the analysis unit 12a sets the minimum point (or maximum point) as a division point of the work and divides the operation log. The analysis unit 12a repeats this process until all operations are determined in order. For example, as illustrated in FIG. 11, when the analysis unit 12a determines that the operation sequence "4" is a division point, it divides the area between the fourth and fifth operations in the operation log. Note that the process of determining the division point by the determination unit 12d is not limited to determining whether the depth (d=d1+d2) is equal to or greater than a threshold, and the determination unit 12d may determine whether both the length (d1) from the start of the decrease in similarity to the minimum point of similarity and the length (d2) from the minimum point to the end of the increase in similarity are equal to or greater than a predetermined threshold, and if both are satisfied, the minimum point (or maximum point) may be set as a division point of the work.

[0053] The analysis unit 12a classifies each divided operation set into classes based on the number of operation types common between the operation sets. For example, the analysis unit 12a compares the types of operations in the operation sets that appear in the already classified classes with the types of operations in the operation set to be classified, and calculates the number of common operation types. Then, the analysis unit 12a classifies the operation set to be classified into a class in which the number of common operation types is equal to or greater than a threshold and is the largest, and classifies the operation set into a new class if the number of common operation types is equal to or less than the threshold. As a result, the analysis unit 12a classifies the divided operation sets into classes 1 to 3, for example, as illustrated in FIG. 12. FIG. 12 is a diagram showing an example of the classification result.

[0054] Also, for example, the analysis unit 12a may classify the operation sets in order from the operation set with the largest number of operation types. In this case, as illustrated in FIG. 13, the analysis unit 12a obtains the number of operation types for each operation set, and classifies the operation sets in order from the operation set with the largest number of operation types. FIG. 13 is a diagram showing an example of the number of operation types for each operation set. Note that the analysis unit 12a is not limited to classifying the operation sets in order from the operation set with the largest number of operation types, and may classify the operation sets in chronological order, for example. As another method, the analysis unit 12a may create a vector by counting the operations appearing in the operation set, with the number of operation types as the number of dimensions for each operation set, and classify the operation sets based on a threshold value using a method such as a hierarchical clustering method.

[0055] In the following, a series of processes for the analysis unit 12a to classify the operation log by task for the chronological operations illustrated in FIG. 14 will be described using specific examples in FIG. 15 to FIG. 21. FIG. 14 is a diagram showing an example of an operation log in chronological order. Each character illustrated in FIG. 14 indicates an operation identified by the same operation identification unit 12a, and the same character indicates an operation of the same type. That is, the operation sequence illustrated in FIG. 14 means that the operations were performed in the order of a, b, c, d, e, f, etc. In addition, in the operation sequence in FIG. 14, "abcdefg" is the operation of task 1, "hijklmn" is the operation of task 2, and "opqrstuvwxyz" is the operation of task 3, and there are three types of classified tasks.

[0056] FIG. 15 is a diagram showing an example of a co-occurrence matrix. As shown in FIG. 15, the analysis unit 12a creates a co-occurrence matrix by using n operations before and after each operation. In the example of FIG. 15, n=3. The value of n is set as the range of operations before and after the operation to be determined as co-occurrence. In addition, in creating the co-occurrence matrix, weighting may be added not only based on the count of the presence or absence of an operation, but also based on the adjacent status of operations before and after the target operation and whether or not the operations are in the same window. This allows the analysis unit 12a to create a co-occurrence matrix that takes into account the status of the actual operations, making it possible to more accurately classify the group of tasks.

[0057] For example, the analysis unit 12a may create a co-occurrence matrix by using a value obtained by weighting, for each operation, the number of other operations performed within a predetermined number of times before and after a sequence of operations performed in chronological order according to the distance from the other operations, and create a vector for each operation based on the co-occurrence matrix.

[0058] FIG. 16 is a diagram for explaining an example of weighting a co-occurrence matrix according to distance. In the example of FIG. 16, n=5. For example, as illustrated in FIG. 16, when the analysis unit 12a determines the weight according to the distance between the target operation and other operations, the distance between the operation "e" and the operation "g" on both sides of the target operation "f" is "1", and when the distance is "1", the analysis unit 12a counts 0.5 as the weight and creates a co-occurrence matrix. Also, for example, the distance between the operation "d" and the operation "o" of the target operation "f" is "2", and when the distance is "2", the analysis unit 12a counts 0.25 as the weight and creates a co-occurrence matrix.

[0059] The analysis unit 12a may also count operations before and after the target operation with weighting depending on whether they were performed in the same window (web page, file, etc.). For example, the analysis unit 12a may create a co-occurrence matrix using a value obtained by weighting the number of other operations performed within a predetermined number of times before and after the operation sequence performed in chronological order for each identified operation, depending on whether the other operations were performed in the same window, and create a vector for each operation based on the co-occurrence matrix. In this case, for example, the analysis unit 12a may use application information in the operation log or a window title in the window information to identify the operation, and determine whether the operations before and after the target operation are on the same screen.

[0060] FIG. 17 is a diagram for explaining an example of weighting a co-occurrence matrix according to whether or not the operations are performed in the same window. In the example of FIG. 17, n=5, and the operation abcdefg is an operation on the same screen (window) A, and the operation opqrstu is an operation on the same screen B. For example, as illustrated in FIG. 17, when the weight of the same window is "1" and the weight of another screen is "0.5", the analysis unit 12a counts the weight of the operation abcdeg of the same window that is within 5 positions before and after the target operation "f" as "1" to create a co-occurrence matrix. In addition, the analysis unit 12a counts the weight of the operation opqr of another window that is within 5 positions before and after the target operation "f" as "0.5" to create a co-occurrence matrix.

[0061] Then, the analysis unit 12a generates a vector of each operation from the co-occurrence matrix. For example, in the example of FIG. 15, the analysis unit 12a creates, for example, an operation a=[0, 6, 5···1, 1,···2, 2, 2] as a vector of the operation "a" by using the co-occurrence frequency with all operations. In addition, when there are many types of operations, the analysis unit 12a may perform dimensional compression using SVD, for example, by compressing the dimensions into a 50-dimensional vector while retaining the features of 1000 types of operations.

[0062] Next, the analysis unit 12a reads the operation log in chronological order, acquires m operations including the operation to be judged for division and m operation series following the operation to be judged for division, and calculates the similarity between the m operation series including the adjacent operation to be judged for division (operation series A in FIG. 18) and the m operation series following the operation to be judged for division (operation series B in FIG. 18) using the vectors of each operation created. Here, the calculation process of the center of gravity vector will be described with reference to FIG. 18. FIG. 18 is a diagram for explaining the calculation process of the center of gravity vector. In the example of FIG. 18, m=5, and the operation "g" indicated by "▽" is the object of the division judgment. In this case, the operation series A and operation series B to be calculated for similarity are operation series A=bdefg and operation series B=opqrs.

[0063] Then, the analysis unit 12a acquires the vector of each operation (operation vector) in the operation sequence A, and calculates the center of gravity of the operation sequence A by using the following formula (1).

[0064]

number

[0065] Similarly, the analysis unit 12a also acquires vectors of each operation (operation vector) for operation sequence B, and calculates the center of gravity of operation sequence B using the following formula (2).

[0066]

number

[0067] Then, the analysis unit 12a calculates the similarity between the center of gravity vectors of the operation sequence A and the operation sequence B. For example, when the analysis unit 12a adopts the cosine similarity for calculating the similarity, the analysis unit 12a calculates the similarity between the center of gravity vectors of the operation sequence A and the operation sequence B by using the following formula (3).

[0068]

number

[0069] Then, the analysis unit 12a determines a portion of the operation log where there is a difference (difference d) equal to or greater than a threshold value among similarities between operation sequences before and after each operation in the operation log as a division point of the operation log, and divides the time-series operations into operation sets based on the division point. Figure 19 is a diagram showing the transition of similarities. For example, the determination unit 12d calculates the difference d as follows, as exemplified in Figure 19. Difference d = (start position of monotonically decreasing part - minimum point) + (end position of monotonically increasing part - minimum point)

[0070] Then, when the difference d is equal to or greater than a threshold, the analysis unit 12a determines that the minimum point is a division point of the task. In the example of FIG. 19, the analysis unit 12a determines that the division point is between the operation "g" corresponding to the minimum point and the next operation "o". The analysis unit 12a also determines that the division point is between the operation "z" corresponding to the minimum point and the next operation "h". The analysis unit 12a then divides the operations based on the division points, thereby dividing the time-series operations illustrated in FIG. 14 into a plurality of operation sets as illustrated in FIG. 20. In the example of FIG. 20, the analysis unit 12a divides the time-series operations illustrated in FIG. 14 into an operation set "abcdefg", an operation set "opqrstuvwxyz", an operation set "hijklmn", and so on. FIG. 20 is a diagram for explaining an example of divided operation sets.

[0071] Next, the classification process of the operation sequence will be described with reference to FIG. 21 and FIG. 22. FIG. 21 and FIG. 22 are diagrams for describing the classification process of the operation sequence. The analysis unit 12a classifies each divided operation set into classes based on the number of types of operations common to the operation sets. For example, the analysis unit 12a compares the types of operations in the operation set that appear in the already classified class with the types of operations in the operation set to be classified, and calculates the number of common operation types. Then, the analysis unit 12a classifies the operation set to be classified into a class in which the number of common operation types is equal to or greater than a threshold and is the largest, and classifies it as a new class if the number of common operation types is equal to or less than the threshold. As a result, the analysis unit 12a classifies the divided operation sets into classes 1 to 3, for example, as illustrated in FIG. 21.

[0072] Also, for example, the analysis unit 12a may classify the operation sets in order from the operation set with the largest number of operation types. In this case, as illustrated in FIG. 22, the analysis unit 12a obtains the number of operation types for each operation set, starts the classification process from the operation set "opqrstuvwxyz" with the largest number of operation types, and initially classifies it into "Class 1". Next, the analysis unit 12a compares the operation set "abcdefgabcdefefg" with the next largest number of operation types with "opqrstuvwxyz" classified into Class 1, and calculates the number of common operation types (common operation set). Here, the analysis unit 12a classifies the operation set "abcdefgabcdefefg" into a new Class 2 because the number of common operation types is "0".

[0073] In this way, the analysis unit 12a classifies the divided operation sets into classes, so that each operation can be classified by task. As described above, in the operation sequence of FIG. 14, each operation of "abcdefg" is an operation of task 1, each operation of "hijklmn" is an operation of task 2, and each operation of "opqrstuvwxyz" is an operation of task 3. In the example of FIG. 21, the information processing device 10 classifies, for example, the operation set "abcdefgabcdefefg" and the operation set "abcdefg" into class 2 as a series of tasks in the same task type. In addition, the analysis unit 12a classifies, for example, the operation set "opqrstuvwxyz" into class 1 as a series of tasks in the same task type. In addition, the analysis unit 12a classifies, for example, the operation set "hijkmklmn" and the operation set "hijklmn" into class 3 as a series of tasks in the same task type. In this way, the analysis unit 12a can extract a series of tasks from the operation log and classify them by task type. The analysis unit 12a classifies the series of tasks by task type and assigns a task ID to each task.

[0074] [Identifying the case ID] Next, the case ID specification process will be described. In the case ID specification process described below, for example, the operation log 121 is classified by the work ID (label) assigned in the above-mentioned work ID specification process, and the case ID specification process is performed for each case.

[0075] An example of the operation log 121 used in the description here will be described. Fig. 23 is a diagram showing an example of the operation log 121. As shown in Fig. 23, the operation log 121 is a collection of records corresponding to each event that occurs in response to an operation on the terminal device 20. Each column in the operation log 121 in Fig. 23 is called an attribute. Also, the value for each attribute of each record (row) in the operation log 121 is called an attribute value.

[0076] Also, the operation log 121 in Fig. 23 is information regarding operations on a Web browser. The attributes of the operation log 121 are not limited to those shown in Fig. 23. The attributes of the operation log 121 may include, for example, a history of input from an input device (keyboard, mouse, etc.), a character string of a command input on a CUI (Character User Interface), etc.

[0077] The terminal device 20 executes a Web browser to display a Web page on the screen, and accepts operations such as input to GUI parts on the Web page. The operation log 121 is information related to operations on the Web page.

[0078] The attributes included in the operation log 121 are as follows: Date and Time: The time when the event occurred (including the date) Operation type: A string representing the type of operation Application: A string representing the executable name of the running application. URL: A string representing the URL of the web page being displayed. Title: A string representing the title of the web page. tagName: A string representing the tag that the GUI part is in. type: A string that represents the type of GUI part. value: A string representing the input value to the GUI part id: A string to identify the GUI part name: A string representing the name of the GUI part className: A string representing the name of the GUI part class. left: A number that represents the position of the GUI part top: A number that represents the position of the GUI part width: A number representing the position of the GUI part height: A number representing the position of the GUI part image: Screen capture image data (e.g., a string representing a file path)

[0079] Each attribute may correspond to an HTML tag of the same name. Furthermore, a null value is set in the operation log 121 for an attribute that is not set or for which the attribute value cannot be obtained.

[0080] The attribute value included in the operation log 121 does not need to be the information itself acquired from the terminal device 20. The attribute value may be obtained by processing the acquired information, or may be a combination of multiple pieces of information. In addition, the multiple pieces of information to be combined may not appear in the operation log 121.

[0081] The attribute "operation ID" is information for identifying the operation of the event indicated in each record. A method for identifying the operation ID will be described later.

[0082] For example, the record on the first line of Figure 23 shows information about an event that occurred on "2021 / 03 / 10 12:02:10" while the application "Example.exe" was displaying the web page at the URL "http: / / www.sample.jp / transit / ?session=0001."

[0083] Additionally, the record on the first line of Figure 23 shows that the value "YRP Nobi" was entered into a GUI part on the screen titled "Route Search" with a tagName of "input", type of "text", id of "null", and name of "on_st".

[0084] The record on the first line in Fig. 23 indicates that the GUI part is located at a position represented by left "217", top "84", width "195", and height "24". It also indicates that a capture image of the screen during operation is stored in a file named "1.png".

[0085] The GUI parts include text boxes, buttons (including submit buttons), pull-down menus, check boxes, radio buttons, and the like.

[0086] Fig. 24 is a diagram showing an example of the work model information 122. Work models are stored in the work model information 122. As shown in Fig. 24, a work model is a pattern of combination and order of operations. An operation is represented by an operation ID.

[0087] FIG. 24 shows that the work model with the work name “on-site process search work” is a pattern that combines the following operations in the order of operation a, operation b, operation c, operation d, operation e, operation f, operation g, and operation h.

[0088] Note that the work model may be identified by the work name as in FIG. 24, or may be identified by an ID given to each work model.

[0089] The analysis unit 12a identifies an operation corresponding to an event indicated in the operation log 121 of the computer.

[0090] The analysis unit 12a identifies the operation ID based on each attribute value of the operation log 121. The operation ID is an ID for identifying events (records) corresponding to the same operation.

[0091] That is, the same operation may cause different events to occur in the terminal device 20. Conversely, different events may be regarded as being caused by the same operation.

[0092] For example, the event of inputting the character string "abc" into a text box and the event of inputting the character string "xyz" into the same text box are both caused by the operation of "inputting a character string into a text box." However, since the character strings input are different, each event is considered to be a different event.

[0093] The analysis unit 12a identifies events of the same operation by applying one of the following rules to each attribute of the operation log 121: (1) determining whether a character string is an exact match, (2) determining whether a character string is a partial match, (3) determining whether a character string is a range of numerical values, or (4) not using the attribute for determination.

[0094] The analysis unit 12a applies (1) to the operation type, application, tagName, type, id, and name among the attributes of the operation log 121. The analysis unit 12a applies (2) to the URL and title among the attributes of the operation log 121. The analysis unit 12a applies (3) to the width and height among the attributes of the operation log 121. The analysis unit 12a applies (4) to the date and time, value, className, left, and top among the attributes of the operation log 121.

[0095] For example, the analysis unit 12a sets the operation ID of the records on the first and ninth lines of the operation log 121 shown in Fig. 23 to "operation a". For example, the operation type and application match in the records on the first and ninth lines, but the date and time and value do not match.

[0096] The analysis unit 12a creates an activity model, which is information that associates the identified combination and order patterns of operations with tasks in the business. The activity model is a model that represents a series of operations that are performed to complete a task in the business.

[0097] The analysis unit 12a may create a task model based on a user's specification received via a task model creation screen. For example, the analysis unit 12a creates a task model based on an operation associated with a capture image selected by the user from among the captured images on a computer screen.

[0098] 25 is a diagram showing an example of a working model creation screen. For example, the working model creation screen 31 is created by the analysis unit 12a and displayed by the terminal device 20. In addition, the working model creation screen 31 may be displayed by a device different from the terminal device 20.

[0099] The analysis unit 12a extracts the operation ID from the operation log 121 and the capture image associated with the operation ID, and displays the capture image for each operation ID in the area 311. The analysis unit 12a extracts the capture image by referring to the image in the operation log 121.

[0100] Further, supplemental information regarding the captured image is displayed in the area 311. For example, the area 311 in Fig. 25 indicates that the captured image is from an airplane search screen.

[0101] The analysis unit 12a may display the captured images in the area 311 for each piece of information about the screen (title, URL) so that the user can easily select the captured images.

[0102] In addition, the analysis unit 12a may crop out only the area surrounding the operation location in the captured image so as to clearly display the operation location, and may also use information about the operation location (top, left, width, height) to highlight the portion of the captured image that corresponds to the operation location (thick-framed portion of the captured image in area 311).

[0103] The user selects and specifies the order of the captured images shown in the area 311. For example, the user moves the captured image selected in the area 311 to the area 312.

[0104] In the area 312, the selected captured images are arranged and displayed in positions corresponding to the specified order. Also, a text box for inputting the work name is provided in the area 312. In the example of Fig. 25, "on-site process search work" is input as the work name.

[0105] For example, when the save button is pressed, the analysis unit 12a stores in the work model information 122 the operation ID of the operation corresponding to the capture image placed in the area 312 and the order based on the placement position in association with the input work name.

[0106] The analysis unit 12a uses the work model to assign, to each of the events shown in the operation log 121, first information (work information) that is information indicating the corresponding work.

[0107] For example, the analysis unit 12a assigns information indicating the work included in the work model as work information to consecutive events in a sequence in which events shown in the operation log 121 are arranged according to the occurrence date and time, and which satisfy a condition regarding the degree of match with a pattern included in the work model.

[0108] Fig. 26 is a diagram for explaining the procedure for adding information. The sequence in Fig. 26 is a sequence of events in the operation log 121 arranged in the order of the occurrence date and time. Rectangular shapes correspond to each event, and alphabets represent operation IDs. For example, in Fig. 26, the operation ID "operation a" is abbreviated to "a".

[0109] Moreover, a blank figure with no operation ID written therein corresponds to an event with no specified operation ID, or an event with an operation ID unrelated to the target work model.

[0110] The analysis unit 12a sequentially examines the events in the series and identifies a range that may match the activity model. For example, the analysis unit 12a identifies consecutive events whose operation IDs match the operation ID sequence of the activity model.

[0111] For example, the analysis unit 12a detects an event with an operation ID of "a" from the sequence as the starting point. "a" is the operation ID that is first in the sequence in the activity model A.

[0112] Then, the analysis unit 12a determines whether or not the operation ID of the event following the identified starting point is included in the activity model.

[0113] Furthermore, the analysis unit 12a may also determine that an event whose operation ID is not included in the work model but whose title or URL matches that of an event determined to be included in the work model is included in the work model.

[0114] Then, the analysis unit 12a determines that the event immediately before the event that is determined not to be included in the activity model is the end point.

[0115] 26, the analysis unit 12a identifies a range 51, a range 52, a range 53, and a range 54 as ranges that may match the working model A.

[0116] Next, the analysis unit 12a determines whether or not to assign task information corresponding to the task model to the specified range based on at least one of the degree of agreement of the elements and the length of the sequence.

[0117] The analysis unit 12a looks at each event in the identified range, and if the operation ID of the event matches an operation ID in the work model, it adds 1 to the degree of match, and if the operation ID of the event does not match an operation ID in the work model (corresponding to a blank figure), it adds 0.5 to the degree of match.

[0118] The analysis unit 12a calculates the number of coincidences between the operation IDs of the events in the specified range and the operation IDs of the activity model as the length of the sequence.

[0119] For example, the analysis unit 12a calculates the degree of match for range 51 as 9.0 and the length of the sequence as 7.

[0120] The analysis unit 12a assigns task information to each event in a range in which at least one or both of the degree of match and the length of the sequence exceed a threshold. For example, the threshold of the length of the sequence may be a value such as 50% of the number of operation IDs of the task model.

[0121] For example, assume that the degree of match and the length of the sequence in the range 51 exceed the threshold value, and the task name of the task model A is “on-site process search task.” In this case, the analysis unit 12a assigns “on-site process search task” to the attribute “task name” added to the operation log 121 of the event included in the range 51.

[0122] In addition, the range identified by the analysis unit 12a may correspond to a plurality of working models. For example, as shown in Fig. 27, consider a case where a range 51 may correspond to both working model A and working model B. Fig. 27 is a diagram for explaining the procedure of adding information.

[0123] In this case, the analysis unit 12a determines which working model the identified range is closer to, based on the degree of agreement and the length of the sequence.

[0124] In the example of FIG. 27, working model A has a higher degree of agreement and a higher sequence length, so analysis unit 12a determines that working model A falls within range 51.

[0125] Furthermore, the analysis unit 12a may determine that the working model having the larger weighted sum of the degree of agreement and the length of the sequence as shown below is the working model that falls within the specified range. Weighted sum = agreement x 1 + length of series x 2

[0126] The analysis unit 12a assigns, to each of the events to which work information has been assigned, second information (case information) that is information indicating the corresponding case, based on a captured image of the computer screen, among the events shown in the operation log 121.

[0127] The events in the range to which the analysis unit 12a has assigned the task information may have occurred due to the repetition of operations related to one or more cases. Therefore, the analysis unit 12a further assigns case information to the events in the range to which the task information has been assigned.

[0128] An example in which the analysis unit 12a further assigns case information to a range to which task information corresponding to task model A has been assigned will be described.

[0129] The analysis unit 12a compares information about the computer screens corresponding to two consecutive events in a sequence in which events shown in the operation log 121 to which work information has been assigned are arranged according to the date and time of occurrence, and assigns case information to the two events based on the results of determining whether the two events correspond to the same case.

[0130] First, the analysis unit 12a determines, as the start point of the case, an event whose operation ID matches the first operation ID in the work model. For example, the first operation ID in the work model A is "a".

[0131] The analysis unit 12a determines whether to assign the same case information as the first event to the second event of two consecutive events that occur after the starting point.

[0132] Here, it is assumed that work information is added to ranges 51, 52, 53, and 54. Furthermore, the analysis unit 12a focuses only on events to which work information is added. In other words, an event subsequent to the event at the right end of range 51 is regarded as the event at the left end of range 52.

[0133] First, when the screens of two consecutive events are the same, the analysis unit 12a assigns the same case information to the two consecutive events if all of the following conditions C1 to C4 are satisfied.

[0134] Furthermore, when the screens of two consecutive events are not the same, the analysis unit 12a assigns the same case information to the two consecutive events if any of the following conditions C1 to C4 except for condition C3 is satisfied.

[0135] Whether the screens are the same or not is determined based on whether the titles of the operation logs 121 are the same or not. If the same case information is not assigned by the above procedure, the case information assigning unit 134 assigns the second event with case information different from that of the first event.

[0136] (Condition C1) The operation ID of the second event is not the first operation ID in the working model. (Condition C2) The window handles of the screens in the two events match. (Condition C3) The score of template matching, which determines whether or not a capture image of an operation location of a first event is included in a capture image of a screen of a second event, is equal to or greater than a threshold. However, the operation location is acquired from the left, top, width, and height of the capture image of the screen and the operation log 121. (Condition C4) The difference in the occurrence dates and times of the first and second events is less than or equal to the threshold.

[0137] Regardless of the above conditions, the analysis unit 12a determines that an event whose operation ID matches the last operation ID in the work model is the end point of the case. For example, the first operation ID in the work model A is “h”.

[0138] 26, the analysis unit 12a assigns the same case information (information indicating case α) to ranges 51, 52, and 531. On the other hand, the analysis unit 12a assigns the same case information (information indicating case β) to ranges 532 and 54. Even ranges such as ranges 531 and 532 that are continuous and have the same work information assigned to them may have different case information.

[0139] Returning to the explanation of Fig. 1, the visualization unit 12b visualizes an image including an object that is an object of an operation log of a predetermined unit and that is selectable by a user operation, based on the attribute information identified by the analysis unit 12a. For example, the visualization unit 12b visualizes an image including an object that is an object of an operation log of a predetermined unit and that is selectable by a user operation, based on the task ID, business ID, and case ID identified by the analysis unit 12a.

[0140] As a display mode, the visualization unit 12b performs visualization in three modes, for example, "timeline," "process visualization," and "operation sequence visualization." As a timeline, the visualization unit 12b displays a list of used applications, windows, files, and the like on a time axis for each date, task, job, and the like. For example, the visualization unit 12b visualizes a timeline as an object in multiple hierarchical layers based on one or more attributes of the user who performed the operation, the terminal who performed the operation, the date of the operation, the application name of the operation target, the window title of the operation target, and the file name of the operation target as an image including the object.

[0141] Here, an example of a screen in which an operation log is visualized in a timeline format will be described using the example of FIG. 28. FIG. 28 is a diagram showing an example of a screen in which an operation log is visualized in a timeline format. It is assumed that one or more attribute information of a task ID, a task ID, a case ID, and an operation type ID to be displayed is specified in the visualization unit 12b. As exemplified in FIG. 28, the visualization unit 12b displays a list in a timeline format in which the horizontal axis indicates time and the vertical axis indicates objects included in a focused attribute type (hierarchy) based on the operation log of the specified attribute information. In the timeline of FIG. 28, the time zone in which the item was used is displayed in color, and active or inactive is indicated by a shade of color. In addition, the visualization unit 12b hierarchically displays attributes associated with the operation log, such as a user name, a terminal name, a date, an application name, a window title, and a file name. In addition, the hierarchy can be switched and expanded / collapsed by a user operation.

[0142] The visualization unit 12b can specify an object to be passed to the RPA linking unit 12c for a characteristic work area by the following method, and further has a function of transmitting the object to the RPA linking unit 12c described below by a simple operation such as drag and drop. The user can select a displayed object. Methods for specifying an object include, for example, selecting a rectangle, a line, a selection or an operation, or selecting a range by drawing a rectangle. Also, the range selected may differ depending on the operation position. For example, when an operation is performed on a line label, multiple lines are selected, and when an operation is performed on a time axis, a time period is selected.

[0143] Also, for example, the visualization unit 12b may visualize an image including a node indicating the operation content and an edge indicating the order of the operations based on the attribute information as an image including an object. For example, the visualization unit 12b accepts the designation of one or more attribute information of the business ID, task ID, case ID, and operation type ID of the display target, and acquires a capture screen of the operation screen for the operation log to which the designated attribute information is assigned from the operation log storage unit 13a. Note that the designation is not limited to the business ID, task ID, case ID, and operation type ID, and may also accept the designation of a user or date. Then, the visualization unit 12b creates and displays a screen in which the captured screen of the acquired operation screen is used as a node and each node is connected by an edge.

[0144] For example, the visualization unit 12b may visualize processes by displaying the same types of operations for each case in a flow format in which the operations are superimposed as nodes and connected by arrows, in order to grasp the flow of operations for each task or work.

[0145] Here, a screen in which an operation process is visualized will be described using the example of FIG. 29. FIG. 29 is a diagram showing an example of a screen in which an operation process is visualized. The visualization unit 12b is assumed to be specified with one or more attribute information of the business ID, task ID, case ID, and operation type ID of the display target. As exemplified in FIG. 29, the visualization unit 12b visualizes the operation process with nodes (boxes) and edges (arrows) based on the attribute information (business ID, task ID, case ID, operation type ID) given by the analysis unit 12a. The visualization unit 12b displays the same operation type ID for each business ID or task ID by overlapping them as one node. For a characteristic work area, an object to be passed to the RPA collaboration unit 12c can be specified by object selection or range selection by drawing a rectangle, and can be transmitted to the RPA collaboration function by a simple operation such as drag & drop.

[0146] In the example of Fig. 29, the visualization unit 12b displays actions (applications, windows, operations) in a process as nodes, and displays operations on the same operation location or the same sequence index as one node. Note that the thick frame in the node indicates the operation location. The visualization unit 12b also displays edges with arrow directions that indicate the order of operations.

[0147] Also, for example, the visualization unit 12b may visualize the capture screen of the operation screen in chronological order as an image including an object based on the attribute information. For example, the visualization unit 12b accepts the designation of one or more attribute information of the business ID, task ID, case ID, and operation type ID to be displayed, and acquires the capture screen of the operation screen of the operation log to which the designated attribute information is assigned from the operation log storage unit 13a. Note that the designation is not limited to the business ID, task ID, case ID, and operation type ID, and may also accept the designation of a user or date. Then, the visualization unit 12b creates and displays a screen in which the capture screen of the acquired operation screen is arranged in chronological order. As a result, for example, the visualization unit 12b displays the operations in chronological order for each date, business, and task as an operation sequence visualization to grasp the flow of the operations.

[0148] Here, a screen on which operations are visualized in a predetermined unit will be described using the example of FIG. 30. FIG. 30 is a diagram showing an example of a screen on which operations are visualized in a predetermined unit. The visualization unit 12b acquires operation logs in units of tasks, tasks, cases, logs, and users and dates based on the attribute information (task ID, task ID, case ID, operation type ID) assigned by the analysis unit 12a, and visualizes the operation sequence using the operation logs. The visualization unit 12b displays a capture image of the operation screen with the operation location at the center in the order of operation as each node, so that the analyst can use it to identify the operation location and content.

[0149] The visualization unit 12b accepts designation of objects to be passed to the RPA collaboration unit 12c for characteristic work areas by object selection or range selection in display units, and further transmits them to the RPA collaboration unit 12c described below by simple operations such as drag and drop.

[0150] When an object included in the image visualized by the visualization unit 12b is selected by a user operation, the RPA cooperation unit 12c generates an executable automatic operation program based on the selected object. The RPA cooperation unit 12c has a function of creating an automatic operation flow based on the object selected by the user.

[0151] Here, an example of a screen on which a user creates a desired automatic operation flow will be described with reference to FIG. 31. FIG. 31 is a diagram showing an example of a screen on which a user creates a desired automatic operation flow. As shown in FIG. 31, the RPA cooperation unit 12c has a function of displaying operation nodes transmitted from the visualization unit 12b and connecting nodes with a user's operation in order for the user to create a desired automatic operation flow. The RPA cooperation unit 12c displays nodes visualized by the visualization unit 12b and connects nodes with a user's operation. For example, as illustrated in FIG. 31, the RPA cooperation unit 12c can edit an automatic operation flow by moving objects and connecting objects with arrows based on objects transmitted from the visualization unit 12b.

[0152] The RPA linking unit 12c outputs the flow created by the user as an automatic operation program. In other words, the RPA linking unit 12c outputs the generated automatic operation flow as an automatic operation program that can be executed by various RPA tools. For example, the RPA linking unit 12c reads the operation flow in order, and generates an automatic operation program by matching the application, window title, operation location, and capture image associated with the object to the format of the RPA tool.

[0153] In this way, the information processing device 10 allows the user to understand the analysis results, discover areas for improvement in business operations, and assist in considering improvement measures by enabling multifaceted observation of the analysis results and in-depth investigation of specific areas, and when an object to be the subject of automatic operation is identified, an automatic operation flow can be generated based on that object.

[0154] [Processing procedure of information processing device] Next, a procedure of the classification process executed by the information processing device 10 will be described with reference to Fig. 32. Fig. 32 is a flowchart showing an example of the procedure of the operation log acquisition process according to the embodiment.

[0155] 32, the analysis unit 12a of the information processing device 10 acquires an operation log related to operation information (step S101), and analyzes the operation log (step S102). Then, attribute information of the operation log of the analysis unit 12a is identified.

[0156] Next, the visualization unit 12b visualizes an image including an object that is an object of the operation log of a predetermined unit and is selectable by a user operation based on the attribute information identified by the analysis unit 12a (step S103). For example, the visualization unit 12b visualizes an image including an object that is an object of the operation log of a predetermined unit and is selectable by a user operation based on the work information, task information, and case information identified by the analysis unit 12a.

[0157] Thereafter, when an object included in the image visualized by the visualization unit 12b is selected by a user's operation, the RPA collaboration unit 12c performs an RPA collaboration process to generate an automatic operation flow based on the selected object (step S104).

[0158] [Effects of the embodiment] In this way, the information processing device 10 according to the embodiment acquires an operation log related to operation information, analyzes the operation log, and identifies attribute information of the operation log. Then, based on the identified attribute information, the information processing device visualizes an image including objects that are objects of a predetermined unit of operation logs and that can be selected by a user's operation. Next, when an object included in the visualized image is selected by a user's operation, the information processing device generates an executable automatic operation program based on the selected object. Therefore, the information processing device 10 can visualize an operation process and easily create an automatic operation program.

[0159] In other words, while there are conventional methods for identifying repetitive tasks through statistical analysis of operation logs to support the introduction of RPA, there was no method for inputting the results into an RPA tool. This meant that it was necessary to manually launch a separate RPA tool and create an automated operation program with the RPA tool while viewing the analysis results, which was a time-consuming process.

[0160] In contrast, the information processing device 10 according to the embodiment combines an analysis function with an editing function so that the analysis results can be directly input to an automatic operation program, and furthermore, by using a mechanism that outputs an edited scenario as an executable automatic operation program file, it is possible to reduce the effort required to create an automatic operation program. As a result, it is possible to significantly reduce the effort required to introduce RPA, and more effective business improvements can be quickly achieved.

[0161] [System configuration of the embodiment] 1 are conceptual functional components, and are not necessarily physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the information processing device 10 is not limited to that shown in the figure, and all or part of the information processing device 10 can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc.

[0162] Furthermore, all or any part of each process performed in the information processing device 10 may be realized by a CPU and a program analyzed and executed by the CPU. Furthermore, each process performed in the information processing device 10 may be realized as hardware using wired logic.

[0163] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and shown in the figures can be changed as appropriate unless otherwise specified.

[0164] [program] 33 is a diagram showing an example of a computer in which a program is executed to realize the information processing device 10. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0165] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

[0166] The hard disk drive 1031 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the information processing device 10 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1031. For example, the program module 1093 for executing the same process as the functional configuration of the information processing device 10 is stored in the hard disk drive 1031. The hard disk drive 1031 may be replaced by an SSD (Solid State Drive).

[0167] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1031. Then, CPU 1020 reads out program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1031 into RAM 1012 as necessary and executes them.

[0168] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1031, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0169] Although the embodiment of the invention made by the present inventor has been described above, the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0170] 10. Information processing device 11 Communications Department 12 Control section 12a Analysis Department 12b Visualization part 12c RPA Collaboration Department 13 Storage section 13a Operation log storage section 20 Terminal Equipment 21 Operation log acquisition unit

Claims

1. an analysis unit that acquires an operation log related to operation information, analyzes the operation log, and identifies attribute information of the operation log; a visualization unit that visualizes an image including objects that are objects of a predetermined unit of operation logs and that can be selected by a user's operation, based on the attribute information identified by the analysis unit; a linking unit that generates an executable automatic operation program based on an object selected by a user operation when the object included in the image visualized by the visualization unit is selected by the user operation; 13. An information processing device comprising:

2. the analysis unit identifies task information indicating task contents of the operation logs, identifies work information indicating work contents of the operation logs included in each identified task information, and identifies case information of the operation logs included in each identified work content; The information processing device according to claim 1, characterized in that the visualization unit visualizes an image including objects that are objects of an operation log of a predetermined unit and that can be selected by a user's operation based on the work information, the business information, and the case information identified by the analysis unit.

3. The information processing device according to claim 1, characterized in that the visualization unit visualizes a timeline as the object in multiple hierarchical hierarchies based on one or more attributes of the user who performed the operation, the terminal on which the operation was performed, the date on which the operation was performed, the application name of the target application, the window title of the target application, and the file name of the target application as an image including the object.

4. The information processing device according to claim 1 , characterized in that the visualization unit visualizes, as an image including the object, an image including nodes indicating operation contents and edges indicating an order of operations based on the attribute information.

5. The information processing apparatus according to claim 1 , wherein the visualization unit visualizes, as an image including the object, a capture screen of an operation screen in chronological order based on the attribute information.

6. The information processing apparatus according to claim 4 , wherein the linking unit displays the nodes visualized by the visualization unit, and connects the nodes together based on a user's operation.

7. An information processing method executed by an information processing device, an analysis step of acquiring an operation log related to operation information, analyzing the operation log, and identifying attribute information of the operation log; a visualization step of visualizing an image including objects that are objects of operation logs of a predetermined unit and that can be selected by a user operation, based on the attribute information identified by the analysis step; a linking step of generating an executable automatic operation program based on an object selected by a user operation when the object included in the image visualized by the visualization step is selected by a user operation; 13. An information processing method comprising:

8. an analysis step of acquiring an operation log related to operation information, analyzing the operation log, and identifying attribute information of the operation log; a visualization step of visualizing an image including objects that are objects of a predetermined unit of operation logs and that can be selected by a user operation, based on the attribute information identified by the analysis step; a linking step of generating an executable automatic operation program based on an object selected by a user operation when the object included in the image visualized by the visualization step is selected by a user operation; An information processing program characterized by causing a computer to execute the above.

Images