Task analysis system, task analysis device, task analysis method, and program

The business analysis system addresses the challenge of large log files by focusing on operation positions and processing images to reduce data, allowing for efficient data collection and analysis without increasing file size, thereby optimizing storage and analysis efficiency.

WO2024236708A1PCT designated stage expired Publication Date: 2024-11-21NT T INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/018165
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing business analysis methods relying on manual identification of tasks suitable for RPA face challenges in efficiently collecting data without increasing log file size, as capturing peripheral information leads to larger log files and resource pressure.

Method used

A business analysis system that captures operation screens, generates images focusing on the operation position, processes images to reduce data, and combines them with low-resolution background images to create a composite log file, allowing for detailed analysis without increasing file size.

Benefits of technology

Enables effective data collection for business analysis without expanding log file size, reducing storage capacity requirements while retaining necessary information for visualization and analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023018165_21112024_PF_FP_ABST
    Figure JP2023018165_21112024_PF_FP_ABST
Patent Text Reader

Abstract

A task analysis system according to an embodiment of the present invention comprises a terminal to be analyzed on which an application is installed and an analysis terminal capable of communicating with the terminal to be analyzed. The terminal to be analyzed is provided with a log acquisition unit. The analysis terminal is provided with a visualization unit. The log acquisition unit: when the application is operated, captures the operation screen of the application to generate a first image; crops areas including operation positions of a user on the operation screen from the first image to generate second images; performs image processing on the first image to generate a third image comprising a reduced amount of data; and generates a log file in which the third image and each second image form a set. The visualization unit: acquires a second image related to an analysis target and the third image that forms a set with this second image from the log file; and combines the acquired second image and third image to create an operation flow diagram regarding the operation of the application.
Need to check novelty before this filing date? Find Prior Art

Description

Work analysis system, work analysis device, work analysis method, and program

[0001] One aspect of the present invention relates to a task analysis device, a task analysis method, and a program.

[0002] In business activities, it is particularly important to carry out daily operations efficiently and smoothly. Because the business environment changes due to both external and internal factors, it is necessary to continuously improve business operations. To improve business operations, it is effective to carry out the following steps in order: "Understanding the actual situation," "Discovering and analyzing areas for improvement," "Drafting improvement plans," and "Implementing improvement plans."

[0003] Robotic Process Automation (RPA) is a software tool that performs computer operations on behalf of humans and is becoming increasingly popular as a powerful tool for improving business efficiency. However, to effectively implement RPA, it is necessary to identify tasks suitable for RPA and comprehensively analyze the entire business process. Traditionally, this type of analysis has been performed manually, which tends to depend on the analyst's know-how and skills, resulting in ineffective use of RPA. Therefore, a method is known for extracting routine operations by recording user terminal operations as an operation log and using software to mechanically repeat and statistically analyze the tasks based on the operation log (see, for example, Patent Document 1).

[0004] International Publication No. 2020 / 235085

[0005] Patent Literature 1 proposes capturing operation logs as images and creating an operation flow diagram to visualize user operation procedures. Many operating systems (OS) have the ability to capture monitor displays, which has the advantage of being system-independent and reducing costs. By utilizing this method, repetitive tasks that are candidates for RPA can be easily identified without relying on know-how or skills.

[0006] However, if you focus too much on the operation area when capturing an image of the operation screen, peripheral information will be lost. On the other hand, if you increase the image size to capture more peripheral information, the log file size will become too large. In this way, the more useful data you try to capture, the larger the log file will become, putting a strain on resources. A solution to these issues was eagerly awaited.

[0007] The present invention has been made in light of the above circumstances, and aims to provide a technique that can collect data related to business analysis without increasing the size of the log file.

[0008] A task analysis system according to one aspect of the present invention includes an analyzed terminal having an application installed thereon and an analyzing terminal capable of communicating with the analyzed terminal. The analyzed terminal includes a log acquisition unit. The analyzing terminal includes a visualization unit. The log acquisition unit captures an operation screen of the application when the application is operated to generate a first image, crops from the first image an area including the user's operation position on the operation screen to generate a second image, processes the first image to generate a third image with reduced data volume, and generates a log file by combining the third image and the second image. The visualization unit acquires from the log file the second image related to the analysis target and a third image paired with the second image, and creates an operation flow diagram related to the operation of the application by combining the acquired second image and third image.

[0009] According to one aspect of the present invention, data related to business analysis can be collected without increasing the size of the log file.

[0010] FIG. 1 is a diagram showing an example of an operation flow diagram using a capture image of an operation screen. FIG. 2 is a functional block diagram showing an example of a business analysis system according to an embodiment. FIG. 3 is a diagram showing an example of an operation screen. FIG. 4 is a diagram showing an example of a capture image. FIG. 5 is a diagram showing an example of a third image. FIG. 6 is a diagram showing an example of a composite image generated by the visualization unit 21a. FIG. 7 is a diagram showing an example of a composite image generated by the visualization unit 21a. FIG. 8 is a flowchart showing an example of a processing procedure in the business analysis system according to an embodiment.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, an operation flow diagram using a captured image of an operation screen of a business application will be described. In the embodiment, a transit guide application is assumed as the business application, but the present invention is not limited to this.

[0012] FIG. 1 shows an example of an operational flow diagram using captured images of an operation screen. An operational flow diagram is a diagram that visualizes the state transitions of an operation screen that change in response to user operations, and is the basis for business analysis. In FIG. 1, node (1) is a captured image showing an action (application, window, operation) within a process. In other words, node (1) is a captured image of an actual operation screen. Therefore, in order to visualize a business process with a large number of operations, it is necessary to record many captured images, which results in an increase in the amount of data. Below, we disclose a technology that makes it possible to reduce the amount of data. Note that operations on the same operation location or the same sequence index are displayed as a single node.

[0013] Edges (2) are arrow icons connecting nodes, and the direction of the arrows indicates the order of operations. As there are multiple nodes (1), multiple edges (2) are drawn between them. Typically, nodes (1) include operation points (3) that accept manual input operations by the user. The operation points (3) are, for example, input fields for text information, clickable icons containing hyperlinks, or pull-down menus containing multiple options. Technically, the operation points (3) are logs whose type attribute is indicated by operation.

[0014] Furthermore, the operation flow diagram may display additional information related to the operations on the operation screen in text format. The additional information may include, for example, items such as window title, URL / file path, operation location, operation type, number of orders, and input value type.

[0015] 2 is a functional block diagram showing an example of a business analysis system according to an embodiment. The system shown in FIG. 2 includes an analyzed terminal 1 that has a business application installed and is the subject of business analysis, and an analyzing terminal 2 that can communicate with the analyzed terminal 1. The two are connected via a hub, for example, and are communicatively connected via an office network or the like.

[0016] The analyzed terminal 1 is a computer (information processing device) that includes a storage unit 12, a processor 11, and a display 13, and realizes its functions through a program. The processor 11 includes a log acquisition unit 11a as a processing function related to the embodiment. When a business application is operated on the analyzed terminal 1, the log acquisition unit 11a captures a portion of the operation screen related to the operation and obtains a captured image. Figure 3 is a diagram showing an example of the operation screen. The operation screen as the first image is an image corresponding to the background.

[0017] 4 shows an example of a capture image. The capture image as the second image is a small area (area) smaller than the first image, including the user's operation position (e.g., click position) on the operation screen of the display 13. Capture image (A) shows the area when the user enters "Tokyo" in the "Departure Point" dialog box, for example. Capture image (B) shows the area when the ticket card use radio button is clicked on the "Fare" line, for example. Both capture images are cut out as a rectangular area including the coordinates of the position operated by the user (operation position coordinates).

[0018] The log acquisition unit 11a generates a log file by associating the captured image with attached information related to the operation when the captured image was generated. The log file is stored in the storage unit 12 (log file 12a). That is, in the log file, the captured image is managed by being associated (linked) with the attached information by the user. The attached information may be the user's operation history (operation log), the application name (executable file name) of the target system, the window title, or a uniform resource locator (URL). In short, information required for associating with the first image can be used as the attached information.

[0019] The log acquisition unit 11a also processes the image data of the operation screen to generate a third image with reduced data volume, and stores the third image together with the captured image in the log file 12a.

[0020] FIG. 5 is a diagram showing an example of a third image. As shown in FIG. 5, the amount of data can be reduced by lowering the resolution of the original image through image processing. The file size of the third image is significantly smaller than the file size of the first image at the original resolution. In addition, the file size of an image can also be reduced by image processing such as grayscaling and binarization. These image processing techniques can also be used in combination.

[0021] The analyzing terminal 2 (FIG. 2) is a computer (information processing device) that includes a processor 21 and a storage unit 22 and realizes its functions using a program 22a. The processor 21 includes a visualization unit 21a as a processing function according to the embodiment. Regarding task analysis of the analyzed terminal 1, the visualization unit 21a acquires, from the log file 12a, a capture image related to the analysis target and a third image corresponding to the attached information of the capture image. The visualization unit 21a then synthesizes the acquired capture image and the third image to generate a composite image. Based on this composite image, the visualization unit 21a creates an operation flow diagram such as that shown in FIG. 1. The functions of the visualization unit 21a are realized based on the program 22a stored in the storage unit 22. That is, the program 22a causes the computer to function as the analyzing terminal 2 equipped with the visualization unit 21a.

[0022] 6 and 7 are diagrams showing examples of composite images generated by the visualization unit 21a. The composite image of Fig. 6 is generated by combining the captured image (A) with the third image of Fig. 5. The composite image of Fig. 7 is generated by combining the captured image (B) with the third image of Fig. 5. Here, the captured image is simply superimposed at the same position in the third image.

[0023] FIG. 8 is a flowchart illustrating an example of a processing procedure in a task analysis system according to an embodiment. In FIG. 8, when the analyzed terminal 1 detects a user operation event (step S1), it captures information related to the operation event (attached information) and the operation screen on which the operation was performed, acquiring a first image and the operation position (coordinates) (step S2). Next, the analyzed terminal 1 generates a captured image by cropping the area including the operation position from the first image. The analyzed terminal 1 then processes the first image to reduce its resolution and generates image data for storage (step S4). This image data for storage is the third image. The analyzed terminal 1 then combines the captured image with the third image, overlays the captured image on the original position of the third image, and stores the combined image (step S5). The analyzing terminal 2 acquires a composite image from the log file 12a of the analyzed terminal 1 during task analysis, and creates an operation flow diagram.

[0024] As described above, according to the embodiment, the first image of the system application to be operated by the user is image-processed, and the resolution and number of colors are reduced to reduce the amount of data, and the image is then recorded in the log file 12a. On the other hand, the area that has changed during the user's terminal operation is recorded as is as a captured image, and during task analysis, the low-resolution first image and the captured image of the operation area are combined to create an operation flow diagram.

[0025] According to the embodiment, most of the image area that does not reflect user operations is saved with reduced resolution. On the other hand, the captured image that reflects user operations is saved with the same resolution. As a result, the amount of data to be saved on the analyzed terminal 1 is significantly reduced, while information necessary for task analysis is not lost. In other words, the storage capacity required on the analyzed terminal 1 for recording information on the operation screen can be reduced.

[0026] That is, according to the embodiment, in a first image obtained by capturing an operation screen, the resolution is changed between an area close to the operation position and other areas and saved. This reduces the size of the log file and allows detailed information about the operation to be saved while preserving bird's-eye view information of the operation screen. In other words, when acquiring a log, the resolution is changed between the necessary area and other areas on the operation screen, so that the area necessary for analysis and visualization is appropriately saved and the resolution of other areas is reduced.

[0027] For these reasons, according to the embodiment, it is possible to collect data related to business analysis without increasing the size of the log file, and ultimately to significantly reduce the size of the log file on the analyzed terminal compared to existing technologies.

[0028] It should be noted that this invention is not limited to the above-described embodiment. For example, in order to reduce the amount of data on the operation screen for operation analysis, various known methods can be applied instead of the above-described method. Furthermore, the exchange of various data between the analyzed terminal 1 and the analyzing terminal 2 is not limited to communication via a network. For example, it is also possible to store the log file 12a in a storage device (such as a USB memory) and exchange data directly.

[0029] That is, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0030] REFERENCE SIGNS LIST 1... Analyzed terminal 2... Analyzing terminal 3... Server 11... Processor 11a... Log acquisition unit 12... Memory unit 12a... Log file 13... Display 21... Processor 21a... Visualization unit 22... Memory unit 22a... Program.

Claims

1. A business analysis system comprising: an analyzed terminal having an application installed; and an analyzing terminal capable of communicating with the analyzed terminal, wherein the analyzed terminal is equipped with a log acquisition unit; and wherein the analyzing terminal is equipped with a visualization unit, wherein the log acquisition unit captures an operation screen of the application when the application is operated to generate a first image, wherein an area including a user's operation position on the operation screen is cut out from the first image to generate a second image, wherein the first image is image-processed to generate a third image with a reduced amount of data, wherein the third image and the second image are set to generate a log file, and wherein the visualization unit acquires from the log file a second image related to an analysis subject and a third image set with the second image, and wherein the visualization unit synthesizes the acquired second image and third image to create an operation flow diagram related to the operation of the application.

2. The business analysis system according to claim 1, wherein the log acquisition unit generates the third image by reducing the resolution of the first image to a lower value than the original image.

3. The business analysis system according to claim 1, wherein the log acquisition unit generates the third image by grayscaling the first image.

4. The business analysis system according to claim 1, wherein the log acquisition unit binarizes the resolution of the first image to generate the third image.

5. A task analysis device comprising: a means for communicating with an analyzed terminal having an application installed; and a visualization unit, wherein the visualization unit acquires from the analyzed terminal a third image with reduced data volume, the third image being generated by cutting out a second image including a user's operation position on the operation screen from a first image generated by capturing an operation screen of the application when the application is operated, and a second image related to an analysis subject and set with the third image, and synthesizes the acquired second image and third image to create an operation flow diagram related to the operation of the application.

6. A task analysis method for a task analysis system comprising an analyzed terminal having an application installed and an analyzing terminal capable of communicating with the analyzed terminal, wherein the analyzed terminal captures an operation screen of the application when the application is operated to generate a first image, the analyzed terminal cuts out an area including a user's operation position on the operation screen from the first image to generate a second image, the analyzed terminal processes the first image to generate a third image with reduced data volume, the analyzed terminal generates a log file by setting the third image and the second image, the analyzing terminal obtains from the log file a second image related to the subject of analysis and a third image set with the second image, and the analyzing terminal synthesizes the obtained second image and third image to create an operation flow diagram related to the operation of the application.

7. A program comprising instructions for causing a computer to function as the business analysis device according to claim 5.

Citation Information

Patent Citations

  • Software robot definition information generation system, software robot definition information generation method, and program

    JP2019169044A

  • Data processing device, data processing method, and data processing program

    WO2022054262A1

  • Processing device, processing method, and processing program

    WO2023281718A1