Information processing apparatus, flow generation method, and storage medium

Abstract

To provide an information processor for more efficiently generating the flow of an operation including a conditional branch to be processed by a computer for executing an application.SOLUTION: The information processing apparatus includes a display control unit configured to display one or more screens for receiving an operation from a user, an operation item specifying unit configured to specify an operation item corresponding to the operation from display information of the screen based on the operation on the screen, an operation component generating unit configured to generate an operation component associated with a condition corresponding to the operation item, and a flow generating unit configured to generate an operation flow from the operation component according to an operation order of the user. An information processing apparatus includes difference extraction means for extracting a difference between operation components constituting a flow, and conditional branch setting means for setting an operation component of a conditional branch before the operation component having the difference.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a flow generation method, and a program. [Background technology]

[0002] In recent years, a technology called RPA (Robotic Process Automation) has come into use to improve business productivity. RPA automates business processes by teaching software robots the routine tasks that humans perform using a keyboard and mouse, and then having the software robots perform computer operations instead of humans.

[0003] For example, Patent Document 1 discloses a software robot definition information generation system that includes an operation history detection and recording unit that records operations on a computer that has a software robot installed as an operation history in a memory unit; an operation source identification unit that determines whether the operation was performed by a software robot or a human; an operation content comparison unit that, if the operation was performed by a human, searches for repeated similar operations by comparing the operation history of the relevant location with previously recorded operation histories; and a software robot definition information generation unit that, if similar operations have been performed repeatedly, generates software robot definition information based on the operation history of the relevant location. Summary of the Invention [Problem to be solved by the invention]

[0004] An operational flow to be processed by a computer that executes an application may include conditional branching. For an information processing device that generates an operational flow including such conditional branching, there has been a demand for a mechanism that allows users, such as administrators, to generate the flow more efficiently. However, Patent Document 1 does not solve the above problem.

[0005] An object of one embodiment of the present invention is to provide an information processing device that can more efficiently generate an operation flow including conditional branches to be processed by a computer that executes an application. [Means for solving the problem]

[0006] In order to achieve the above object, one embodiment of the present invention is an information processing device that generates an operation flow to be processed by a computer that executes an application, and includes: a display control means that displays one or more screens that accept operations from a user; an operation item identification means that identifies an operation item corresponding to the operation from the display information of the screen based on the operation on the screen; an operation component generation means that generates operation components that are associated with conditions corresponding to the operation items; a flow generation means that generates an operation flow from the operation components in accordance with the order of operations performed by the user; a difference extraction means that compares the generated multiple flows and extracts differences between the operation components that make up the flow; and a conditional branch setting means that sets a conditional branch operation component before the operation component that has a difference. [Effects of the Invention]

[0007] According to one embodiment of the present invention, a flow of operations including conditional branches to be processed by a computer executing an application can be generated more efficiently. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 3] 1 is a functional configuration diagram of an example of an information processing device according to an embodiment of the present invention; [Figure 4] FIG. 10 is an explanatory diagram of an example of a procedure for generating a BOT flow. [Figure 5] FIG. 2 is an explanatory diagram of an example of a procedure for generating a BOT flow according to the present embodiment. [Figure 6]10A and 10B are explanatory diagrams illustrating an example of operations on a screen by a user such as an administrator and operations recorded by a recording function. [Figure 7] FIG. 10 is an explanatory diagram of an example of a process for generating a component from a recorded series of operation flows. [Figure 8] FIG. 10 is an explanatory diagram of an example of analysis processing when a component is generated; [Figure 9] 10A and 10B are explanatory diagrams illustrating an example of operations on a screen by a user such as an administrator and operations recorded by a recording function. [Figure 10] FIG. 10 is an explanatory diagram of an example of a process for generating a conditional branch. [Figure 11] FIG. 10 is an explanatory diagram of an example showing component information before editing and component information after editing. [Figure 12] FIG. 10 is an explanatory diagram of an example of a flow after component information is edited. [Figure 13] 10 is a diagram illustrating an example of a display screen of a flow after component information is edited and the flow is regenerated. [Figure 14] 10 is a diagram illustrating an example of a display screen of a flow after component information is edited and the flow is regenerated. [Figure 15] FIG. 10 is a diagram illustrating an example of a variation in component generation. [Figure 16] FIG. 10 is a diagram illustrating an example of a variation in component generation. [Figure 17] FIG. 10 is a diagram illustrating an example of a variation in component generation. [Figure 18] 10 is an explanatory diagram of another example of operations on the screen by a user such as an administrator and operations recorded by the recording function. FIG. [Figure 19] FIG. 10 is an explanatory diagram of an example of an operation flow when using a recording function. [Figure 20] FIG. 10 is an explanatory diagram of an example of components automatically generated from each operation recorded by the recording function. [Figure 21]FIG. 10 is an image diagram of an example of a screen that accepts operations from a user such as an administrator. [Figure 22] FIG. 10 is an explanatory diagram of an example of an operation flow when using a recording function. [Figure 23] FIG. 10 is an explanatory diagram of an example of components automatically generated from each operation recorded by the recording function. [Figure 24] FIG. 10 is an explanatory diagram of an example of a process for generating a conditional branch. [Figure 25] FIG. 10 is an explanatory diagram of an example showing component information before editing and component information after editing. [Figure 26] FIG. 10 is an explanatory diagram of an example of a flow after component information is edited. [Figure 27] FIG. 10 is an explanatory diagram of an example of a device that displays a screen of a flow generated according to the present embodiment and accepts operations from a user. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0010] [First embodiment] <System configuration> Fig. 1 is a configuration diagram of an example of an information processing system according to this embodiment. The information processing system according to this embodiment is configured by an information processing device 10 as shown in Fig. 1(A) or an information processing system 1 as shown in Fig. 1(B). The information processing device 10 in Fig. 1(A) is a computer such as a PC (Personal Computer) operated by a user such as an administrator, and accepts operations from the user via a touch panel, controller, mouse, keyboard, or the like, executes information processing according to the operations, and displays the execution results.

[0011] 1(B) is a configuration in which an information processing device 10 and a server device 12 are communicably connected via a network 18 such as the Internet or a LAN (Local Area Network). The information processing device 10 in FIG. 1(B) is a computer similar to the information processing device 10 in FIG. 1(A). The server device 12 in FIG. 1(B) can also be realized by a computer similar to that in FIG. 1(A).

[0012] The server device 12 transmits and receives data to and from the information processing device 10, thereby executing information processing in accordance with the user's operation accepted by the information processing device 10, and provides the execution result to the information processing device 10. The information processing device 10 displays the execution result provided by the server device 12.

[0013] The server device 12 may be realized by a cloud computer. The number of server devices 12 shown in FIG. 1(B) is not limited to one, and two or more may be used for distributed processing. The server device 12 may be used for downloading programs (applications) and web pages to the information processing device 10, for user login processing, or for managing various databases. The system configuration in FIG. 1 is an example.

[0014] The information processing device 10 is a PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a wearable PC, or the like, which is operated by a user.

[0015] <Hardware configuration> "computer" The information processing device 10 in Fig. 1 is realized, for example, by a computer 500 having the hardware configuration shown in Fig. 2. The configuration of the server device 12 is similar to that of the information processing device 10, and therefore a description thereof will be omitted. Fig. 2 is a diagram showing the hardware configuration of an example of a computer according to this embodiment.

[0016] The computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HD 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0017] Of these, the CPU 501 controls the overall operation of the computer 500 in accordance with a program. The ROM 502 stores programs used to drive the CPU 501, such as the IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 in accordance with the control of the CPU 501.

[0018] The display 506 displays various types of information such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various types of external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory. The network I / F 509 is an interface for data communication using the network 18. The data bus 510 is an address bus, data bus, or the like for electrically connecting each component such as the CPU 501.

[0019] The keyboard 511 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515, such as a flash memory.

[0020] The hardware configuration shown in FIG. 2 is an example, and it is not necessary to include all of the components shown in FIG. 2, or it may include components other than those shown in FIG.

[0021] <Functional configuration> An example of the functional configuration of the information processing device 10 of Fig. 1(A) will be described below. Note that the example of the functional configuration shown in Fig. 3 below is just one example. For example, in the case of the information processing device 10 of Fig. 1(B), at least a part of the functional configuration shown in Fig. 3 may be provided in the server device 12 of Fig. 1(B), and the information processing device 10 and the server device 12 may perform processing in cooperation with each other.

[0022] Fig. 3 is a functional configuration diagram of an example of an information processing device according to this embodiment. Note that the functional configuration diagram of Fig. 3 omits components that are not necessary for explaining this embodiment. For example, the information processing device 10 realizes the functional configuration of Fig. 3 by executing programs such as an OS and applications.

[0023] The information processing device 10 in Figure 3 has an application processing unit 30, a display control unit 32, an operation receiving unit 34, an operation item identification unit 40, an operation component generation unit 42, a flow generation unit 44, an operation item change unit 46, an operation component editing unit 48, a flow memory unit 50, a software robot processing unit 52, a difference extraction unit 54, and a conditional branch setting unit 56.

[0024] The application processing unit 30 is an example of a function realized by a program or service operated by the software robot processing unit 52. The software robot processing unit 52 may operate a plurality of programs or services.

[0025] The display control unit 32 displays one or more screens in accordance with image data received from the application processing unit 30. The display control unit 32 controls the display of one or more screens that accept operations from the user. In addition, the operation accepting unit 34 accepts user operations on the screens displayed by the display control unit 32.

[0026] The operation item identification unit 40 receives information about the displayed screen (screen display information) from the display control unit 32. In addition, the operation item identification unit 40 receives information about the user's operation on the displayed screen (operation position information) from the operation reception unit 34.

[0027] The operation item identification unit 40 identifies the operation item for which an operation has been accepted from the user based on the received screen display information and operation position information. Details of the operation items will be described later. The operation item change unit 46 accepts an operation from the user to change the operation item identified by the operation item identification unit 40, and changes the identified operation item.

[0028] The operation component generating unit 42 generates an operation component (described later) to which a condition corresponding to the identified operation item is associated. The operation components will be described in detail later. The operation component editing unit 48 receives an operation to edit the operation component generated by the operation component generating unit 42 from the user, and edits the operation component.

[0029] The flow generation unit 44 generates a flow from the operation parts according to the operation order of the user as described later. The flow storage unit 50 stores the generated flow. The difference extraction unit 54 compares a plurality of flows stored in the flow storage unit 50 and extracts the differences of the operation parts constituting the flow as described later. The conditional branch setting unit 56 sets (adds) the operation parts of the conditional branch to the flow as described later based on the differences of the operation parts extracted by the difference extraction unit 54 and stores them in the flow storage unit 50. The details of the processing of the difference extraction unit 54 and the conditional branch setting unit 56 will be described later.

[0030] The software robot processing unit 52 reads a flow from the flow storage unit 50. The software robot processing unit 52 displays a screen as described later according to the read flow and accepts the user's operation on the screen. The software robot processing unit 52 performs an operation on the operation reception unit 34 according to the operation received from the user and causes the application processing unit 30 to execute the processing.

[0031] <BOT Flow Generation> In this embodiment, a BOT (bot) is an example of software that simplifies the user's operation for performing processing according to a flow. A BOT is sometimes called a software robot. Here, an example of the flow of the BOT operation (hereinafter simply referred to as a flow) will be described. The generation of the flow of a normal BOT is performed as shown in FIG. 4, for example.

[0032] FIG. 4 is an explanatory diagram of an example of the procedure for generating the flow of the BOT. The flow of the BOT may be a flow that simplifies the operation of a desktop application or a flow that simplifies the operation of a cloud service.

[0033] In FIG. 4, an RPA service for simplifying business processes is described as an example. It is assumed that this RPA service provides a template of a flow pattern with high usage frequency as a template.

[0034] In the first step of generating a BOT flow, a user such as an administrator selects a template. In the next step, a user such as an administrator adds and configures components. In Figure 4, adding and configuring components is done in the steps of adding a component, configuring input settings, and configuring output settings. In Figure 4, input settings and output settings are added for each component. A component is an example of an operation part displayed on the screen, and is a part for which various definitions such as input settings and output settings can be made.

[0035] When a component is added, input settings are made to define the type of input string, etc., and output settings are made to define the type of output string, etc. In Figure 4, the necessary input and output settings are made for each added component. The procedure for adding and configuring components is repeated for the number of required interactions. In the next step, a user such as an administrator rearranges the components and adds conditions, etc., to generate a flow.

[0036] In the procedure for generating the BOT flow shown in Figure 4, it was necessary to add and configure components one by one for the required number of interactions. Also, in the procedure for generating the BOT flow shown in Figure 4, it was necessary for a user such as an administrator to rearrange the components and add conditions, etc. to generate the flow.

[0037] Therefore, in this embodiment, a BOT flow is generated as shown in Fig. 5. Fig. 5 is an explanatory diagram of an example of a procedure for generating a BOT flow according to this embodiment. Fig. 5 uses a recording function that records a series of operations such as user clicks or inputs on a screen such as a web page using a mouse, keyboard, etc.

[0038] In the first step of generating a BOT flow, a user such as an administrator starts recording using a recording function and then performs operations on a screen such as a web page. The recording function records operation information such as an operation log and operation target information such as operation target locations. In the next step, the information processing device 10 automatically generates components from the recorded operation information and operation target information.

[0039] When a component is automatically generated, the information processing device 10 performs component settings including input settings, output settings, and processing settings. The information processing device 10 generates input settings based on operation information and operation target information. The input settings inherit information such as validation checks and lists. The information processing device 10 performs output settings as necessary. The information processing device 10 also performs processing settings such as combining components and adding processing to input conditions, etc.

[0040] In the next step, the information processing device 10 automatically generates a flow in accordance with the order of operations performed by a user such as an administrator. The information processing device 10 may be configured to accept operations such as changing the order of the flow or adding conditions from a user such as an administrator.

[0041] In this embodiment, by generating a BOT flow as shown in Figure 5, necessary components can be automatically generated by performing operations on a screen such as a web page. For example, in this embodiment, input conditions for a web page or the like can be inherited, reducing the effort required for input settings. Also, in this embodiment, a flow can be automatically generated from components according to the order of operations performed by a user such as an administrator.

[0042] In this way, in this embodiment, by using the recording function, operation information and operation target information of a user such as an administrator on a screen such as a web page can be recorded, components can be generated, and a flow can be automatically generated by matching the components according to the order of operations performed by the user such as an administrator.

[0043] <Operation and recording functions> Here, an example of operations on a screen by a user such as an administrator will be described. Fig. 6 is an explanatory diagram of an example of operations on a screen by a user such as an administrator and operations recorded by the recording function.

[0044] Fig. 6(A) is an image diagram of an example of screens 1000 and 1010 to which an operation is received from a user such as an administrator and transitions. Fig. 6(B) is a diagram showing the flow of a series of operations performed on screens 1000 and 1010 of Fig. 6(A) by a user such as an administrator. Figs. 6(A) and 6(B) show the operation in which a user such as an administrator clicks the "Input daily" button on screen 1000 on the left side of Fig. 6(A), transitioning to screen 1010 on the right side of Fig. 6(A), and inputting data into screen 1010.

[0045] 6(A) and 6(B) show an operation in which a user such as an administrator inputs information into the input fields for "Start of work," "End of work," and "Reason for overtime / Remarks" on the screen 1010 of Fig. 6(A). Furthermore, Fig. 6(A) and 6(B) show an operation in which a user such as an administrator clicks the "Save and exit" button on the screen 1010 of Fig. 6(A).

[0046] For example, the flow of a series of operations by a user such as an administrator shown in FIGS. 6(A) and 6(B) can be recorded by starting and ending recording using the recording function as shown in FIG. 6(C).

[0047] Figure 7 is an explanatory diagram of an example of the process of generating components from the flow of a series of recorded operations. Figure 7(A) is the same as Figure 6(C) and shows the flow of a series of operations by a user such as an administrator, recorded using the recording function. Each operation recorded using the recording function is an example of an operation item. Figure 7(B) is an example of a component automatically generated from each operation recorded using the recording function.

[0048] From the record of an operation in which a user such as an administrator clicks the "Daily Input" button on screen 1000 of Figure 6(A), the information processing device 10 generates a component that includes, for example, "Title: Daily Input", "Action: None", and "Comment: Daily Input" as information.

[0049] From the record of an operation in which a user such as an administrator enters a time into the "Start of work" input field on screen 1010 of Figure 6(A), the information processing device 10 generates a component that includes the information "Title: Start of work", "Action: Select from list box", "Input value: 5-41, 5-55", and "Comment: Start of work".

[0050] From the record of an operation in which a user such as an administrator enters a time into the "End of work" input field on screen 1010 of Figure 6(A), the information processing device 10 generates a component that includes the information "Title: End of work", "Action: Select from list box", "Input value: 5-41, 5-55", and "Comment: End of work".

[0051] From the record of an operation in which a user such as an administrator enters time into the "Reason for overtime / Notes" input field on screen 1010 of Figure 6(A), the information processing device 10 generates a component that includes the information "Title: Reason for overtime / Notes", "Action: Text input", "Field rule: None", and "Comment: Reason for overtime / Notes".

[0052] From a record of an operation in which a user such as an administrator clicks the "Save and Exit" button on screen 1010 of Figure 6(A), the information processing device 10 generates a component that includes information such as "Title: Save and Exit", "Action: None", and "Comment: Save and Exit".

[0053] Note that Figure 7 shows an example in which the "input value" was "5-41, 5-55" based on the record of the operation of entering the time in the "start of work" and "end of work" input fields, but the input value can be any number within the input range, such as "1-99."

[0054] Here, as an example of analysis processing when generating a component, a case (start of work case) will be described in which a component is generated from a record of an operation in which a time is entered into the "start of work" input field on screen 1010 in Fig. 6(A). Also, as an example of analysis processing when generating a component, a case (reason for overtime / remarks case) will be described in which a component is generated from a record of an operation in which a time is entered into the "reason for overtime / remarks" input field on screen 1010 in Fig. 6(A).

[0055] Figure 8 is an explanatory diagram of an example of analysis processing when generating a component. Figure 8(A) is an explanatory diagram of an example of the case of starting work. Figure 8(B) is an explanatory diagram of an example of the case of reason / remarks for overtime work.

[0056] For example, in the case of the start of work, the operation of a user such as an administrator on screen 1010 in Figure 6(A) is detected, and the input field operated by the user such as an administrator is analyzed, for example, as follows, to create a component.

[0057] First, the information processing device 10 searches for character strings near the input field for "Start of work" and lists similar ones as title candidates. For example, in the example of Fig. 8A, "Start of work," "Managerial position," and "Automatic calculation" are listed as title candidates. The information processing device 10 can provisionally set, for example, the closest "Start of work" from the listed title candidates as the title.

[0058] Existing methods can be used to search for nearby strings. For example, strings placed near the start of work input field can be searched for from information extracted using web scraping technology. For example, methods such as those described at http: / / datajournalismjp.github.io / handbook / getting_data_3.html can be used to search for strings near input fields on web pages.

[0059] Furthermore, the vicinity of an input field is a position close to the input field (within a predetermined range), as indicated by the arrow in Fig. 8(A). The information processing device 10 can search for character strings located near the input field for "start of work" by listing character strings located within a predetermined range from the input field for "start of work" based on the position information of the character strings. The listed character strings may be prioritized based on the direction as viewed from the input field, for example, giving priority to character strings located at the top right of the input field.

[0060] Next, the information processing device 10 analyzes the input field for "start of work." For example, Fig. 8(A) shows an example where the input field is a list box, and a list of input values ​​(5 to 41, 0 to 55) is obtained. Then, the information processing device 10 generates a component based on the input field for "start of work," the character strings adjacent to the input field, and the analysis result of the input field.

[0061] For example, in the case of overtime reason / remarks, the component is created by detecting the operation of a user such as an administrator on screen 1010 in Figure 6(A) and analyzing the input field operated by the user such as an administrator, for example, as follows.

[0062] The information processing device 10 searches for character strings near the input field for the reason for overtime / remarks and lists the closest ones as title candidates. For example, in the example of Fig. 8(B), the reason for overtime / remarks, actual working hours, 7 hours 30 minutes, etc. are listed as title candidates. The information processing device 10 can provisionally set the closest one from the listed title candidates, for example, "Reason for overtime / remarks," as the title.

[0063] Next, the information processing device 10 analyzes the input field for "Reason for overtime work / Remarks." For example, FIG. 8(B) shows an example where the input field is a text box, and obtains the validation of the input field. The validation of the input field is information for determining whether the input character string, etc. is valid in light of rules such as specifications and grammar. The information processing device 10 generates a component based on the input field for "Reason for overtime work / Remarks," the character strings adjacent to the input field, and the analysis results of the input field.

[0064] Next, a user such as an administrator performs an operation on the screen that is different from that shown in Figure 6, as shown in Figure 9, thereby generating a conditional branch that is different from that shown in Figure 6. Figure 9 is an explanatory diagram of an example of an operation on the screen by a user such as an administrator and an operation that is recorded by the recording function.

[0065] Fig. 9(A) is an image diagram of an example of screens 1000 and 1010 to which an operation is received from a user such as an administrator and transitions. Fig. 9(B) is a diagram showing the flow of a series of operations performed on screens 1000 and 1010 of Fig. 9(A) by a user such as an administrator. Figs. 9(A) and 9(B) show the operation in which a user such as an administrator clicks the "Input daily" button on screen 1000 on the left side of Fig. 9(A), transitioning to screen 1010 on the right side of Fig. 9(A), and inputting data into screen 1010.

[0066] 9(A) and 9(B) show an operation in which a user such as an administrator inputs data into the input fields for "start of work" and "end of work" on screen 1010 of Fig. 9(A). Furthermore, Fig. 9(A) and 9(B) show an operation in which a user such as an administrator clicks the "save and exit" button on screen 1010 of Fig. 9(A).

[0067] For example, the flow of a series of operations by a user such as an administrator shown in FIGS. 9(A) and 9(B) can be recorded by starting and ending recording using the recording function as shown in FIG. 9(C).

[0068] Fig. 10 is an explanatory diagram of an example of a process for generating a conditional branch. Fig. 10(A) shows a flow in which the components of Fig. 7(B) are automatically generated according to the operation sequence of a user, such as an administrator, on screens 1000 and 1010. Fig. 10(B) shows a flow in which components are automatically generated from the flow of a series of operations recorded as in Fig. 9(C), and according to the operation sequence of a user, such as an administrator, on screens 1000 and 1010.

[0069] The difference extraction unit 54 compares the flows of Figures 10(A) and 10(B) stored in the flow storage unit 50. The difference extraction unit 54 extracts components including, as information, "Title: Reason for overtime work / remarks", "Action: Text input", "Field rule: None", and "Comment: Reason for overtime work / remarks" shown in Figure 10(C) as differences in the operation components that make up the flows of Figures 10(A) and 10(B).

[0070] The conditional branch setting unit 56 sets (adds) a conditional branch operation component before the differential operation component based on the difference between the operation components extracted by the difference extraction unit 54. The conditional branch operation component is a component that includes the information "Title: Condition", "Action: Select (True / False)", and "Condition: Not Input". In addition, the conditional branch setting unit 56 sets the branch of the conditional branch operation component so that it converges (skips) to an operation component without a difference that is after the differential operation component.

[0071] The information processing apparatus 10 according to this embodiment can accept an operation from a user to edit information on components included in an automatically generated flow, and edit the information on the components as shown in FIG. 11, for example.

[0072] Fig. 11 is an explanatory diagram showing an example of component information before editing and component information after editing. In Fig. 11, component information on the left side represents the information before editing, and component information on the right side represents the information after editing.

[0073] For example, in Figure 11, a component containing the information "Title: Enter a day", "Action: None", and "Comment: Enter a day" has been edited into a component containing the information "Title: Enter a day", "Action: Select (Yes / No)", and "Comment: Enter a day?"

[0074] Also, in Figure 11, a component containing the information "Title: Start of work", "Action: Select from list box", "Input values: 5-41, 5-55", and "Comment: Start of work" has been edited into a component containing the information "Title: Start of work", "Action: Select from list box", "Input values: 5-41, 5-55", "Output values: Number, Number", and "Comment: When does work start?"

[0075] Also, in Figure 11, a component containing the information "Title: End of work", "Action: Select from list box", "Input value: 5-41, 5-55", and "Comment: End of work" has been edited into a component containing the information "Title: End of work", "Action: Select from list box", "Input value: 5-41, 5-55", "Output value: Number, Number", and "Comment: End of work?"

[0076] Also, in Figure 11, a component containing "Title: Condition", "Action: Select (True / False)", and "Condition: Not Entered" as information has been edited into a component containing "Title: Condition", "Action: Select (True / False)", and "Condition: End of Work - Start of Work > 8.5" as conditions.

[0077] Also, in Figure 11, a component containing the information "Title: Reason for overtime / Remarks", "Action: Text input", "Field rule: None", and "Comment: Reason for overtime / Remarks" has been edited into a component containing the information "Title: Reason for overtime", "Action: Text input", "Field rule: None", "Output value: Text", and "Comment: Please enter the reason for overtime."

[0078] Also, in Figure 11, a component containing the information "Title: Save and Exit", "Action: None", and "Comment: Save and Exit" has been edited into a component containing the information "Title: Confirm Exit", "Action: Confirm (Input Data Output)", and "Comment: Register with."

[0079] By editing the component information shown in FIG. 11, the information processing device 10 can modify the pre-edited flow to an edited flow as shown in FIG. 12, which determines whether the input values ​​of the start of work component and the end of work component exceed "8.5" hours and branches the processing accordingly.

[0080] Figure 12 is an explanatory diagram of an example of the flow after editing the component information. For example, as shown in Figure 12, a user such as an administrator can edit the first component so that the transition destination when "No" is selected is the last component. In addition, a user such as an administrator can edit the fourth component so that the transition destination when the condition "End of work - Start of work > 8.5" is met is the "Reason for overtime" component. Furthermore, a user such as an administrator can edit the fourth component so that the transition destination when the condition "End of work - Start of work > 8.5" is not met is the "Confirm completion" component.

[0081] Figures 13 and 14 show examples of screen displays of a flow after component information has been edited and the flow has been regenerated. Screen 2000 in Figures 13 and 14 is an example of a screen display 2000 that has been polished to look more interactive by editing component information included in the automatically generated flow as shown in Figure 11.

[0082] 13 is an example of a flow when the condition "end of work - start of work > 8.5" is satisfied in the fourth component. Also, the screen 2000 in FIG. 14 is an example of a flow when the condition "end of work - start of work > 8.5" is not satisfied in the fourth component. For example, the user can execute the flow in FIG. 12 by operating the screen 2000 in FIG. 13 or FIG. 14 displayed on the information processing device 10.

[0083] The analysis process when creating a component shown in Fig. 8 is one example, and a component may be created by detecting an operation on a radio button as shown in Fig. 15 and analyzing an input field operated by a user such as an administrator. Fig. 15 is a diagram of an example that explains variations in component generation.

[0084] 15(A) and 15(B) are examples of screens 1040 and 1050 including radio buttons. The information processing device 10 searches for character strings near the input fields of the radio buttons and can tentatively set the closest ones as title candidates. The information processing device 10 can also obtain a list of options by analyzing the radio buttons in the input fields. In this way, the information processing device 10 can create components from the screens 1040 and 1050 including the radio buttons.

[0085] FIG. 16 is a diagram illustrating an example of variations in component generation. FIGS. 16(A) to 16(C) are examples of inheriting validation check logic for input fields. FIG. 16(A) is an example of a screen 1060 including an input field for inputting an email address. FIG. 16(B) is an example of a screen 1070 including input fields for inputting a last name (in Roman letters) and a first name (in Roman letters). FIG. 16(C) is an example of a screen 1080 including an input field for inputting an age.

[0086] When a user such as an administrator inputs an email address using half-width alphanumeric characters, the information processing device 10 can inherit the logic of validation checks for half-width alphanumeric characters in the input fields for last name (Roman letters), first name (Roman letters), and age.

[0087] FIG. 17 is a diagram illustrating an example of variations in component generation. FIGS. 17(A) and 17(B) are examples of screens 1090 and 1100 that include list boxes. The information processing device 10 can search for character strings adjacent to the input field of the list box and provisionally set the closest ones as title candidates. The information processing device 10 can also obtain a list of options by analyzing the list box of the input field. In this way, the information processing device 10 can create components from the screens 1090 and 1100 that include list boxes.

[0088] Next, other operations on the screen by a user such as an administrator will be described by way of example. Fig. 18 is an explanatory diagram of another example of operations on the screen by a user such as an administrator and operations recorded by the recording function.

[0089] Fig. 18 is an image diagram of an example of a screen that accepts operations from a user such as an administrator. Screen 1200 shows an example of a display for registering "Card Services and Payments" without a family card. For example, the flow of a series of operations performed by a user such as an administrator on screen 1200 in Fig. 18 can be recorded by starting and ending recording using the recording function, as shown in Fig. 19.

[0090] FIG. 19 is an explanatory diagram of an example of the flow of operations when using the recording function. In step S100, a user such as an administrator selects a "card brand" from screen 1200. In step S102, a user such as an administrator enters a "PIN" on screen 1200. In step S104, a user such as an administrator enters a "PIN (confirmation)" on screen 1200. In step S106, a user such as an administrator selects a "purpose of transaction" from screen 1200. In step S108, a user such as an administrator selects a "payment date" from screen 1200. In step S110, a user such as an administrator selects either "agree and apply" or "do not apply" for a "family card" on screen 1200. FIG. 19 shows an example of not applying for a family card. In step S112, a user such as an administrator selects the "Next" button on screen 1200.

[0091] From each operation in Fig. 19 recorded by the recording function, the information processing device 10 automatically generates components as shown in Fig. 20. Fig. 20 is an explanatory diagram of an example of components automatically generated from each operation recorded by the recording function.

[0092] From a record of an operation in which a user, such as an administrator, selects a card brand from the "Card Brand" input field on screen 1200 of Figure 18, the information processing device 10 generates a component that includes the information "Title: Card Brand", "Comment: Card Brand", "Action: List Selection", and "Input Value: ○○○ / △△△".

[0093] Furthermore, from a record of an operation in which a user such as an administrator inputs a PIN into the "PIN" input field on screen 1200 in Fig. 18, the information processing device 10 generates a component including, as information, "Title: PIN", "Comment: PIN", "Action: Text", and "Input value: 4-digit number". Thereafter, components such as those shown in Fig. 20 are generated in a similar manner.

[0094] Fig. 21 is an image diagram of an example of a screen that accepts operations from a user such as an administrator. Screen 1200 in Fig. 21 shows an example of a display for registering "Card Services and Payments" when a family card is present. For example, the flow of a series of operations performed by a user such as an administrator on screen 1200 in Fig. 21 can be recorded by starting and ending recording using the recording function, as shown in Fig. 22.

[0095] 22 is an explanatory diagram of an example of the flow of operations when using the recording function. Steps S200 to S208 are the same as steps S100 to S108 in FIG. 19. In step S210, a user such as an administrator selects "Agree and apply" or "Do not apply" for "Family card" on screen 1200. FIG. 22 shows an example of applying for a family card. In steps S212 to S232, a user such as an administrator performs operations to apply for a family card on screen 1200 in FIG. 21. In step S234, a user such as an administrator selects the "Next" button on screen 1200.

[0096] From each operation in Fig. 22 recorded by the recording function, the information processing device 10 automatically generates components as shown in Fig. 23. Fig. 23 is an explanatory diagram of an example of components automatically generated from each operation recorded by the recording function.

[0097] From a record of an operation in which a user such as an administrator selects a card brand from the "Card Brand" input field on screen 1200 of Figure 21, the information processing device 10 generates a component including the information "Title: Card Brand", "Comment: Card Brand", "Action: List Selection", and "Input Value: ○○○ / △△△".

[0098] Furthermore, from a record of an operation in which a user such as an administrator inputs a PIN into the "PIN" input field on screen 1200 in Fig. 21, the information processing device 10 generates a component including, as information, "Title: PIN", "Comment: PIN", "Action: Text", and "Input value: 4-digit number". Thereafter, components such as those shown in Fig. 23 are generated in a similar manner.

[0099] Fig. 24 is an explanatory diagram of an example of processing for generating a conditional branch. Fig. 24(A) is a flow automatically generated in accordance with operations by a user, such as an administrator, on screen 1200 in Fig. 18. Fig. 24(B) is a flow automatically generated in accordance with operations by a user, such as an administrator, on screen 1200 in Fig. 21.

[0100] The difference extraction unit 54 compares the flows of Figures 24(A) and 24(B) stored in the flow storage unit 50. The difference extraction unit 54 extracts differences in the operational components that make up the flows of Figures 24(A) and 24(B).

[0101] The conditional branch setting unit 56 sets (adds) a conditional branch operation component before the differential operation component based on the difference between the operation components extracted by the difference extraction unit 54. The conditional branch operation component is a component that includes the information "Title: Condition", "Action: Select (True / False)", and "Condition: Not Input". In addition, the conditional branch setting unit 56 sets the branch of the conditional branch operation component so that it converges (skips) to an operation component without a difference that is after the differential operation component.

[0102] The information processing apparatus 10 according to this embodiment accepts an operation from the user to edit information on a component included in an automatically generated flow, and can edit information on a component of an operational part for a conditional branch, for example, as shown in FIG.

[0103] Fig. 25 is an explanatory diagram showing an example of component information before editing and component information after editing. In Fig. 25, the component information on the upper side represents the state before editing, and the component information on the lower side represents the state after editing. Fig. 25 also shows the conditional branch operation component among the components included in the automatically generated flow, and does not show other components.

[0104] For example, in Figure 25, a component containing the information "Title: Condition", "Action: Select (True / False)", and "Condition: Not entered" has been edited into a component containing the conditions "Title: Condition", "Action: Select (True / False)", and "Condition: Family card = Agree and apply".

[0105] In addition to editing the component information shown in Figure 25, by editing to display a screen that is more polished to look interactive, the information processing device 10 can modify the pre-editing flows of Figures 20 and 23 into an edited flow as shown in Figure 26, which determines whether to agree and apply for a family card or not and branches the processing accordingly.

[0106] Figure 26 is an explanatory diagram of an example of the flow after editing the component information. For example, as shown in Figure 26, a user such as an administrator can edit the seventh component so that the transition destination when the condition "Family card = Agree and apply" is met will be the eighth component. Furthermore, a user such as an administrator can edit the seventh component so that the transition destination when the condition "Family card = Agree and apply" is not met will be the "Next" component.

[0107] The information processing device 10 automatically generates a flow such as that shown in Fig. 26 according to the order of operations performed by a user such as an administrator on the screen 1200. In this manner, in this embodiment, the recording function is used to record operations performed by a user such as an administrator on the screen 1200, thereby automatically generating a screen for a flow to be processed by a BOT.

[0108] Furthermore, a user such as an administrator can generate a screen that looks more interactive by editing the information of the automatically generated components and the automatically generated flow. For example, a user can execute the flow shown in Fig. 26 by operating the screen that looks more interactive and is displayed on the information processing device 10.

[0109] The generated flow can be displayed on the screen of various devices shown in Fig. 27, for example, and operations can be accepted from the user. Fig. 27 is an explanatory diagram of an example of a device that displays a screen of the flow generated according to this embodiment and accepts operations from the user.

[0110] FIG. 27 shows a desktop PC 3000, an electronic whiteboard 3002, an inkjet printer 3004, a smartphone 3006, a notebook PC 3008, an omnidirectional imaging device 3010, a video conference terminal 3012, a projector 3014, and an image forming device 3016 such as an MFP, as examples of devices that display a screen of a flow generated by this embodiment and accept operations from a user.

[0111] As described above, according to this embodiment, a flow to be processed by a software robot can be generated more efficiently.

[0112] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0113] The devices described in the examples merely represent one of multiple computing environments for implementing the embodiments disclosed herein. The present invention is not limited to these embodiments, and the components in these embodiments include those that would be easily conceived by a person skilled in the art, those that are substantially identical, and those that are within the scope of what is called equivalents. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the present embodiments. [Explanation of symbols]

[0114] 1. Information Processing Systems 10. Information processing equipment 12 Server equipment 18 Network 30 Application processing section 32 Display control unit 34 Operation reception section 40 Operation item identification part 42 Operation component generation unit 44 Flow Generation Unit 46 Operation item change section 48 Operational Components Editorial Department 50 Flow memory unit 52 Software robot processing section 54 Difference extraction part 56 Conditional branch setting section 1000, 1010, 1200, 2000 screen [Prior art documents] [Patent documents]

[0115] [Patent Document 1] Japanese Patent Application Publication No. 2019-169044

Claims

1. An information processing device that generates an operation flow to be processed by a computer that executes an application, a display control means for displaying one or more screens for receiving operations from a user; an operation item specifying means for specifying an operation item corresponding to an operation from display information on the screen based on the operation on the screen; an operation component generating means for generating an operation component associated with a condition according to the operation item; a flow generation means for generating an operation flow from the operation components in accordance with the operation order of the user; a difference extraction means for comparing the generated flows and extracting differences in the operational components that constitute the flows; a conditional branch setting means for setting a conditional branch operation component before the operation component having a difference; An information processing device having the above.

2. The conditional branch setting means sets an operation component for the conditional branch that branches depending on whether a predetermined condition is satisfied.

2. The information processing device according to claim 1.

3. The conditional branch setting means sets a branch when a predetermined condition is satisfied or not satisfied so as to skip to an operation part that is located after the operation part that has a difference and has no difference.

3. The information processing device according to claim 2.

4. an operation component editing means for displaying operation components of the flow and receiving an editing operation for the operation component of the conditional branch from the user; The information processing device according to claim 1 , further comprising:

5. The operation part editing means displays the operation parts of the flow in an interactive format.

5. The information processing device according to claim 4.

6. The operational flow is used by a software robot to operate the application.

6. The information processing device according to claim 1, wherein:

7. A flow generation method executed by an information processing device that generates an operation flow to be processed by a computer that executes an application, comprising: a display control step of displaying one or more screens for receiving operations from a user; an operation item specifying step of specifying an operation item corresponding to the operation from display information on the screen based on the operation on the screen; an operation component generating step of generating an operation component associated with a condition according to the operation item; a flow generation step of generating an operation flow from the operation components in accordance with the operation order of the user; a difference extraction step of comparing the generated flows and extracting differences in the operational components that constitute the flows; a conditional branch setting step of setting a conditional branch operation component before the operation component having a difference; A flow generation method comprising:

8. The conditional branch setting step sets an operation component for the conditional branch that branches depending on whether a predetermined condition is satisfied. The flow generation method according to claim 7.

9. The conditional branch setting step sets a branch when a predetermined condition is satisfied or not satisfied so as to skip to an operation part that is located after the operation part that has a difference and has no difference. The flow generation method according to claim 8.

10. an operation component editing step of displaying operation components of the flow and receiving an editing operation for the operation component of the conditional branch from the user; 10. The method of claim 7, further comprising:

11. The operation component editing step displays the operation components of the flow in an interactive format. The flow generation method of claim 10.

12. The operational flow is used by a software robot to operate the application.

12. The flow generation method according to claim 7, wherein:

13. an information processing device that generates an operation flow to be processed by a computer that executes an application, a display control means for displaying one or more screens for receiving operations from a user; an operation item specifying means for specifying an operation item corresponding to an operation from display information on the screen based on the operation on the screen; an operation component generating means for generating an operation component associated with a condition according to the operation item; a flow generation means for generating an operation flow from the operation components in accordance with the operation order of the user; a difference extraction means for comparing the generated flows and extracting differences in the operational components constituting the flows; a conditional branch setting means for setting a conditional branch operation component before the operation component having a difference; A program to function as a

Images