Execution method

Abstract

To provide an execution method capable of reliably executing automatic operation of an image processing device.SOLUTION: The execution method configured to execute an operation process of an image processing device by a terminal device comprises: a step of acquiring an operation script obtained by storing an operation element based on a position designated by a user within an operation screen of the image processing device; and a step of executing an operation process concerning the operation element, based on the operation script.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a method of implementation. [Background technology]

[0002] In an image processing device such as an MFP (Multi-Function Peripheral), a plurality of operation elements (GUIs (Graphical User Interfaces) such as operation buttons and icons) are arranged at specific positions on an operation screen. For example, Patent Document 1 discloses a configuration in which the display positions of the operation elements can be customized.

[0003] Also, there are known image processing devices that allow a user to remotely operate the operation screen of the image processing device using a terminal device. In such image processing devices, it is possible to automate and simplify the operation process related to a specific operation element (e.g., copy, scan, etc.) by setting up automatic operations such as RPA (Robotic Process Automation). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-208910 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when performing automatic operation of an image processing device, the display position of each operation element may be different from the position at the time of setting the automatic operation. When the display position of a specific operation element is different from the position at the time of setting, there is a risk that the operation process of the specific operation element will not be executed even if the automatic operation is executed.

[0006] SUMMARY OF THE DISCLOSURE An object of the present invention is to provide an execution method which makes it possible to reliably perform automatic operation of an image processing device. [Means for solving the problem]

[0007] The method according to the present invention comprises the steps of: 1. A method for executing an operation process of an image processing device on a terminal device, comprising: acquiring, by the terminal device, an operation script that stores an operation element based on a position designated by a user on an operation screen of the image processing device; executing an operation process for the operational element based on the operation script; has. Effect of the Invention

[0008] According to the present invention, automatic operation of an image processing device can be reliably performed. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an overall configuration of a system capable of executing a method according to an embodiment of the present invention. [Figure 2A] FIG. 2 is a diagram showing a display example of an operation screen of the image processing apparatus. [Figure 2B] FIG. 2 is a diagram showing a display example of an operation screen of the image processing apparatus. [Figure 2C] FIG. 2 is a diagram showing a display example of an operation screen of the image processing apparatus. [Figure 3A] 11 is a diagram showing an example of a display of an operation screen of the image processing device on a display unit of a terminal device. FIG. [Figure 3B] 11 is a diagram showing an example of a display of an operation screen of the image processing device on a display unit of a terminal device. FIG. [Figure 4] 13 is a flowchart showing a flow of generating an operation script by the terminal device. [Diagram 5] 10 is a flowchart showing a flow of execution of an operation process of the image processing device by the terminal device. [Figure 6]11 is a diagram showing an example of a display of an operation screen of the image processing device on a display unit of a terminal device. FIG. [Figure 7] FIG. 1 is a diagram illustrating an overall configuration of a system including a plurality of image processing devices. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Fig. 1 is a diagram showing an outline of the overall configuration of a system capable of executing a method according to an embodiment of the present invention.

[0011] 1, the system 1 has an image processing device 2 and a terminal device 3, and is a system that allows the image processing device 2 to be remotely operated from the terminal device 3. In this system 1, when remotely operated from the terminal device 3, it is possible to automatically execute an operation process related to a specific operation element based on a predetermined operation script.

[0012] The image processing device 2 is, for example, a multifunction device such as an MFP having functions such as copying and scanning. The image processing device 2 has a display unit 21, a control unit 22, a communication unit 23, a storage unit 24, and the like. The control unit 22 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU reads out a program corresponding to the processing contents from the ROM and loads it in the RAM, and in cooperation with the loaded program, centrally controls the operation of each block of the image processing device 2, and the like. At this time, various data stored in the storage unit 24 is referenced. The storage unit 24 is, for example, composed of a non-volatile semiconductor memory (so-called flash memory) or a hard disk drive.

[0013] The control unit 22 is capable of transmitting and receiving various data via the communication unit 23 to and from a terminal device 3 connected to a communication network such as a local area network (LAN) or a wide area network (WAN).

[0014] The display unit 21 is configured, for example, with a liquid crystal display (LCD) with a touch panel. The display unit 21 displays various operation screens, image states, operation statuses of each function, etc., according to a display control signal input from the control unit 22. The display unit 21 also includes various operation keys such as a numeric keypad and a start key, and outputs operation signals to the control unit 22 upon receiving various input operations by the user.

[0015] 2A, a plurality of operation elements A1 to A3 are displayed side by side on the display unit 21. The plurality of operation elements are GUIs relating to functions (e.g., copy, scan, etc.) that the user wishes to execute on the image processing device. When the user operates one of the plurality of operation elements (a specific operation element), a function corresponding to the specific operation element is executed on the image processing device 2. The user's operation may be, for example, an operation of positioning a cursor on a screen and clicking, an operation of touching a touch panel, etc.

[0016] For example, when the user operates an operation element (e.g., A2) indicating "scan," the operation screen of the display unit 21 transitions to an operation screen for detailed settings of "scan," as shown in FIG. 2B. In the operation screen for detailed settings, for example, the color (color, black and white, etc.) for "scanning," the read size, resolution, file format, single-sided and double-sided settings, etc. can be set. When the user inputs each setting and operates the operation element for "start," the "scan" function of the image processing device 2 is executed.

[0017] Moreover, in the display unit 21, it is possible to customize the display position of each operation element. For example, assume that the default display positions of each operation element are set to be arranged in order from the left, A1, A2, and A3, as in the example shown in FIG. 2A. Here, for example, if the user moves the operation element A2 to the right of A3, the operation screen of the display unit 21 is customized from the default state, as shown in FIG. 2C. Furthermore, if a new function is added to the image processing device 2, an operation element related to the new function is added in addition to A1 to A3, and the display position of each operation element on the operation screen is customized accordingly.

[0018] The terminal device 3 is a device different from the image processing device 2, such as a PC (Personal Computer), a mobile terminal, or a tablet terminal, and has a display unit 31, a control unit 32, a communication unit 33, and a storage unit 34. The terminal device 3 can be connected to the image processing device 2 via a network through the communication unit 33. For example, an application of the image processing device 2 is installed in the terminal device 3, and the image processing device 2 and the terminal device 3 can communicate with each other via a network. When the terminal device 3 is connected to the image processing device 2, as shown in FIG. 3A and FIG. 3B, the operation screen of the display unit 21 of the image processing device 2 is displayed on the display unit 31 of the terminal device 3. This makes it possible to remotely operate the image processing device 2 on the terminal device 3. FIG. 3A shows an example in which an operation screen corresponding to FIG. 2A is displayed, and FIG. 3B shows an example in which an operation screen corresponding to FIG. 2B is displayed.

[0019] Moreover, the control unit 32 of the terminal device 3 includes a CPU, a ROM, a RAM, and an input / output circuit (not shown), and automatically executes the operation process of the image processing device 2. Specifically, an application related to, for example, an RPA tool is installed in the terminal device 3. Therefore, the terminal device 3 can automatically execute the operation process of the image processing device 2 using the RPA tool.

[0020] More specifically, the control unit 32 generates an operation script for executing an operation process based on a user's operation, and stores the operation script in the storage unit 34. Then, the control unit 32 executes the operation process of the image processing device 2 based on the generated operation script.

[0021] Next, the flow of processing by the terminal device 3 will be described. First, the flow of generating an operation script will be described. Fig. 4 is a flowchart showing the flow of generating an operation script by the terminal device 3. Note that the processing shown in Fig. 4 may be performed on the premise that the terminal device 3 and the image processing device 2 are communicatively connected and the generation of an operation script based on RPA is started in the terminal device 3.

[0022] The operation script is acquired by the terminal device 3 to automatically execute the operation process. The operation script may store information on the operation process that links the operation element that is the target of the operation process to the unique information of the image processing device 2 in which the operation process is set. The unique information of the image processing device 2 may be information on the specifications of the image processing device 2 (presence or absence of color / black and white printing, printable paper sizes, presence or absence of a paper feed unit, presence or absence of a paper discharge unit, etc.).

[0023] 4, when the generation of an action script based on RPA is started in the terminal device 3, the control unit 32 activates the RPA creation function in the terminal device 3 and acquires predetermined information from the image processing device 2 (step S101). The predetermined information may be model information of the image processing device 2, information on the screen size of the operation screen of the image processing device 2, etc. As a result, the operation screen of the image processing device 2 is displayed on the display unit 31 of the terminal device 3.

[0024] Next, the control unit 32 acquires operation information by the user (step S102). The operation information by the user may be, for example, information on an operation performed by the user on an operation screen displayed on the display unit 31 of the terminal device 3.

[0025] For example, when a user operates an operation element indicating "scan" on the operation screen shown in Fig. 3A, information on the display position of the operation element (coordinate information, etc.) may be acquired by the terminal device 3. Also, on the operation screen shown in Fig. 3B, input information on detailed settings, information on the display position corresponding to the input information, etc. may be acquired by the terminal device 3.

[0026] After step S102, the control unit 32 determines whether or not the operation screen displayed on the display unit 31 is customizable (step S103). "Customizable" may mean, for example, that the display positions of operation elements are changeable, like the operation screen shown in Fig. 2A. "Not customizable" may mean, for example, that the display positions of operation elements are not changeable, like the operation screen shown in Fig. 2B (such as a detailed setting screen).

[0027] As a result of the determination, if the operation screen is customizable (step S103, YES), the control unit 32 stores information on the operation element at the display position specified by the user (operation information by the user acquired in step S102) and information on the operation process in the operation script (step S104). More specifically, the control unit 32 identifies information on the operation element based on the position specified by the user in the operation screen of the image processing device 2, and stores information on the identified operation element together with information on the operation process in the operation script.

[0028] The information on the operation element stored in the operation script may be information on the operation function related to the operation element, such as "copy" or "scan."

[0029] On the other hand, if the operation screen is not customizable (step S103, NO), the control unit 32 stores information on the operation process in the operation script (step S105).

[0030] The information on the operation process in steps S104 and S105 may include, for example, information indicating that an operation element has been operated by the user, and input information when there is input information by the user. The input information by the user may include, for example, input information on detailed settings in FIG. 2B and information related to the input information. Furthermore, the information indicating that an operation element has been operated may be position information of an operation element whose display position cannot be changed (for example, "Start" in FIG. 2B), information on the operation function, and the like.

[0031] Furthermore, the control unit 32 may determine whether or not the screen is a customized screen depending on the type of the operation screen. For example, if the operation screen is one of the operation screens shown in Fig. 2A and Fig. 2C, the control unit 32 may determine that the operation screen is a customized screen, and if the operation screen is one of the operation screens shown in Fig. 2B, the control unit 32 may determine that the operation screen is not a customized screen.

[0032] After step S104 or step S105, the control unit 32 determines whether or not the creation of the RPA is complete (step S106). If the determination result indicates that the creation of the RPA is not complete (step S106, NO), the process returns to step S102. On the other hand, if the creation of the RPA is complete (step S106, YES), the control unit 32 stores the created operation script in the storage unit (step S107). After step S107, this control ends.

[0033] The determination of whether the creation of the RPA is completed may be performed, for example, based on whether the terminal device 3 acquires information on an operation to complete the RPA creation by a user. The operation to complete the RPA creation may be, for example, the user operating an operation element for completing the RPA creation, or the user operating an operation element for storing the operation script in a storage unit. In addition, if the user forcibly terminates the creation of the RPA, the process of step S107 may not be performed.

[0034] In the created operation script, when the operation element is "copy," information on "copy" is stored as the operation element. In addition, in the operation script, information unique to the image processing device 2 is linked to the operation element. Furthermore, in the operation script, information on the above detailed settings (color or black and white, paper size, and information on the number of copies) is stored as information on the operation process. Furthermore, information on the date and time when the operation script was generated may be stored in the operation script.

[0035] Moreover, an operation script can be generated for each operation element. That is, the storage unit 34 may store a plurality of operation scripts corresponding to a plurality of operation functions. Specifically, an operation script for "copy" and an operation script for "scan" may be stored separately in the storage unit 34. Furthermore, a plurality of operation scripts for "copy" or "scan" with different settings such as the number of copies may be stored separately in the storage unit 34.

[0036] Next, a description will be given of the flow of execution of the operation process of the image processing device 2. Fig. 5 is a flowchart showing the flow of execution of the operation process of the image processing device 2 by the terminal device 3. Note that the process shown in Fig. 5 may be performed on the premise that the terminal device 3 and the image processing device 2 are connected so as to be able to communicate with each other.

[0037] 5, when the terminal device 3 and the image processing device 2 are communicatively connected, the control unit 32 acquires predetermined information from the image processing device 2 (step S201). The predetermined information may be the same as the information acquired in the process of step S101 in FIG.

[0038] Next, the control unit 32 acquires the operation script stored in the storage unit (step S202). In addition, when there are multiple operation scripts, the acquired operation script may be the operation script selected by the user.

[0039] After acquiring the operation script, the control unit 32 judges whether or not the user has performed an operation for executing the operation process by the RPA (step S203). For example, the operation for executing the operation process by the RPA may be the user operating an execution button for the operation process displayed on the display unit 31 (not shown in FIG. 3A and FIG. 3B, etc.).

[0040] As a result of the determination, if an operation for executing the operation process is not performed (step S203, NO), the process of step S203 is repeated. On the other hand, if an operation for executing the operation process is performed (step S203, YES), the control unit 32 determines whether or not the operation screen of the image processing device 2 is customizable (step S204).

[0041] As a result of the determination, if the operation screen is customizable (step S204, YES), the control unit 32 identifies an operation element based on the action script (step S205), and executes an operation process related to the operation element (step S206).

[0042] On the other hand, if the operation screen is not customizable (step S204, NO), the control unit 32 executes the operation process based on the operation script (step S207).

[0043] After step S206 or step S207, the control unit 32 transitions the display on the display unit 31 to an operation screen related to the next operation, and determines whether or not the operation process by the action script has ended (step S208). If the determination result shows that the operation process has not ended (step S208, NO), the process transitions to step S204. On the other hand, if the operation process has ended (step S208, YES), this control ends.

[0044] The transition to the operation screen related to the next operation may be, for example, a transition to a detailed setting screen in FIG. 3B after the "scan" operation is performed in FIG. 3A. The transition to the operation screen related to the next operation may be a transition from the detailed setting screen in FIG. 3B to a screen indicating that "scan" has started and scanning is in progress. The end of the operation process by the operation script may be, for example, the end of the operation stored in the operation script. In other words, the end of the operation process by the operation script may be the completion of the operation of the image processing device 2 related to the operation element.

[0045] According to the present embodiment as described above, an operation process relating to an operation element is executed based on an operation script that stores an operation element based on a position designated by a user on the operation screen of the image processing device 2.

[0046] For example, assume that the operation process is set on the operation screen shown in Fig. 3A. In this case, when the user performs an operation related to A2 (scan), the operation process related to A2 is stored in the operation script.

[0047] Here, for example, it is assumed that the operation screen of the image processing device 2 is customized like the operation screen shown in Fig. 2C. In this case, when the image processing device 2 is remotely operated by the terminal device 3, the customized operation screen is displayed on the display unit 31 as shown in Fig. 6.

[0048] Conventionally, an operation process was executed based on the position of an operation element, so when an operation process of an operation element of A2 is executed on a customized operation screen, the operation process is executed based on the position of A2 before customization (the dotted line part in FIG. 6) However, since the operation element of A2 does not exist on the customized operation screen, the operation process is not executed.

[0049] On the other hand, in this embodiment, an operation process related to an operation element is reliably executed based on the operation script. More specifically, in this embodiment, an operation element stored in the operation script is identified from a plurality of operation elements in the operation screen of the image processing device 2 based on the operation script. As a result, even if an operation element related to the operation process is moved to another display position, the operation process related to the operation element stored in the operation script can be reliably executed.

[0050] That is, in this embodiment, the image processing device 2 can be automatically operated reliably.

[0051] Furthermore, since the storage unit 34 stores a plurality of operation scripts corresponding to each of a plurality of operation functions, an operation process is executed based on an operation script selected from the plurality of operation scripts. Therefore, even if there are a plurality of types of operation functions that are frequently used, it is possible to perform automatic operation for each of the plurality of operation functions by acquiring an operation script from the storage unit 34 when executing each operation function. Furthermore, the selection of the operation script may be performed based on a user's operation, or may be performed automatically by the terminal device 3.

[0052] In the above embodiment, the terminal device 3 is connected to one image processing device 2, but the present invention is not limited to this, and the terminal device 3 may be connectable to multiple image processing devices 2, as shown in FIG. 7.

[0053] In this case, the operation script may be shared among a plurality of image processing devices 2. In other words, the operation script generated via the operation screen of a specific image processing device 2 may be used when executing an operation process of an image processing device 2 other than the specific image processing device 2.

[0054] For example, suppose that an operation script is generated via the image processing device 2 (second image processing device) shown at the top of Fig. 7. Here, when an automatic operation is executed in the image processing device 2 (first image processing device) shown in the middle or bottom of Fig. 7, the control unit 32 acquires the operation script related to the second image processing device. Then, when an operation element stored in the acquired operation script exists in the operation screen of the first image processing device, the control unit 32 executes an automatic process related to the operation element.

[0055] In this way, operation scripts for operation elements common to a plurality of image processing devices 2 can be reused, so that the effort of generating operation scripts for each image processing device 2 can be eliminated.

[0056] In addition, in the above embodiment, the operation script is stored in the memory unit 34 of the terminal device 3, but the present invention is not limited to this, and the operation script may be stored in the memory unit 24 of the image processing device 2.

[0057] In this case, the operation script can be made dedicated to the image processing device 2 in which the operation script is stored. As a result, the operation script can incorporate settings specific to the image processing device 2. Furthermore, if the operation script is stored in the terminal device 3, it becomes unnecessary to obtain the operation script from the image processing device 2 via the network when operating an operation function common to a plurality of image processing devices 2. As a result, the operation process can be executed quickly.

[0058] Furthermore, in the above embodiment, it is determined whether or not the operation screen is customizable, but the present invention is not limited to this, and the operation screen does not have to be customizable.

[0059] For example, in the case where the operation screen cannot be customized and the operation script is shared among a plurality of image processing devices, automatic operation can be reliably performed in each image processing device by identifying the operation elements using the operation script.

[0060] In addition, the above-mentioned embodiments are merely examples of the embodiment of the present invention, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be embodied in various forms without departing from the gist or main characteristics of the present invention. [Explanation of symbols]

[0061] 1 System 2. Image Processing Device 3 Terminal Equipment 21 Display section 22 Control section 23 Communications Department 24 Memory section 31 Display section 32 Control section 33 Communications Department 34 Storage section

Claims

1. 1. A method for executing an operation process of an image processing device on a terminal device, comprising: acquiring, by the terminal device, an operation script that stores an operation element based on a position designated by a user on an operation screen of the image processing device; executing an operation process for the operational element based on the operation script; The method of execution has the following steps.

2. The method further includes a step of determining whether or not the display positions of one or more operation elements on the operation screen of the image processing device are customizable; and executing the operation process when it is determined that the display positions of the one or more operation elements on the operation screen are customizable.

2. The method of claim 1.

3. In the step of executing the operation process, an operation element stored in the operation script is identified from one or more operation elements in an operation screen of the image processing device.

2. The method of claim 1.

4. The operation script is stored in a storage unit of the terminal device.

2. The method of claim 1.

5. The operation script is stored in a storage unit of the image processing device.

2. The method of claim 1.

6. In the step of acquiring the operation script by the terminal device, when executing an operation process of a first image processing device, an operation script related to a second image processing device different from the first image processing device is acquired; In the step of executing the operation process, when the operation element is present on an operation screen of the first image processing device, an operation process related to the operation element is executed.

2. The method of claim 1.

7. The operation script stores information about an operation process that associates the operation element with unique information of an image processing device in which the operation process is set.

2. The method of claim 1.

8. the storage unit stores a plurality of operation scripts corresponding to a plurality of operation functions, the step of executing the operation process includes executing the operation process based on an operation script selected from the plurality of operation scripts. A method according to claim 4 or claim 5.

Images