Control program and control method of portable terminal

Abstract

To improve operability of a portable terminal which can remotely operate an information processing device.SOLUTION: An information processing system (10) includes an image forming device (12) which is one example of an information processing device and a user terminal (14). The image forming device and the user terminal are directly and communicably connected by wireless communication etc. A first operation screen (260) which has an image of a function selection key for selecting a function which remotely operates the information processing device to be operated, and the image of first start keys (184, 186) for making the information processing device start predetermined operation is displayed on a display (154) of the user terminal. Operation instructions according to input operation to the first operation screen are transmitted to the information processing device. Then, according to predetermined user operation, a second operation screen (262) which does not have the image of the function selection key, and has the image of a second start key (266) for making the information processing device start the predetermined operation is displayed on the display.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to a control program and a control method for a mobile terminal, and more particularly to a control program and a control method for a mobile terminal that can remotely control an information processing device, for example. [Background technology]

[0002] An example of this type of background art is disclosed in Patent Document 1. When a portable terminal for remotely controlling an image forming apparatus of the background art is used to remotely control the image forming apparatus, it executes an operation screen program for a standard model that is different from the model of the image forming apparatus, displays the operation screen of the standard model, generates commands in response to user input operations on this operation screen, and transmits the commands to the image forming apparatus. However, on the operation screen of the standard model displayed on the portable terminal, among the operation keys included in the operation screen of the standard model, keys that do not correspond to any of the operation keys included in the operation screen of the image forming apparatus to be operated are disabled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-228872 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the portable terminal for remotely controlling the image forming apparatus of the background art, among the operation keys included in the operation screen of the image forming apparatus to be operated, keys that do not correspond to any of the operation keys included in the operation screen of the standard model displayed on the portable terminal, that is, operation keys that are executable on the image forming apparatus but cannot be executed on the operation screen for the portable terminal, are not displayed. In other words, the portable terminal for remotely controlling the image forming apparatus of the background art has a problem that some of the operation keys that can be executed on the image forming apparatus cannot be used, and there is room for improvement in operability.

[0005] Therefore, a primary object of the present invention is to provide a novel control program and control method for a mobile terminal.

[0006] Another object of the present invention is to provide a control program and a control method for a mobile terminal that can improve operability. [Means for solving the problem]

[0007] A first example embodiment is a control program executed by a control unit of a portable terminal having a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control program causing the control unit to display on the display a first remote control screen having an image indicating a function selection key for selecting a function of the information processing device to be operated, and to display at one longitudinal end of the display a first operation area having an image indicating a start key for causing the information processing device to start a specified operation in response to a specified user operation.

[0008] In the first embodiment, the mobile terminal (14: reference numerals are used to illustrate corresponding parts in the embodiments but are not intended to be limiting; the same applies below) includes a display (154), a touch panel (152) provided on the display, and a communication unit (150) that communicates with the information processing device (12), and the control unit (142, 148) causes the display (154) to display a first remote operation screen (260) having images (242, 248) indicating function selection keys for selecting functions of the information processing device (12) to be operated in accordance with a control program. The control unit (142, 148) further causes a first operation area (262) having an image (266) indicating a start key for starting a predetermined operation on the information processing device (12) in response to a predetermined user operation, at one end in the longitudinal direction of the display (154).

[0009] According to the first embodiment, it is possible to cause the information processing device to start a predetermined operation by operating the image representing the start key displayed in the first operation area.

[0010] The second embodiment is a control program subordinate to the first embodiment, and when the control unit is caused to display a first remote operation screen, the control unit is caused to display the first remote operation screen having an image showing function selection keys for setting any one of functions related to a copy job, including color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page combining.

[0011] The third embodiment is a control program subordinate to the first or second embodiment, and causes the control unit to end the remote control function when the end key is operated.

[0012] A fourth embodiment is a control program subordinate to any of the first to third embodiments, and when there is an unavailable function that cannot be executed by an input device of an information processing device, the control unit displays a first remote operation screen that does not include an image showing a function selection key corresponding to the unavailable function.

[0013] The fifth embodiment is a control program subordinate to the fourth embodiment, which causes the control unit to display a first operation screen and a second operation screen having any one of an operation screen corresponding to a software key type input device, an operation screen corresponding to a hard key type input device, and an operation screen corresponding to a combination type input device when causing the control unit to display a first operation screen and when causing the control unit to display a first operation area.

[0014] A sixth embodiment is a control method for a portable terminal having a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control method including: displaying on the display a first remote control screen having an image indicating a function selection key for selecting a function of the information processing device to be operated; and displaying at one longitudinal end of the display a first operation area having an image indicating a start key for causing the information processing device to start a specified operation in response to a specified user operation.

[0015] The sixth embodiment is expected to provide the same effects as the first embodiment. Effect of the Invention

[0016] According to the present invention, a plurality of information processing devices having different functions and operation methods can be remotely operated from a mobile terminal, thereby improving operability.

[0017] The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of an information processing system according to the present invention. [Diagram 2] FIG. 2 is a block diagram showing the electrical configuration of the image forming apparatus shown in FIG. [Diagram 3] FIG. 3 is a block diagram showing an electrical configuration of the user terminal shown in FIG. [Figure 4] FIG. 4 is an illustrative view showing an example of a home screen of the image forming apparatus. [Diagram 5] FIG. 5 is an illustrative view showing an example of a copy setting screen of the image forming apparatus. [Figure 6] FIG. 6 is an illustrative view showing one example of a selection screen of a user terminal. [Figure 7] FIG. 7 is an illustrative view showing one example of a first remote operation screen of the user terminal. [Figure 8] FIG. 8 is an illustrative view showing an example of a second remote operation screen of the user terminal. [Figure 9] FIG. 9 is an illustrative view showing another example of the second remote operation screen of the user terminal. [Figure 10] FIG. 10 is an illustrative view showing another example of the second remote operation screen of the user terminal. [Figure 11] FIG. 11 is an illustrative view showing an example of a memory map of the RAM of the image forming apparatus shown in FIG. [Figure 12]FIG. 12 is an illustrative view showing an example of a memory map of a RAM of the user terminal shown in FIG. [Figure 13] FIG. 13 is a flow chart showing a part of an example of a remote control process of the CPU of the user terminal. [Figure 14] FIG. 14 is a flow chart showing part of the remote control processing and following FIG. [Figure 15] FIG. 15 is an illustrative view showing an example of the second remote operation screen of the second embodiment. [Figure 16] FIG. 16 is an illustrative view showing another example of the second remote operation screen according to the second embodiment. [Figure 17] FIG. 17 is a flowchart showing a part of an example of the remote control process according to the second embodiment. [Figure 18] FIG. 18 is a flowchart showing a part of an example of the remote control process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] [First Example] Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present invention. Referring to Fig. 1, information processing system 10 according to a first embodiment of the present invention includes an image forming device 12 and a user terminal (mobile terminal) 14. Although details will be described later, image forming device 12 and user terminal 14 are connected to each other so as to be able to directly communicate with each other via wireless communication or the like.

[0020] Although FIG. 1 shows one image forming apparatus 12 and one user terminal 14, there may be two or more image forming apparatuses 12 and user terminals 14.

[0021] Image forming device 12 is an example of an information processing device that is a communication target of user terminal 14, and is a multifunction peripheral (MFP) having a copying function, a printer function, a scanner function, a facsimile (FAX) function, etc. Note that the present invention is applicable not only to multifunction peripherals, but also to other information processing devices such as copying machines (copy machines), printing devices (printers), facsimiles, and image reading devices (scanners).

[0022] Fig. 2 is a block diagram showing an electrical configuration of the image forming apparatus 12 shown in Fig. 1. Referring to Fig. 2, the image forming apparatus 12 includes a CPU 122. To the CPU 122, a RAM 124, an input control circuit 126, a display control circuit 128, an image forming unit 134, an image reading unit 136, a communication unit 138, and a storage unit 118 are connected via a bus 120. In addition, an input device 130 is connected to the input control circuit 126, and a display 132 is connected to the display control circuit 128.

[0023] The CPU 122 is responsible for the overall control of the image forming apparatus 12. The RAM 124 is used as a work area and a buffer area for the CPU 122.

[0024] The storage unit 118 is a main storage device of the image forming apparatus 12, and is implemented by a non-volatile memory such as an HDD and an EPPROM. The storage unit 118 stores data on a control program for the CPU 122 to control the operation of each part of the image forming apparatus 12, data on images (display images) for various screens, data required for executing the control program, and the like. The storage unit 118 may be configured to include a RAM 124. The same applies to the storage unit 156 of the user terminal 14, which will be described later.

[0025] The input device 130 is for accepting input operations to the image forming apparatus 12, and includes, for example, a touch panel. The touch panel is a general-purpose touch panel, and any type of touch panel, such as a capacitive type, an electromagnetic induction type, a resistive film type, or an infrared type, can be used. The touch panel may be provided on the display surface of the display 132, or may be provided separately from the display 132. However, a touch display in which the touch panel is integrally formed with the display 132 may be provided. When the touch panel is provided on the display surface of the display 132, a GUI (Graphical User Interface) including software keys (operation keys) such as various buttons and icons is displayed on the display 132. In this case, the input operation is accepted via the GUI (operation screen). The input device 130 may include hardware operation buttons (hardware keys) instead of or in addition to the touch panel.

[0026] It should be noted that the software key refers to a key or icon reproduced by software on the display surface of the display 132. In contrast, the hardware key refers to a key or a push button provided as a physical device.

[0027] The input control circuit 126 outputs an operation signal or operation data corresponding to an input operation to the input device 130 to the CPU 122. Furthermore, when the input device 130 includes a touch panel, the input control circuit 126 applies a necessary voltage to the touch panel, detects a touch operation (touch input) within the effective touch range of the touch panel, and outputs touch coordinate data indicating the position of the touch input to the CPU 122. Furthermore, when the input device 130 includes a hardware key, the input control circuit 126 outputs an operation signal or operation data corresponding to the operation of the hardware key to the CPU 122.

[0028] Display control circuit 128 includes a GPU and a VRAM, and under the instruction of CPU 122, the GPU generates display image data in the VRAM for displaying various operation screens on display 132 using image generation data 304b (see FIG. 11) stored in RAM 124, and outputs the generated display image data to display 132. On display 132, an operation screen such as a home screen 160 (see FIG. 4) that is a screen for selecting a desired job from various jobs that image forming apparatus 12 can execute is displayed. However, in this first embodiment, the job means scanning, copying (including scanning of an original), faxing, etc. As display 132, an LCD (Liquid Crystal Display) or an EL (Electro-Luminescence) display, etc. can be used.

[0029] Image forming unit (printing device) 134 is a general-purpose laser printer, and includes a photoconductor, a charger, an exposure device, a developing device, a transfer device, a fixing device, etc., and prints an image corresponding to the print data processed by the image processing circuit onto recording paper (paper), etc. However, image forming unit 134 may be configured as an inkjet printer instead of a laser printer.

[0030] Although detailed explanation is omitted, the image forming unit 134 has a color printing function, and for each color of Y (yellow), M (magenta), C (cyan), and K (black), an image forming station including a photoconductor, a charger, a developing device, etc. is configured.

[0031] Image reading unit 136 includes a light source, multiple mirrors, an imaging lens, a line sensor, etc. This image reading unit 136 exposes the surface of the document to the light source, and guides the reflected light from the document surface to the imaging lens via multiple mirrors. The imaging lens then forms an image of the reflected light on the light receiving element of the line sensor. The line sensor detects the luminance and chromaticity of the reflected light that forms an image on the light receiving element, and generates read image data based on the image of the document surface.

[0032] The communication unit 138 includes a wired communication circuit or a wireless communication circuit. The wired communication circuit is a circuit for performing wired communication conforming to a known communication standard such as Ethernet (registered trademark), and performs a process of transmitting and receiving communication data with an external computer via a network such as the Internet. The wireless communication circuit is a circuit for performing wireless communication such as WiFi, and an antenna is connected. This wireless communication circuit performs a process of transmitting and receiving communication data transmitted and received with an external computer using wireless communication such as WiFi based on a wireless communication method conforming to a known communication standard such as IEEE 802.11. In addition, the communication unit 138 performs a process of transmitting and receiving communication data transmitted and received with the user terminal 14 using wireless communication. Note that the image forming apparatus 12 may be connected to a network such as the Internet. In this case, the communication unit 138 performs a process of transmitting and receiving communication data transmitted and received with an external computer via the network.

[0033] 2 is merely an example, and is not limited to this. For example, a hardware keyboard may be connected to the image forming apparatus 12 as another input means.

[0034] The user terminal 14 is, for example, a portable terminal (mobile terminal) that can be carried around, such as a general-purpose smartphone, feature phone, or tablet PC, and has various functions, such as a calling function, an email function, and a browser function.

[0035] Fig. 3 is a block diagram showing the electrical configuration of the user terminal 14 shown in Fig. 1. Referring to Fig. 3, the user terminal 14 of the first embodiment includes a CPU 142. The CPU 142 is connected to a RAM 144, a touch panel control circuit 146, a display control circuit 148, a wireless communication unit 150, and a storage unit 156 via a bus 140. A touch panel 152 is connected to the touch panel control circuit 146, and a display 154 is connected to the display control circuit 148.

[0036] The CPU 142 is responsible for overall control of the user terminal 14. The RAM 144 is used as a work area and a buffer area for the CPU 142.

[0037] Touch panel control circuit 146 applies the necessary voltage and the like to touch panel 152, and when a touch input is detected, outputs to CPU 142 touch coordinate data indicating the position of the touch input.

[0038] The display control circuit 148 includes a GPU, a VRAM, etc., and under the direction of the CPU 142, the GPU generates display image data in the VRAM for displaying various screens on the display 154 using image generation data 324b (see FIG. 12) stored in the RAM 144, and outputs the generated display image data to the display 154.

[0039] The wireless communication unit 150 includes a wireless communication circuit and the like, and performs transmission and reception processing of communication data transmitted and received between the image forming apparatus 12 and the image forming apparatus 12 based on a wireless communication method conforming to a known communication standard.

[0040] The electrical configuration of the user terminal 14 shown in Fig. 3 is merely an example, and is not limited thereto. For example, the user terminal 14 may be provided with a hardware operation button as another input means. For simplicity, components related to calls, etc. are omitted in Fig. 3.

[0041] An example of a basic operation of the information processing system 10 will be described below with reference to Fig. 4 and Fig. 5. When the main power supply of the image forming apparatus 12 is turned on and the image forming apparatus 12 is in a standby state in which each function can be executed, a home screen 160 as shown in Fig. 4 is displayed on the display 132 of the image forming apparatus 12. As described above, the home screen 160 is a screen for selecting various jobs, and is an example of an operation screen. Images of a plurality of operation keys (selection keys) 162 for selecting each job such as copy, fax, and scan are provided (displayed) on this home screen 160.

[0042] When a selection key 162 is touched (operated) on the home screen 160, a job assigned to the operated selection key 162 is selected, and a setting screen for configuring the job is displayed on the display 132. Below, an example in which a copy job is selected will be described.

[0043] When the selection key 162 to which a copy job is assigned on the home screen 160 is operated (a copy job is selected), a copy setting screen 180 as shown in Fig. 5 is displayed on the display 132. This copy setting screen 180 is a screen for setting operating conditions for the copy job, and is an example of an operation screen.

[0044] On the copy setting screen 180, operation keys such as a function selection key 182, a black and white start key 184, a color start key 186, and a numeric keypad 188 are provided.

[0045] Function selection key 182 is located at the left end of copy setting screen 180. A plurality of function selection keys 182 are provided and arranged in a vertical line. Each of the plurality of function selection keys 182 corresponds to an operating condition of a copy job or a function related to a copy job. Specifically, each of function selection keys 182 is assigned a function for making detailed settings of each of the operating conditions of color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page aggregation.

[0046] A black and white start key (hereinafter referred to as the "black and white start key") 184 and a color start key (hereinafter referred to as the "color start key") 186 are displayed in the lower right corner area of ​​the copy setting screen 180. The black and white start key 184 and the color start key 186 are keys for starting a predetermined operation in the image forming apparatus 12. When the black and white start key 184 is operated, black and white copying is started. Also, when the color start key 186 is operated, color copying is started according to the color setting.

[0047] Note that the process of generating black-and-white or color output image data reflecting various settings from the read image data, and the process of forming black-and-white or color images based on the output image data are already well known, and are different from the essential content of the present invention, so detailed explanations of these will be omitted.

[0048] The numeric keypad 188 is a set of numeric keys, and is provided in the center of the copy setting screen 180. When the numeric keypad 188 is operated, the operated number is displayed in a numeric keypad input display field above the numeric keypad 188. That is, the number of copies is set.

[0049] As described above, the example has been described where a copy job is selected on home screen 160. However, when a FAX job or a scan job is selected, a setting screen required for setting the operating conditions of the job is displayed on display 132, and the operating conditions of the job can be set or the job can be executed.

[0050] In a conventional information processing system, an operation screen for operating the image forming apparatus is displayed on a user terminal that can communicate with the image forming apparatus, and the image forming apparatus can be remotely operated from the user terminal. In addition, there is a conventional technology in which, among operation keys included in the operation screen of the user terminal, keys that do not correspond to any of the operation keys included in the operation screen of the image forming apparatus to be operated are disabled.

[0051] However, in the above-mentioned conventional technology, among the operation keys included in the operation screen of the image forming device to be operated, keys that do not correspond to any of the operation keys included in the operation screen of the standard model displayed on the user terminal, that is, operation keys that are executable by the image forming device but cannot be executed on the operation screen for the user terminal, are not displayed. In other words, in the above-mentioned conventional technology, there is a problem that some of the operation keys that can be executed by the image forming device cannot be used, and there is room for improvement in operability.

[0052] Therefore, in the information processing system 10 of the first embodiment, it is possible to operate operation keys that are operation functions (operation keys) that can be executed on the image forming apparatus 12 but cannot be executed on the operation screen for the user terminal.

[0053] An example of the operation of the information processing system 10 will be described below with reference to Figs. 6 to 10. First, in the information processing system 10 of this embodiment, an application program (hereinafter referred to as a "remote operation application") for remotely operating the image forming apparatus 12 by communicating with the image forming apparatus 12 is stored in the user terminal 14. In brief, by executing this remote operation application, a remote operation screen for the user terminal for remotely operating the image forming apparatus 12 (hereinafter sometimes simply referred to as a "remote operation screen") is displayed on the display 154 of the user terminal 14. Then, an operation instruction (remote operation instruction) corresponding to a touch operation (user operation) on the remote operation screen is input from the user terminal 14 to the image forming apparatus 12. When the image forming apparatus 12 accepts the remote operation instruction, it operates according to the operation instruction.

[0054] However, in the remote control application of this embodiment, multiple types of remote control screens can be used. For example, in the remote control application of this embodiment, multiple types of remote control screens that imitate input devices of various models of image forming apparatuses that actually exist can be used. The input devices of the image forming apparatus include an input device of a type that accepts user operations via software keys displayed on a touch panel display (a software key type input device), an input device of a type that accepts user operations via hardware keys or a hardware keyboard (a hard key type input device), and an input device of a type that uses both software keys and hard keys (a combined type input device).

[0055] Further, the image forming apparatus 12 includes a business image forming apparatus that is mainly used in an office, etc., and a public image forming apparatus that is mainly used in a public facility, etc. Since the business image forming apparatus and the public image forming apparatus are intended for different user groups, the specifications of the input device are significantly different between the business image forming apparatus and the public image forming apparatus.

[0056] In the remote operation application, a remote operation screen including an operation area simulating a software key type input device, a remote operation screen including an operation area simulating a hard key type input device, and a remote operation screen including an operation area simulating a combination type input device are available. In addition, each of the remote operation screens corresponding to each type of input device includes a remote operation screen corresponding to an input device of a business image forming device and a remote operation screen corresponding to an input device of an image forming device for public use. Furthermore, a remote operation screen including an operation area dedicated to remote operation, apart from an actual input device, is also available.

[0057] When the remote control application is executed (launched), a selection screen 200 is displayed on the display 154 of the user terminal 14, as shown in Fig. 6. The selection screen 200 displays operation keys (selection keys) 202 for selecting a remote control screen to be actually used from among a plurality of types of remote control screens available in the remote control application. Although not shown, the selection keys 202 may display characters, symbols, figures, etc. indicating information related to the assigned remote control screen. In addition, when the remote control screen assigned to the selection keys 202 corresponds to a specific model of input device, characters, symbols, figures, etc. indicating information related to the model may be displayed.

[0058] When the selection key 202 is operated on the selection screen 200, a remote operation screen assigned to the operated selection key 202 is selected. When the remote operation screen is selected, information on operation functions executable on the operated remote operation screen (remote operation screen specification information) is acquired. Also, when the remote operation screen is selected, the user terminal 14 communicates with the image forming apparatus 12 to acquire information on operation functions executable on the input device 130 of the image forming apparatus 12 to be operated (apparatus side specification information). However, since the user operation on the selection screen 200 is not an operation for remotely operating the image forming apparatus 12 (it is an operation for internal processing of the user terminal 14), a remote operation instruction corresponding to the user operation on the selection screen 200 is not input from the user terminal 14 to the image forming apparatus 12.

[0059] Furthermore, when a remote operation screen is selected, a remote operation screen of the operated type is displayed on the display 154. For example, as shown in Fig. 7, a first remote operation screen 220 is displayed on the display 154. The first remote operation screen 220 is a screen for selecting a desired job from various jobs that the image forming apparatus 12 can execute, and corresponds to the home screen 160 described above.

[0060] Images of operation keys (selection keys) 222 for selecting each job such as copy, fax, scan, and network print are displayed on the first remote operation screen 220. The selection keys 222 correspond to the selection key 162 on the home screen 160.

[0061] However, if there is an operation function (unusable function) that is executable on the remote operation screen but cannot be executed on the input device of the image forming apparatus to be operated, that is, if there is an operation key (unusable key) that is executable on the remote operation screen but cannot be executed on the input device of the image forming apparatus to be operated, the unusable key is disabled. For example, the image forming apparatus 12 to be operated in this embodiment cannot execute a network print job. Therefore, the input device 130 of the image forming apparatus 12 to be operated does not have a selection key for selecting a network print job. Therefore, on the first remote operation screen 220, the selection key 222a for selecting a network print job corresponds to the unusable key and is disabled.

[0062] Moreover, in order to indicate that the operation key cannot be used, the unavailable key is displayed semi-transparently and has a cross mark thereon. Note that instead of displaying the cross mark, the unavailable key may be displayed in a grayed-out state or in a low-luminance color.

[0063] Furthermore, if there is an unavailable key, a notification section 224 is displayed on the remote operation screen to notify the user that there is an unavailable key. The notification section 224 is displayed in a blank space on the remote operation screen, or is displayed overlapping (popping up) on top of the remote operation screen.

[0064] In the example shown in FIG. 7, a notification section 224 including a message to the effect that a network print job cannot be selected is displayed at the top of the first remote operation screen 220.

[0065] When the selection key 222 is operated on the first remote operation screen 220, a job assigned to the operated selection key 222 is selected, and a remote operation screen equivalent to a setting screen for setting the job is displayed on the display 154. However, when a user operation is accepted on an operation screen displayed after the first remote operation screen 220, a remote operation instruction corresponding to the user operation (corresponding to the function of the operated key) is input from the user terminal 14 to the image forming apparatus 12. The following describes an example in which a copy job is selected.

[0066] When the selection key 222 to which a copy job is assigned is operated (a copy job is selected) on the first remote operation screen 220, a second remote operation screen 240 as shown in Fig. 8 is displayed on the display 154. This second remote operation screen 240 is a screen for setting operating conditions for the copy job, and corresponds to the copy setting screen 180 described above.

[0067] The second remote operation screen 240 is provided with a function selection key 242, a black and white start key 244, a color start key 246, and a numeric keypad 248. However, the function selection key 242 corresponds to the function selection key 182, the black and white start key 244 corresponds to the black and white start key 184, the color start key 246 corresponds to the color start key 186, and the numeric keypad 248 corresponds to the numeric keypad 188. These operation keys are assigned the same functions as the corresponding operation keys on the copy setting screen 180, so detailed explanations will be omitted.

[0068] For example, when either the black and white start key 244 or the color start key 246 is operated, a remote operation instruction to start a job is input to the image forming apparatus 12, the remote operation application is terminated, and the remote operation screen is hidden. The remote operation application is also terminated if no operation is performed for a predetermined period of time while the remote operation screen is displayed. An end key for terminating the remote operation application may be provided on the remote operation screen. In this case, when the end key is operated, the remote operation application is terminated.

[0069] However, if there is an operation function (additional function) that can be executed on the input device of the image forming apparatus to be operated but cannot be executed on the initial remote operation screen, i.e., if there is an operation key (additional function key) that can be executed on the input device of the image forming apparatus to be operated but cannot be executed on the initial remote operation screen, the notification function for the additional function is enabled.

[0070] When the notification function of the additional function is enabled, an additional function notification section for notifying the user that the additional function exists is provided on the remote operation screen. For example, the image forming apparatus 12 to be operated in this embodiment can execute double-sided printing, post-processing (e.g., stapling, punching, folding, etc.), and special paper copying in a copy job. On the other hand, the second remote operation screen 240 does not have operation keys corresponding to double-sided printing, post-processing, and special paper copying. Therefore, in this embodiment, the double-sided printing function, post-processing function, and special paper copying function correspond to the additional functions, and the operation keys corresponding to double-sided printing, post-processing, and special paper copying correspond to the additional function keys. Therefore, the second remote operation screen 240 is provided with an additional function notification section 250 for notifying the user that the double-sided printing function, post-processing function, and special paper copying function exist.

[0071] The added function notification section 250 displays a message indicating that an added function exists. The added function notification section 250 also displays added function keys 250a to 250c for executing the added functions. That is, the added function notification section 250 is also an extended operation section including the added function keys 250a to 250c assigned to the added functions. For example, the added function notification section 250 is displayed in a blank portion of the remote operation screen (a portion where operation keys other than the added function keys are not arranged). In this embodiment, the added function notification section 250 is displayed at the upper end portion of the second remote operation screen 240.

[0072] Although not shown in the figure, when any of the additional function keys 250a to 250c is operated, an additional function setting screen for setting operating conditions related to the additional function corresponding to the additional function key is displayed. The additional function setting screen is displayed in another blank area of ​​the remote operation screen, or is displayed overlapping on the front of the remote operation screen. The same applies to the additional function keys 254a to 254c and 264a to 264c described later.

[0073] Note that the example shown in Fig. 8 is just one example, and the additional function notification section 252 may be displayed overlapping on the front of the remote operation screen as shown in Fig. 9. In this case, the additional function notification section 252 displays a message indicating that an additional function exists, a first key ("Yes" key) 252a for displaying additional function keys corresponding to the additional functions, and a second key ("No" key) 252b for not displaying the additional function keys corresponding to the additional functions.

[0074] When the first key 252a is operated, an extended operation section 254 including additional function keys 254a to 254c is displayed as shown in Fig. 10. The extended operation section 254 is displayed overlapping the front of the remote operation screen. On the other hand, when the second key 252b is operated, the additional function notification section 252 is hidden. Note that, in the state where the additional function notification section 252 is displayed, operations other than the operation of the first key 252a or the second key 252b are not possible, and other operations are not possible.

[0075] The above-mentioned operation of the information processing system 10 is realized by the CPU 122 of the image forming apparatus 12 executing an information processing program for the image forming apparatus 12 stored in the RAM 124, and the CPU 142 of the user terminal 14 executing an information processing program for the user terminal 14 stored in the RAM 144. Specific processing will be described later using a flow chart.

[0076] Fig. 11 is an illustrative view showing an example of a memory map 300 of the RAM 124 of the image forming apparatus 12 shown in Fig. 2. As shown in Fig. 11, the RAM 124 includes a program storage area 302 and a data storage area 304. As described above, the program storage area 302 of the RAM 124 stores an information processing program for the image forming apparatus 12. The information processing program for the image forming apparatus 12 includes a communication program 302a, an operation detection program 302b, a display program 302c, a transmission program 302d, and a reception program 302e.

[0077] The communication program 302a is a program for controlling the communication unit 138 to communicate (transmit / receive) data with an external computer via a network in a wired manner, and also for controlling the communication unit 138 to wirelessly transmit / receive data with the user terminal 14 and other external computers.

[0078] The operation detection program 302b is a program for detecting an input operation to the input device 130. For example, the operation detection program 302b is a program for acquiring touch coordinate data output from the input control circuit 126 when a touch panel is operated, and detecting that a software key displayed on the display 132 has been operated. The operation detection program 302b is also a program for acquiring operation data output from the input control circuit 90 when a hardware key is operated, and detecting that the hardware key has been operated.

[0079] Display program 302c is a program for generating display image data, that is, screen data such as home screen 160 and copy setting screen 180 described above, using image generation data 304b described below, and outputting the data to display 132.

[0080] The transmission program 302d is a program for transmitting the device side specification information data 304c to the user terminal 14. However, when transmitting, the communication program 302a is executed. The reception program 302e is a program for receiving the operation instruction data transmitted from the user terminal 14. However, when receiving, the communication program 302a is executed.

[0081] Although not shown, the program storage area 302 also stores an image reading program and an image forming program. The image reading program is a program for controlling the image reading unit 136 to read an image of a document and output an image signal (read image data) corresponding to the read image. The image forming program is a program for controlling the image forming unit 134 to print a multi-color or monochrome image on paper. The program storage area 302 also stores programs for selecting and executing various functions provided in the image forming device 12.

[0082] Furthermore, data storage area 304 of RAM 124 stores operation input data 304a, image generation data 304b, device side specification information data 304c, operation instruction data 304d, and the like.

[0083] The operation input data 304a is touch coordinate data and / or operation data detected according to the operation detection program 302b. The detected touch coordinate data and / or operation data are stored in the data storage area 304 in chronological order. The image generation data 304b is data such as image data of various operation keys for generating display image data corresponding to various screens to be displayed on the display 132. The device side specification information data 304c is data of information on operation functions executable by the input device 130 of the image forming device 12 (information on operation keys executable by the input device 130). The operation instruction data 304d is data corresponding to the operation instruction data 324e transmitted from the user terminal 14.

[0084] Although not shown in the figure, the data memory area 304 stores other data necessary for executing the information processing program for the image forming device 12, and is provided with timers (counters) and registers necessary for executing the information processing program for the image forming device 12.

[0085] Fig. 12 is an illustrative view showing an example of a memory map 320 of the RAM 144 of the user terminal 14 shown in Fig. 3. As shown in Fig. 12, the RAM 144 includes a program storage area 322 and a data storage area 324. As described above, the program storage area 322 of the RAM 144 stores an information processing program for the user terminal 14. The information processing program for the user terminal 14 includes a communication program 322a, an operation detection program 322b, a display program 322c, a receiving program 322d, a determination program 322e, a function display control program 322f, an operation instruction generating program 322g, and a transmission program 322h.

[0086] The communication program 322a is a program for controlling the communication unit 208 to directly transmit and receive data to and from the image forming apparatus 12 using wireless communication.

[0087] The operation detection program 322b is a program for detecting touch input, acquiring touch coordinate data output from the touch panel 152, and detecting that a software key included in various screens displayed on the display 154 has been operated.

[0088] Display program 322c is a program for generating display image data, that is, screen data such as the remote control screen described above, using image generation data 324b described below, and outputting it to display 154.

[0089] The reception program 322d is a program for receiving the apparatus side specification information data 304c transmitted from the image forming apparatus 12. However, during reception, the communication program 322a is executed.

[0090] The judgment program 322e is a program for comparing the device-side specification information indicated by the device-side specification information data 324c of the image forming device to be operated with the specification information of the remote operation screen indicated by the terminal-side specification information data 324d to judge whether or not there are any unavailable functions and unavailable keys, as well as additional functions and additional function keys.

[0091] The function display control program 322f is a program for disabling an operation key corresponding to an unavailable function (unavailable key) among the operation keys displayed on the remote operation screen when an unavailable function (unavailable key) exists. The function display control program 322f is also a program for displaying an additional function notification section including the additional function key on the remote operation screen when an additional function (additional function key) exists, thereby making the additional function key operable.

[0092] The operation instruction generating program 322g is a program for generating operation instruction data 324e regarding a remote operation instruction (remote operation command) corresponding to a touch operation on the remote operation screen.

[0093] The transmission program 322e is a program for transmitting the operation instruction data 324e to the image forming apparatus 12. However, during transmission, the communication program 322a is executed.

[0094] Although not shown, the program storage area 322 also stores programs for selecting and executing various functions provided in the user terminal 14.

[0095] Furthermore, data storage area 324 of RAM 144 stores operation input data 324a, image generation data 324b, device side specification information data 324c, terminal side specification information data 324d, operation instruction data 324e, and the like.

[0096] The operation input data 324a is touch coordinate data detected according to the operation detection program 302b. The detected touch coordinate data is stored in the data storage area 324 in chronological order. The image generation data 324b is data such as polygon data or texture data for generating display image data corresponding to various screens to be displayed on the display 132. The device side specification information data 324c is data corresponding to the device side specification information data 304c transmitted from the image forming device 12. The terminal side specification information data 324d is data regarding information on operation functions that can be executed from the beginning on the selected remote operation screen (information regarding operation keys that can be executed on the remote operation screen in the initial state). The operation instruction data 324e is data regarding operation instructions corresponding to a touch operation on the remote operation screen.

[0097] Although not shown in the figure, the data memory area 324 stores other data necessary for executing the information processing program for the user terminal 14, and is provided with timers (counters) and registers necessary for executing the information processing program for the user terminal 14.

[0098] Fig. 13 is a flow diagram showing an example of remote control processing executed by the CPU 142 of the user terminal 14 shown in Fig. 3. This remote control processing starts when a remote control application is executed (launched).

[0099] As shown in FIG. 13, when the CPU 142 of the user terminal 14 starts the remote control process, in step S1, it displays a selection screen 200 on the display 154 and determines whether or not a remote control screen to be used has been selected from among multiple types of remote control screens available in the remote control application.

[0100] If step S1 is "NO", that is, if it is determined that the remote operation screen to be used has not been selected, the process returns to the same step S1. On the other hand, if step S1 is "YES", that is, if it is determined that the remote operation screen to be used has been selected, then in step S3, information on the operation functions of the remote operation screen to be used, that is, the specification information on the terminal side, is obtained, and in step S5, information on the operation functions of the image forming apparatus 12 to be operated, that is, the specification information on the apparatus side, is obtained, and the process proceeds to step S7.

[0101] Next, in step S7, the specification information on the device side is compared with the specification information on the terminal side, and in step S9, it is determined whether or not an unavailable function (unavailable key) exists. If the answer is "NO" in step S9, that is, if it is determined that an unavailable function does not exist, the process proceeds to step S13, which will be described later. On the other hand, if the answer is "YES" in step S9, that is, if it is determined that an unavailable function exists, the operation key corresponding to the unavailable function is disabled in step S11, and the process proceeds to step S13.

[0102] In step S13, it is determined whether or not an additional function (additional function key) exists. If "NO" in step S13, that is, if it is determined that an additional function does not exist, the process proceeds to step S17 shown in Fig. 14. On the other hand, if "YES" in step S13, that is, if it is determined that an additional function exists, the notification function for the additional function is enabled in step S15, and the process proceeds to step S17.

[0103] As shown in FIG. 14, in step S17, a remote operation screen is displayed on the display 154, and in step S19, it is determined whether or not an operation key has been operated. If it is "NO" in step S19, that is, if it is determined that the operation key has not been operated, the process proceeds to step S25 described later. On the other hand, if it is "YES" in step S19, that is, if it is determined that the operation key has been operated, in step S21, a remote operation instruction is generated according to the function assigned to the operated operation key, and in step S25, the remote operation instruction is transmitted to the image forming apparatus 12, and the process proceeds to step S25. In step S25, it is determined whether or not to end the remote operation process. If it is "NO" in step S25, the process returns to step S17. On the other hand, if it is "YES" in step S25, the remote operation process is ended.

[0104] In the first embodiment, an additional function notification unit is provided for notifying the user of the presence of an additional function that is an operation function executable on the input device 130 of the image forming apparatus 12 to be operated but cannot be executed on the initial remote operation screen. This allows the user to recognize the presence of the additional function, thereby improving the operability of the user terminal 14 for remotely operating the image forming apparatus 12.

[0105] In the first embodiment, an extended operation section including additional function keys assigned to additional functions is displayed on the remote operation screen, so that the user can execute the additional functions, improving the operability of the user terminal 14 for remotely operating the image forming apparatus 12.

[0106] [Second Example] The information processing system 10 of the second embodiment is the same as the first embodiment except that the remote operation screen includes an operation area 260 that imitates the input device 130 of the image forming device 12 and a margin area 262 that is the part other than the operation area 260, and the extended operation section is displayed in the margin area 262. Therefore, we will explain the differences from the first embodiment and omit redundant explanations.

[0107] In the information processing system 10, the display 132 of the image forming apparatus 12 and the display 154 of the user terminal 14 may have different aspect ratios. For example, a display with an aspect ratio of 4:3 (square screen) may be adopted as the display 132 of the image forming apparatus 12, and a display with an aspect ratio of 16:9 (wide screen) may be adopted as the display 154 of the user terminal 14. In this way, when the aspect ratios of the display 132 of the image forming apparatus 12 and the display 154 of the user terminal 14 are different, as shown in FIG. 15, the remote operation screen (second remote operation screen 240) includes an operation area 260 imitating the input device 130 of the image forming apparatus 12, and a margin area 262 which is a portion other than the operation area 260.

[0108] 15, the aspect ratio of the display 132 of the image forming device 12 is 4:3, and the aspect ratio of the display 154 of the user terminal 14 is 16:9. In this case, a blank area 262 is generated at one end in the longitudinal direction of the display 154 of the user terminal 14. On the other hand, although not shown, if the aspect ratio of the display 132 of the image forming device 12 is 16:9, and the aspect ratio of the display 154 of the user terminal 14 is 4:3, a blank area is generated at one end in the lateral direction of the display 154 of the user terminal 14.

[0109] A message (additional function notification section) indicating the presence of an additional function and an extended operation section 264 including additional function keys 264a to 264c are displayed in the margin area 262. Therefore, the additional function keys can be operated without changing the layout of the operation area 260 that imitates the input device 130 of the image forming apparatus 12.

[0110] 16, an enlarged image 266 obtained by enlarging a part of the operation area 260 may be displayed in the margin area 262. That is, the remote operation screen of the second embodiment includes a first display mode in which an extended operation section 264 is displayed in the margin area 262, and a second display mode in which an enlarged image 266 is displayed in the margin area 262. However, the first display mode is set as an initial setting (default), and the display mode is switched when a predetermined user operation (display mode switching operation) is received. In addition, since the display mode switching operation is not an operation for remotely operating the image forming apparatus 12, an operation instruction for the display mode switching operation is not input from the user terminal 14 to the image forming apparatus 12.

[0111] In the second embodiment, in order to realize the above-mentioned operations, the information processing program for the user terminal 14 includes an enlarged image generating program for generating image data (enlarged image data) of an enlarged image 266 obtained by enlarging a part of the operation area 260 in the margin area 262. The display program 322c is also a program for outputting the enlarged image data to the display 154.

[0112] Hereinafter, the remote control processing in the second embodiment will be explained using a flow chart. The same reference symbols will be used for processes that are the same as those in the remote control processing explained in the first embodiment, and explanations of overlapping content will be omitted or will be explained briefly.

[0113] Fig. 17 is a flow diagram showing an example of the remote operation process of the second embodiment. As shown in Fig. 17, when the CPU 142 of the user terminal 14 starts the remote operation process, if "YES" in step S19, it judges in step S31 whether the display mode is to be switched, that is, whether a display mode switching operation has been accepted. If "NO" in step S31, that is, if it is determined that the display mode switching operation has not been accepted, it proceeds to step S21. On the other hand, if "YES" in step S31, that is, if it is determined that a display mode switching operation has been accepted, it switches the display mode in step S33 and returns to step S17.

[0114] The process contents from step S1 to step S25 are the same as those in the first embodiment, so a detailed description thereof will be omitted.

[0115] According to the second embodiment, the additional function keys can be operated without changing the layout of the operation area 260 that mimics the input device 130 of the image forming device 12, thereby improving the operability of the user terminal 14 for remotely operating the image forming device 12.

[0116] [Third Example] The information processing system 10 of the third embodiment is the same as that of the first embodiment except that the operation contents on the remote control screen are stored as standard operations. Therefore, only the contents that differ from the first embodiment will be described and duplicate explanations will be omitted.

[0117] In the third embodiment, the operation contents on the remote control screen may be stored as a standard operation. However, the operation contents on the remote control screen refer to the contents of a series of operations from when the type of the remote control screen is selected to when the remote control application is terminated.

[0118] Although not shown in the figure, when the remote operation application is terminated, the user is presented with a choice of whether to store the operation contents as a standard operation, and if there is an operation to store the operation contents as a standard operation, the operation contents are stored as the standard operation. In addition, the remote operation screen is provided with an operation key (standard operation read key) for reading out the standard operation, and when the standard operation read key is operated, a list of stored standard operations is displayed, and the standard operation can be selectively read out. When the standard operation to be read out is determined, an operation instruction according to the content of the standard operation is transmitted to the image forming apparatus 12.

[0119] In the third embodiment, in order to realize the above-mentioned operation, the information processing program for the user terminal 14 includes a storage program for storing the operation contents on the remote operation screen as a standard operation. The data of the standard operation (standard operation data) may be stored in the storage unit 156 of the user terminal 14, or may be stored in a storage unit of an external computer that can communicate with the user terminal 14 (for example, the storage unit 118 of the image forming apparatus 12).

[0120] Hereinafter, the remote control processing in the third embodiment will be explained using a flow chart. The same reference symbols will be used for processes that are the same as those in the remote control processing explained in the first embodiment, and explanations of overlapping content will be omitted or will be explained briefly.

[0121] Fig. 18 is a flow diagram showing an example of the remote operation process of the third embodiment. As shown in Fig. 18, when the CPU 142 of the user terminal 14 starts the remote operation process, if "YES" is determined in step S19, it determines in step S51 whether or not to read out a standard operation. If "NO" is determined in step S51, that is, if it is determined not to read out a standard operation, it proceeds to step S21. On the other hand, if "YES" is determined in step S51, that is, if it is determined to read out a standard operation, it transmits an operation instruction according to the content of the standard operation to the image forming apparatus 12 in step S53, and proceeds to step S23.

[0122] Also, if step S25 is "YES", then in step S55 it is determined whether or not to store the operation contents on the remote operation screen as a standard operation. If step S55 is "YES", that is, if it is determined that the operation contents on the remote operation screen are to be stored as standard operations, then in step S57 the operation contents on the remote operation screen are stored as standard operations, and the remote operation process is terminated. On the other hand, if step S55 is "NO", that is, if it is determined that the operation contents on the remote operation screen are not to be stored as standard operations, then the remote operation process is terminated without going through step S57.

[0123] The process contents from step S1 to step S25 are the same as those in the first embodiment, so a detailed description thereof will be omitted.

[0124] In the third embodiment, the operation contents on the remote operation screen can be stored as standard operations, and the standard operations can be read and executed. This simplifies the operation when the same operation is repeated, and the same operation can be executed regardless of the type of input device 130 of image forming apparatus 12, improving the operability of user terminal 14 for remotely operating image forming apparatus 12.

[0125] The aspect shown in the third embodiment can also be combined with the second embodiment.

[0126] The numerical values, screens, and specific configurations given in the above embodiment are merely examples and can be changed as appropriate according to the actual product. Also, the order in which each step in the flow chart shown in the above embodiment is processed can be changed as appropriate as long as the same results are obtained. [Explanation of symbols]

[0127] 10. Information processing systems 12...Image forming device 130...Input device 132…Display 138…Communications Department 14...User terminal 152…Touch panel 154…Display 150…Wireless communication

Claims

1. A control program executed by a control unit of a mobile terminal including a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control program including: displaying on the display a first remote operation screen having an image showing a function selection key for selecting a function of the information processing device to be operated; A control program that displays, in response to a predetermined user operation, a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation, at one end in a longitudinal direction of the display.

2. 2. The control program according to claim 1, wherein when the control unit is caused to display the first remote operation screen, the control unit is caused to display the first remote operation screen having an image showing function selection keys for setting any one of functions related to a copy job, including color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page aggregation.

3. 3. The control program according to claim 1, wherein the control unit terminates the remote control function when an end key is operated.

4. A control program as described in any one of claims 1 to 3, wherein, when there is an unavailable function that cannot be executed by the input device of the information processing device, the control unit displays the first remote operation screen that does not include an image indicating a function selection key corresponding to the unavailable function.

5. A control program as described in any one of claims 1 to 4, which, when causing the control unit to display the first operation screen and when causing the control unit to display the first operation area, displays the first operation screen and the second operation screen having any one of an operation screen corresponding to a software key type input device, an operation screen corresponding to a hard key type input device, and an operation screen corresponding to a combination type input device.

6. A method for controlling a mobile terminal including a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, comprising: displaying on the display a first remote operation screen having an image showing a function selection key for selecting a function of the information processing device to be operated; A control method comprising: displaying, at one end of the display in a longitudinal direction, a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation in response to a predetermined user operation.

Images