Control program and control method of portable terminal

Abstract

To improve operability of a portable terminal which can remotely operate an information processing device.SOLUTION: An information processing system (10) includes an image forming device (12) which is one example of an information processing device and a user terminal (14). The image forming device and the user terminal are directly and communicably connected by wireless communication etc. A first operation screen (260) which has an image of a function selection key for selecting a function which remotely operates the information processing device to be operated, and the image of first start keys (184, 186) for making the information processing device start predetermined operation is displayed on a display (154) of the user terminal. Operation instructions according to input operation to the first operation screen are transmitted to the information processing device. Then, according to predetermined user operation, a second operation screen (262) which does not have the image of the function selection key, and has the image of a second start key (266) for making the information processing device start the predetermined operation is displayed on the display.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to a control program and a control method for a mobile terminal, and more particularly to a control program and a control method for a mobile terminal that can remotely control an information processing device, for example. [Background technology]

[0002] An example of this type of background art is disclosed in Patent Document 1. When a portable terminal for remotely controlling an image forming apparatus according to the background art is used to remotely control the image forming apparatus, it executes an operation screen program for a standard model that is different from the model of the image forming apparatus in question, displays the operation screen for the standard model, generates commands in response to user input operations on this operation screen, and transmits the commands to the image forming apparatus. However, on the operation screen for the standard model displayed on the portable terminal, operation keys included in the operation screen for the standard model that do not correspond to any of the operation keys included in the operation screen of the image forming apparatus being operated are disabled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-228872 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the portable terminal for remotely controlling the image forming apparatus of the background art, among the operation keys included in the operation screen of the image forming apparatus to be operated, keys that do not correspond to any of the operation keys included in the operation screen of the standard model displayed on the portable terminal are not displayed, i.e., operation keys that can be executed on the image forming apparatus but cannot be executed on the operation screen for the portable terminal are not displayed. In other words, the portable terminal for remotely controlling the image forming apparatus of the background art has a problem in that some of the operation keys that can be executed on the image forming apparatus cannot be used, and there is room for improvement in operability.

[0005] Therefore, a primary object of the present invention is to provide a novel control program and control method for a mobile terminal.

[0006] Another object of the present invention is to provide a control program and a control method for a mobile terminal that can improve operability. [Means for solving the problem]

[0007] A first embodiment is a control program executed by a control unit of a mobile terminal having a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control program including a control unit that does not include a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation, but includes a second operation area having images indicating function selection keys for selecting functions of the information processing device to be operated, causes the control unit to display a first remote operation screen on the display that can accept predetermined user operations, displays the second operation area on one end of the display in the longitudinal direction so that a margin area that is the part other than the second operation area is formed, and when a predetermined user operation is accepted while the first remote operation screen is displayed, causes the control program to display the first operation area on the opposite side of one end of the display in the longitudinal direction.

[0008] In the first embodiment, a mobile terminal (14: reference numerals illustrate corresponding parts in the embodiments but are not intended to be limiting; the same applies hereinafter.) includes a display (154), a touch panel (152) provided on the display, and a communication unit (150) that communicates with an information processing device (12), and its control units (142, 148) do not include a first operation area (264) having an image indicating a start key for causing the information processing device to start a predetermined operation, but include a second operation area (260) having images (242, 248) indicating function selection keys for selecting functions of the information processing device to be operated according to a control program, and causes the display (154) to display a first remote operation screen (240) that can accept predetermined user operations, and also causes the second operation area (260) to be displayed on one end side in the longitudinal direction of the display so as to form a margin area (262) that is a portion other than the second operation area. When a predetermined user operation is received while the first remote operation screen (240) is displayed, a first operation area (264) is displayed on the opposite side of one end of the display in the longitudinal direction.

[0009] According to the first embodiment, it is possible to cause the information processing device to start a predetermined operation by operating the image showing the start key displayed in the first operation area.

[0010] The second embodiment is a control program that is subordinate to the first embodiment, and causes the control unit to display the second operation area so that a margin area is formed when the aspect ratio of the display differs from the aspect ratio of the display of the information processing device.

[0011] The third embodiment is a control program subordinate to the first or second embodiment, and when the control unit is caused to display the first remote operation screen, the control unit displays the first remote operation screen having images of function selection keys for setting any of the functions related to the copy job, such as color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page combining.

[0012] The fourth embodiment is a control program that is subordinate to any of the first to third embodiments, and causes the control unit to terminate the remote control function when the end key is operated.

[0013] The fifth embodiment is a control program subordinate to any of the first to fourth embodiments, and causes the control unit to display a first remote operation screen that does not include an image of a function selection key corresponding to the unavailable function when there is an unavailable function that cannot be executed by the input device of the information processing device. When the control unit is caused to display the first remote operation screen, the control unit displays the first remote operation screen having an image showing function selection keys for setting any of the functions related to a copy job, including color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page aggregation.

[0014] The sixth embodiment is a control program subordinate to any of the first to fifth embodiments, and when causing the control unit to display the first remote operation screen and the first operation area, causes the control unit to display the first remote operation screen and the first operation area having either an operation screen corresponding to a software key type input device, an operation screen corresponding to a hard key type input device, or an operation screen corresponding to a combined type input device.

[0015] The seventh embodiment is a control program dependent on any of the first to sixth embodiments, and the first operation area has an image showing a reset key.

[0016] An eighth embodiment is a control method for a mobile terminal having a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control method not including a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation, but including a second operation area having an image indicating a function selection key for selecting a function of the information processing device to be operated, the control method displays a first remote control screen on the display that can accept predetermined user operations, the second operation area is displayed on one end of the display in the longitudinal direction so that a margin area that is the part other than the second operation area is formed, and when a predetermined user operation is accepted while the first remote control screen is displayed, the control method displays the first operation area on the opposite side of one end of the display in the longitudinal direction.

[0017] The eighth embodiment is also expected to have the same effects as the first embodiment. [Effects of the Invention]

[0018] According to the present invention, it is possible to improve operability by remotely operating a plurality of information processing devices with different functions and operation methods using a mobile terminal.

[0019] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments with reference to the drawings. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing system according to the present invention. [Figure 2] FIG. 2 is a block diagram showing the electrical configuration of the image forming apparatus shown in FIG. [Figure 3] FIG. 3 is a block diagram showing the electrical configuration of the user terminal shown in FIG. [Figure 4] FIG. 4 is an illustrative view showing an example of a home screen of the image forming apparatus. [Figure 5] FIG. 5 is an illustrative view showing an example of a copy setting screen of the image forming apparatus. [Figure 6] FIG. 6 is an illustrative view showing an example of a selection screen of a user terminal. [Figure 7] FIG. 7 is an illustrative view showing one example of a first remote operation screen of the user terminal. [Figure 8] FIG. 8 is an illustrative view showing an example of a second remote operation screen of the user terminal. [Figure 9] FIG. 9 is an illustrative view showing another example of the second remote operation screen of the user terminal. [Figure 10] FIG. 10 is an illustrative view showing another example of the second remote operation screen of the user terminal. [Figure 11] FIG. 11 is an illustrative view showing an example of a memory map of the RAM of the image forming apparatus shown in FIG. [Figure 12] 1. FIG. 12 is an illustrative view showing one example of a memory map of the RAM of the user terminal shown in FIG. [Figure 13] FIG. 13 is a flowchart showing a part of an example of a remote control process of the CPU of the user terminal. [Figure 14] FIG. 14 is a flowchart showing part of the remote control processing, which follows FIG. [Figure 15] FIG. 15 is an illustrative view showing one example of the second remote operation screen of the second embodiment. [Figure 16] FIG. 16 is an illustrative view showing another example of the second remote operation screen of the second embodiment. [Figure 17] FIG. 17 is a flowchart showing a part of an example of the remote control process of the second embodiment. [Figure 18] FIG. 18 is a flowchart showing a part of an example of the remote control process of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] [First Example] Fig. 1 is a diagram showing an example of the configuration of an information processing system 10 according to the present invention. Referring to Fig. 1, information processing system 10 according to a first embodiment of the present invention includes an image forming device 12 and a user terminal (portable terminal) 14. As will be described in detail later, image forming device 12 and user terminal 14 are connected to each other so as to be able to communicate directly with each other via wireless communication or the like.

[0022] Although FIG. 1 shows one image forming apparatus 12 and one user terminal 14, there may be two or more image forming apparatuses 12 and user terminals 14.

[0023] Image forming device 12 is an example of an information processing device that is a communication target of user terminal 14, and is a multifunction peripheral (MFP) that has a copying function, a printer function, a scanner function, a facsimile (FAX) function, etc. Note that the present invention is applicable not only to multifunction peripherals but also to other information processing devices such as copying machines (copy machines), printing devices (printers), facsimiles, and image reading devices (scanners).

[0024] Fig. 2 is a block diagram showing the electrical configuration of image forming apparatus 12 shown in Fig. 1. Referring to Fig. 2, image forming apparatus 12 includes a CPU 122. Connected to CPU 122 via bus 120 are RAM 124, input control circuit 126, display control circuit 128, image forming unit 134, image reading unit 136, communication unit 138, and storage unit 118. Also connected to input control circuit 126 is input device 130, and connected to display control circuit 128 is display 132.

[0025] The CPU 122 is responsible for overall control of the image forming apparatus 12. The RAM 124 is used as a work area and a buffer area for the CPU 122.

[0026] Storage unit 118 is the main storage device of image forming apparatus 12, and is implemented by a non-volatile memory such as an HDD or EPPROM. Storage unit 118 stores data about a control program for CPU 122 to control the operation of each part of image forming apparatus 12, data about images (display images) for various screens, and data necessary for executing the control program. Storage unit 118 may also be configured to include RAM 124. The same applies to storage unit 156 of user terminal 14, which will be described later.

[0027] The input device 130 is used to accept input operations to the image forming apparatus 12 and includes, for example, a touch panel. The touch panel is a general-purpose touch panel, and any type of touch panel can be used, such as a capacitive type, an electromagnetic induction type, a resistive film type, or an infrared type. The touch panel may be provided on the display surface of the display 132 or separately from the display 132. However, a touch display in which the touch panel is integrally formed with the display 132 may also be provided. When a touch panel is provided on the display surface of the display 132, the display 132 displays a GUI (Graphical User Interface) including software keys (operation keys) such as various buttons and icons. In this case, input operations are accepted via the GUI (operation screen). The input device 130 may also include hardware operation buttons (hardware keys) instead of or in addition to the touch panel.

[0028] It should be noted that a software key refers to a key or icon that is reproduced by software on the display surface of the display 132. In contrast, a hardware key refers to a key or push button that is provided as a physical device.

[0029] The input control circuit 126 outputs an operation signal or operation data corresponding to an input operation to the input device 130 to the CPU 122. Furthermore, if the input device 130 includes a touch panel, the input control circuit 126 applies the necessary voltage to the touch panel, detects a touch operation (touch input) within the effective touch range of the touch panel, and outputs touch coordinate data indicating the position of the touch input to the CPU 122. Furthermore, if the input device 130 includes a hardware key, the input control circuit 126 outputs an operation signal or operation data corresponding to the operation of the hardware key to the CPU 122.

[0030] Display control circuit 128 includes a GPU, a VRAM, and the like. Under the direction of CPU 122, the GPU uses image generation data 304b (see FIG. 11) stored in RAM 124 to generate display image data in VRAM for displaying various operation screens on display 132, and outputs the generated display image data to display 132. Display 132 displays operation screens such as home screen 160 (see FIG. 4), which is a screen for selecting a desired job from various jobs that image forming apparatus 12 can execute. In this first embodiment, however, the jobs refer to scan, copy (including scanning of an original), fax, and the like. Display 132 can be an LCD (Liquid Crystal Display), an EL (Electro-Luminescence) display, or the like.

[0031] Image forming unit (printing device) 134 is a general-purpose laser printer that includes a photosensitive element, a charger, an exposure device, a developing device, a transfer device, a fixing device, etc., and prints an image corresponding to the print data processed by the image processing circuit onto recording paper (paper), etc. However, image forming unit 134 may be configured as an inkjet printer instead of a laser printer.

[0032] Although detailed explanation is omitted, the image forming unit 134 has a color printing function, and for each color of Y (yellow), M (magenta), C (cyan), and K (black), an image forming station including a photosensitive member, a charger, a developing device, etc. is configured.

[0033] Image reading unit 136 includes a light source, multiple mirrors, an imaging lens, a line sensor, etc. This image reading unit 136 exposes the surface of the document to the light source, and guides the light reflected from the document surface to the imaging lens using multiple mirrors. The imaging lens then forms an image of the reflected light on the light-receiving elements of the line sensor. The line sensor detects the luminance and chromaticity of the reflected light that forms an image on the light-receiving elements, and generates read image data based on the image of the document surface.

[0034] The communication unit 138 includes a wired communication circuit or a wireless communication circuit. The wired communication circuit is a circuit for performing wired communication in accordance with a known communication standard such as Ethernet (registered trademark) and performs processing for transmitting and receiving communication data to and from an external computer via a network such as the Internet. The wireless communication circuit is a circuit for performing wireless communication such as Wi-Fi and is connected to an antenna. This wireless communication circuit performs processing for transmitting and receiving communication data to and from an external computer using wireless communication such as Wi-Fi based on a wireless communication method in accordance with a known communication standard such as IEEE 802.11. The communication unit 138 also performs processing for transmitting and receiving communication data to and from the user terminal 14 using wireless communication. Note that the image forming apparatus 12 may be connected to a network such as the Internet. In this case, the communication unit 138 performs processing for transmitting and receiving communication data to and from the external computer via the network.

[0035] 2 is merely an example, and the electrical configuration of the image forming apparatus 12 is not limited to this. For example, a hardware keyboard may be connected to the image forming apparatus 12 as another input means.

[0036] The user terminal 14 is, for example, a portable (portable) terminal (mobile terminal) that can be carried around, such as a general-purpose smartphone, feature phone, or tablet PC, and is equipped with various functions, such as a calling function, an email function, and a browser function.

[0037] Fig. 3 is a block diagram showing the electrical configuration of the user terminal 14 shown in Fig. 1. Referring to Fig. 3, the user terminal 14 of the first embodiment includes a CPU 142. Connected to the CPU 142 via a bus 140 are a RAM 144, a touch panel control circuit 146, a display control circuit 148, a wireless communication unit 150, and a storage unit 156. Also connected to the touch panel control circuit 146 is a touch panel 152, and connected to the display control circuit 148 is a display 154.

[0038] The CPU 142 is responsible for overall control of the user terminal 14. The RAM 144 is used as a work area and a buffer area for the CPU 142.

[0039] Touch panel control circuit 146 applies the necessary voltage to touch panel 152, and when a touch input is detected, outputs touch coordinate data indicating the position of the touch input to CPU 142.

[0040] The display control circuit 148 includes a GPU, a VRAM, etc., and under the direction of the CPU 142, the GPU generates display image data in the VRAM for displaying various screens on the display 154 using the image generation data 324b (see Figure 12) stored in the RAM 144, and outputs the generated display image data to the display 154.

[0041] The wireless communication unit 150 includes a wireless communication circuit and the like, and performs transmission and reception processing of communication data transmitted and received to and from the image forming apparatus 12 based on a wireless communication method that complies with a known communication standard.

[0042] Note that the electrical configuration of the user terminal 14 shown in Figure 3 is merely an example and is not limited to this. For example, the user terminal 14 may be provided with hardware operation buttons as other input means. Also, for simplicity, components related to calls, etc. are omitted from Figure 3.

[0043] An example of the basic operation of the information processing system 10 will be described below with reference to Figures 4 and 5. When the main power of the image forming apparatus 12 is turned on and the image forming apparatus 12 is in a standby state in which each function can be executed, a home screen 160 as shown in Figure 4 is displayed on the display 132 of the image forming apparatus 12. As described above, the home screen 160 is a screen for selecting various jobs and is an example of an operation screen. Images of a plurality of operation keys (selection keys) 162 for selecting each job, such as copy, fax, and scan, are provided (displayed) on this home screen 160.

[0044] When a selection key 162 is touched (operated) on the home screen 160, a job assigned to the operated selection key 162 is selected, and a setting screen for configuring the job is displayed on the display 132. The following describes an example in which a copy job is selected.

[0045] When the selection key 162 to which a copy job is assigned on the home screen 160 is operated (a copy job is selected), a copy setting screen 180 such as that shown in Fig. 5 is displayed on the display 132. This copy setting screen 180 is a screen for setting the operating conditions of the copy job, and is an example of an operation screen.

[0046] The copy setting screen 180 is provided with operation keys such as a function selection key 182, a black and white start key 184, a color start key 186, and a numeric keypad 188.

[0047] The function selection key 182 is located at the left edge of the copy setting screen 180. A plurality of function selection keys 182 are provided and arranged in a vertical line. Each of the plurality of function selection keys 182 corresponds to an operating condition of a copy job or a function related to a copy job. Specifically, each of the function selection keys 182 is assigned a function for making detailed settings for each of the operating conditions, such as color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page combination.

[0048] A black and white start key (hereinafter referred to as the "black and white start key") 184 and a color start key (hereinafter referred to as the "color start key") 186 are displayed in the lower right corner area of ​​the copy setting screen 180. The black and white start key 184 and the color start key 186 are keys for causing the image forming apparatus 12 to start a predetermined operation. When the black and white start key 184 is operated, black and white copying is started. When the color start key 186 is operated, color copying is started according to the color setting.

[0049] Note that the process of generating black-and-white or color output image data that reflects various settings from scanned image data, and the process of forming black-and-white or color images according to the output image data are already well known, and as they differ from the essential content of the present invention, detailed explanations of these will be omitted.

[0050] Numeric keys 188 are number keys and are provided in the center of copy setting screen 180. When numeric keys 188 are operated, the operated number is displayed in a numeric key input display field above numeric keys 188. In other words, the number of copies is set.

[0051] As described above, the example was given of a case where a copy job was selected on the home screen 160. However, when a fax job or a scan job is selected, the setting screen required to set the operating conditions for those jobs is displayed on the display 132, and the operating conditions for the job can be set or the job can be executed.

[0052] In a conventional information processing system, an operation screen for operating an image forming apparatus is displayed on a user terminal that can communicate with the image forming apparatus, and the image forming apparatus can be remotely operated from the user terminal. Also, there is a conventional technology in which operation keys included in the operation screen of the user terminal that do not correspond to any of the operation keys included in the operation screen of the image forming apparatus to be operated are disabled.

[0053] However, in the above-mentioned conventional technology, among the operation keys included in the operation screen of the image forming device to be operated, keys that do not correspond to any of the operation keys included in the operation screen of the standard model displayed on the user terminal, that is, operation keys that can be executed by the image forming device but cannot be executed on the operation screen for the user terminal, are not displayed. In other words, in the above-mentioned conventional technology, there is a problem that some of the operation keys that can be executed by the image forming device cannot be used, and there is room for improvement in operability.

[0054] Therefore, in the information processing system 10 of the first embodiment, operation functions (operation keys) that can be executed on the image forming apparatus 12 but cannot be executed on the operation screen for the user terminal can also be operated.

[0055] An example of the operation of the information processing system 10 will be described below with reference to FIGS. 6 to 10. First, in the information processing system 10 of this embodiment, an application program (hereinafter referred to as a "remote operation application") for communicating with the image forming apparatus 12 and remotely operating the image forming apparatus 12 is stored in the user terminal 14. Simply put, by executing this remote operation application, a remote operation screen for the user terminal (hereinafter sometimes simply referred to as a "remote operation screen") for remotely operating the image forming apparatus 12 is displayed on the display 154 of the user terminal 14. Then, an operation instruction (remote operation instruction) corresponding to a touch operation (user operation) on the remote operation screen is input from the user terminal 14 to the image forming apparatus 12. When the image forming apparatus 12 accepts the remote operation instruction, it operates in accordance with the operation instruction.

[0056] However, the remote control application of this embodiment can use a plurality of types of remote control screens. For example, the remote control application of this embodiment can use a plurality of types of remote control screens that simulate input devices of various models of image forming apparatuses that actually exist. Examples of input devices for image forming apparatuses include input devices that accept user operations via software keys displayed on a touch panel display (software key type input devices), input devices that accept user operations via hardware keys or a hardware keyboard (hard key type input devices), and input devices that use both software keys and hard keys (combined type input devices).

[0057] Furthermore, the image forming apparatus 12 is classified into a business image forming apparatus that is mainly used in offices, etc. and a public image forming apparatus that is mainly used in public facilities, etc. Since the business image forming apparatus and the public image forming apparatus are intended for different user groups, the specifications of the input device are significantly different between the business image forming apparatus and the public image forming apparatus.

[0058] The remote control app can use a remote control screen including an operation area that simulates a software key input device, a remote control screen including an operation area that simulates a hard key input device, and a remote control screen including an operation area that simulates a combination type input device. Furthermore, each of the remote control screens corresponding to each type of input device includes a remote control screen corresponding to an input device for a business image forming device and a remote control screen corresponding to an input device for a public image forming device. Furthermore, a remote control screen including an operation area dedicated to remote control, separate from the actual input device, can also be used.

[0059] When the remote control application is executed (launched), a selection screen 200 is displayed on the display 154 of the user terminal 14, as shown in Fig. 6. The selection screen 200 displays operation keys (selection keys) 202 for selecting a remote control screen to be actually used from among multiple types of remote control screens available in the remote control application. Although not shown, the selection keys 202 may display characters, symbols, figures, etc. indicating information related to the assigned remote control screen. Furthermore, when the remote control screen assigned to the selection keys 202 corresponds to a specific model of input device, characters, symbols, figures, etc. indicating information related to the model may be displayed.

[0060] When a selection key 202 is operated on the selection screen 200, a remote operation screen assigned to the operated selection key 202 is selected. When a remote operation screen is selected, information about operation functions that can be executed on the operated remote operation screen (remote operation screen specification information) is acquired. Furthermore, when a remote operation screen is selected, the user terminal 14 communicates with the image forming apparatus 12 to acquire information about operation functions that can be executed on the input device 130 of the image forming apparatus 12 to be operated (apparatus side specification information). However, since the user operation on the selection screen 200 is not an operation for remotely operating the image forming apparatus 12 (it is an operation for internal processing of the user terminal 14), a remote operation instruction corresponding to the user operation on the selection screen 200 is not input from the user terminal 14 to the image forming apparatus 12.

[0061] Furthermore, when a remote operation screen is selected, a remote operation screen of the selected type is displayed on the display 154. For example, as shown in Fig. 7, a first remote operation screen 220 is displayed on the display 154. The first remote operation screen 220 is a screen for selecting a desired job from various jobs that can be executed by the image forming apparatus 12, and corresponds to the home screen 160 described above.

[0062] Images of operation keys (selection keys) 222 for selecting jobs such as copy, fax, scan, and network print are displayed on the first remote operation screen 220. The selection keys 222 correspond to the selection keys 162 on the home screen 160.

[0063] However, if there is an operation function (unusable function) that can be executed on the remote operation screen but cannot be executed on the input device of the image forming apparatus to be operated, i.e., if there is an operation key (unusable key) that can be executed on the remote operation screen but cannot be executed on the input device of the image forming apparatus to be operated, the unusable key is disabled. For example, the image forming apparatus 12 to be operated in this embodiment cannot execute a network print job. Therefore, the input device 130 of the image forming apparatus 12 to be operated does not have a selection key for selecting a network print job. Therefore, on the first remote operation screen 220, the selection key 222a for selecting a network print job corresponds to an unusable key and is disabled.

[0064] Furthermore, an unavailable key is displayed semi-transparently and marked with an X to indicate that the operation key cannot be used. Note that instead of displaying an X, the key may be displayed in gray or in a low-intensity color.

[0065] Furthermore, if there is an unavailable key, a notification section 224 is displayed on the remote operation screen to notify the user that there is an unavailable key. The notification section 224 is displayed in a blank space on the remote operation screen, or is displayed in front of the remote operation screen (as a pop-up).

[0066] In the example shown in FIG. 7, a notification section 224 containing a message to the effect that a network print job cannot be selected is displayed at the top of the first remote operation screen 220.

[0067] When a selection key 222 is operated on the first remote operation screen 220, a job assigned to the operated selection key 222 is selected, and a remote operation screen corresponding to a setting screen for setting the job is displayed on the display 154. However, when a user operation is accepted on an operation screen displayed after the first remote operation screen 220, a remote operation instruction corresponding to the user operation (corresponding to the function of the operated key) is input from the user terminal 14 to the image forming apparatus 12. Below, an example in which a copy job is selected will be described.

[0068] When the selection key 222 to which the copy job is assigned is operated (the copy job is selected) on the first remote operation screen 220, a second remote operation screen 240 as shown in Fig. 8 is displayed on the display 154. This second remote operation screen 240 is a screen for setting the operating conditions of the copy job, and corresponds to the copy setting screen 180 described above.

[0069] The second remote operation screen 240 is provided with a function selection key 242, a black and white start key 244, a color start key 246, and a numeric keypad 248. However, the function selection key 242 corresponds to the function selection key 182, the black and white start key 244 corresponds to the black and white start key 184, the color start key 246 corresponds to the color start key 186, and the numeric keypad 248 corresponds to the numeric keypad 188. These operation keys are assigned the same functions as the corresponding operation keys on the copy setting screen 180, so detailed explanations will be omitted.

[0070] For example, when either the monochrome start key 244 or the color start key 246 is operated, a remote control instruction to start a job is input to the image forming apparatus 12, the remote control app is terminated, and the remote control screen is hidden. The remote control app is also terminated if no operation is performed for a predetermined period of time while the remote control screen is displayed. An end key for terminating the remote control app may also be provided on the remote control screen. In this case, operating the end key terminates the remote control app.

[0071] However, if there is an operation function (additional function) that can be executed on the input device of the image forming apparatus to be operated but cannot be executed on the initial state remote operation screen, i.e., if there is an operation key (additional function key) that can be executed on the input device of the image forming apparatus to be operated but cannot be executed on the initial state remote operation screen, the notification function for the additional function will be enabled.

[0072] When the additional function notification function is enabled, an additional function notification section is provided on the remote operation screen to notify the user that the additional function is available. For example, the image forming apparatus 12 to be operated in this embodiment can perform double-sided printing, post-processing (e.g., stapling, punching, folding, etc.), and special paper copying in a copy job. On the other hand, the second remote operation screen 240 does not have operation keys corresponding to double-sided printing, post-processing, and special paper copying. Therefore, in this embodiment, the double-sided printing function, post-processing function, and special paper copying function correspond to additional functions, and the operation keys corresponding to double-sided printing, post-processing, and special paper copying correspond to additional function keys. Therefore, the second remote operation screen 240 has an additional function notification section 250 to notify the user that the double-sided printing function, post-processing function, and special paper copying function are available.

[0073] The added function notification section 250 displays a message indicating that an additional function exists. The added function notification section 250 also displays additional function keys 250a to 250c for executing the additional functions. That is, the added function notification section 250 is also an extended operation section including the additional function keys 250a to 250c assigned to the additional functions. For example, the added function notification section 250 is displayed in a blank area of ​​the remote operation screen (an area where no operation keys other than the added function keys are arranged). In this embodiment, the added function notification section 250 is displayed at the top end of the second remote operation screen 240.

[0074] Although not shown in the figures, when one of the additional function keys 250a to 250c is operated, an additional function setting screen is displayed for setting operating conditions for the additional function corresponding to the additional function key. The additional function setting screen is displayed in another blank space on the remote operation screen or is displayed overlapping the front of the remote operation screen. The same applies to the additional function keys 254a to 254c and 264a to 264c described below.

[0075] 8 is just an example, and additional function notification section 252 may be displayed overlapping on top of the remote operation screen as shown in Fig. 9. In this case, additional function notification section 252 displays a message indicating that additional functions exist, a first key ("Yes" key) 252a for displaying additional function keys corresponding to the additional functions, and a second key ("No" key) 252b for not displaying additional function keys corresponding to the additional functions.

[0076] When the first key 252a is operated, an extended operation section 254 including additional function keys 254a to 254c is displayed, as shown in Fig. 10. The extended operation section 254 is displayed overlapping the front of the remote operation screen. On the other hand, when the second key 252b is operated, the additional function notification section 252 is hidden. Note that while the additional function notification section 252 is displayed, no operations other than operating the first key 252a or the second key 252b can be performed, and no other operations are permitted.

[0077] The above-described operation of the information processing system 10 is realized by the CPU 122 of the image forming apparatus 12 executing an information processing program for the image forming apparatus 12 stored in the RAM 124, and the CPU 142 of the user terminal 14 executing an information processing program for the user terminal 14 stored in the RAM 144. Specific processing will be described later using a flow chart.

[0078] Fig. 11 is an illustrative view showing an example of a memory map 300 of the RAM 124 of the image forming apparatus 12 shown in Fig. 2. As shown in Fig. 11, the RAM 124 includes a program storage area 302 and a data storage area 304. As described above, the program storage area 302 of the RAM 124 stores information processing programs for the image forming apparatus 12. The information processing programs for the image forming apparatus 12 include a communication program 302a, an operation detection program 302b, a display program 302c, a transmission program 302d, and a reception program 302e.

[0079] The communication program 302a is a program for controlling the communication unit 138 to communicate (transmit / receive) data with an external computer via a network via a wired connection, and also for controlling the communication unit 138 to transmit / receive data wirelessly with the user terminal 14 and other external computers.

[0080] The operation detection program 302b is a program for detecting an input operation to the input device 130. For example, the operation detection program 302b is a program for acquiring touch coordinate data output from the input control circuit 126 when a touch panel is operated, and detecting that a software key displayed on the display 132 has been operated. The operation detection program 302b is also a program for acquiring operation data output from the input control circuit 90 when a hardware key is operated, and detecting that the hardware key has been operated.

[0081] Display program 302c is a program for generating display image data, that is, screen data such as home screen 160 and copy setting screen 180, using image generation data 304b (described later), and outputting the data to display 132.

[0082] The transmission program 302d is a program for transmitting the device side specification information data 304c to the user terminal 14. However, when transmitting, the communication program 302a is executed. The reception program 302e is a program for receiving operation instruction data transmitted from the user terminal 14. However, when receiving, the communication program 302a is executed.

[0083] Although not shown, the program storage area 302 also stores an image reading program and an image forming program. The image reading program is a program for controlling the image reading unit 136 to read an image of a document and output an image signal (read image data) corresponding to the read image. The image forming program is a program for controlling the image forming unit 134 to print a multicolor or monochrome image on paper. The program storage area 302 also stores programs for selecting and executing various functions provided in the image forming device 12.

[0084] Furthermore, data storage area 304 of RAM 124 stores operation input data 304a, image generation data 304b, device side specification information data 304c, operation instruction data 304d, and the like.

[0085] The operation input data 304a is touch coordinate data and / or operation data detected according to the operation detection program 302b. The detected touch coordinate data and / or operation data is stored in the data storage area 304 in chronological order. The image generation data 304b is data such as image data of various operation keys for generating display image data corresponding to various screens to be displayed on the display 132. The apparatus side specification information data 304c is data on information about operation functions executable on the input device 130 of the image forming apparatus 12 (information about operation keys executable on the input device 130). The operation instruction data 304d is data corresponding to the operation instruction data 324e transmitted from the user terminal 14.

[0086] Although not shown in the figure, the data memory area 304 stores other data necessary for executing the information processing program for the image forming device 12, and is provided with timers (counters) and registers necessary for executing the information processing program for the image forming device 12.

[0087] Fig. 12 is an illustrative view showing an example of a memory map 320 of the RAM 144 of the user terminal 14 shown in Fig. 3. As shown in Fig. 12, the RAM 144 includes a program storage area 322 and a data storage area 324. As described above, the program storage area 322 of the RAM 144 stores information processing programs for the user terminal 14. The information processing programs for the user terminal 14 include a communication program 322a, an operation detection program 322b, a display program 322c, a receiving program 322d, a determination program 322e, a function display control program 322f, an operation instruction generation program 322g, and a transmission program 322h.

[0088] The communication program 322a is a program for controlling the communication unit 208 to directly transmit and receive data to and from the image forming apparatus 12 using wireless communication.

[0089] The operation detection program 322b is a program for detecting touch input, and is a program for acquiring touch coordinate data output from the touch panel 152 and detecting that software keys included in various screens displayed on the display 154 have been operated.

[0090] Display program 322c is a program for generating display image data, that is, screen data such as the remote control screen described above, using image generation data 324b (described later), and outputting it to display 154.

[0091] The reception program 322d is a program for receiving the apparatus side specification information data 304c transmitted from the image forming apparatus 12. However, when receiving, the communication program 322a is executed.

[0092] The determination program 322e is a program for comparing the device-side specification information indicated by the device-side specification information data 324c of the image forming device to be operated with the specification information of the remote operation screen indicated by the terminal-side specification information data 324d to determine whether there are any unavailable functions, unavailable keys, or additional functions and additional function keys.

[0093] The function display control program 322f is a program for disabling an operation key corresponding to an unavailable function (unavailable key) among the operation keys displayed on the remote operation screen when an unavailable function (unavailable key) exists. The function display control program 322f is also a program for displaying an additional function notification section including the additional function key on the remote operation screen when an additional function (additional function key) exists, thereby making the additional function key operable.

[0094] The operation instruction generating program 322g is a program for generating operation instruction data 324e regarding a remote operation instruction (remote operation command) in response to a touch operation on the remote operation screen.

[0095] The transmission program 322e is a program for transmitting the operation instruction data 324e to the image forming apparatus 12. However, during transmission, the communication program 322a is executed.

[0096] Although not shown, the program storage area 322 also stores programs for selecting and executing various functions provided in the user terminal 14.

[0097] Furthermore, data storage area 324 of RAM 144 stores operation input data 324a, image generation data 324b, device side specification information data 324c, terminal side specification information data 324d, operation instruction data 324e, and the like.

[0098] The operation input data 324a is touch coordinate data detected in accordance with the operation detection program 302b. The detected touch coordinate data is stored in the data storage area 324 in chronological order. The image generation data 324b is data such as polygon data or texture data for generating display image data corresponding to various screens to be displayed on the display 132. The device-side specification information data 324c is data corresponding to the device-side specification information data 304c transmitted from the image forming device 12. The terminal-side specification information data 324d is data about information about operation functions that can be executed from the beginning on the selected remote operation screen (information about operation keys that can be executed on the remote operation screen in the initial state). The operation instruction data 324e is data about operation instructions corresponding to touch operations on the remote operation screen.

[0099] Although not shown in the figure, the data memory area 324 stores other data necessary for executing the information processing program for the user terminal 14, and is provided with timers (counters) and registers necessary for executing the information processing program for the user terminal 14.

[0100] Fig. 13 is a flow diagram showing an example of remote control processing executed by the CPU 142 of the user terminal 14 shown in Fig. 3. This remote control processing starts when a remote control application is executed (launched).

[0101] As shown in FIG. 13, when the CPU 142 of the user terminal 14 starts the remote control processing, in step S1, it displays a selection screen 200 on the display 154 and determines whether a remote control screen to be used has been selected from among multiple types of remote control screens available in the remote control application.

[0102] If step S1 is "NO," that is, if it is determined that the remote operation screen to be used has not been selected, the process returns to the same step S1. On the other hand, if step S1 is "YES," that is, if it is determined that the remote operation screen to be used has been selected, then in step S3, information about the operation functions of the remote operation screen to be used, that is, the terminal-side specification information, is obtained, and in step S5, information about the operation functions of the image forming apparatus 12 to be operated, that is, the apparatus-side specification information, is obtained, and the process proceeds to step S7.

[0103] Next, in step S7, the specification information on the device side is compared with the specification information on the terminal side, and in step S9, it is determined whether or not there is an unavailable function (unavailable key). If the answer is "NO" in step S9, that is, if it is determined that there is no unavailable function, the process proceeds to step S13, which will be described later. On the other hand, if the answer is "YES" in step S9, that is, if it is determined that there is an unavailable function, the operation key corresponding to the unavailable function is disabled in step S11, and the process proceeds to step S13.

[0104] In step S13, it is determined whether or not an additional function (additional function key) exists. If "NO" in step S13, that is, if it is determined that an additional function does not exist, the process proceeds to step S17 shown in Fig. 14. On the other hand, if "YES" in step S13, that is, if it is determined that an additional function exists, the notification function for the additional function is enabled in step S15, and the process proceeds to step S17.

[0105] 14, in step S17, a remote operation screen is displayed on display 154, and in step S19, it is determined whether or not an operation key has been operated. If the result in step S19 is "NO," that is, if it is determined that an operation key has not been operated, the process proceeds to step S25, which will be described later. On the other hand, if the result in step S19 is "YES," that is, if it is determined that an operation key has been operated, in step S21, a remote operation instruction is generated according to the function assigned to the operated operation key, and in step S25, the remote operation instruction is sent to image forming apparatus 12, and the process proceeds to step S25. In step S25, it is determined whether or not to end the remote operation process. If the result in step S25 is "NO," the process returns to step S17. On the other hand, if the result in step S25 is "YES," the remote operation process ends.

[0106] In the first embodiment, an additional function notification unit is provided to notify the user that there is an additional function that is an operation function that can be executed on the input device 130 of the image forming apparatus 12 to be operated but cannot be executed on the initial remote operation screen. This allows the user to recognize the existence of the additional function, thereby improving the operability of the user terminal 14 for remotely operating the image forming apparatus 12.

[0107] In the first embodiment, an extended operation section including additional function keys assigned to additional functions is displayed on the remote operation screen, allowing the user to execute the additional functions, thereby improving the operability of the user terminal 14 for remotely operating the image forming apparatus 12.

[0108] [Second Example] The information processing system 10 of the second embodiment is the same as the first embodiment except that the remote operation screen includes an operation area 260 that imitates the input device 130 of the image forming device 12 and a margin area 262 that is the part other than the operation area 260, and the extended operation section is displayed in the margin area 262. Therefore, we will only explain the differences from the first embodiment and omit redundant explanations.

[0109] In the information processing system 10, the aspect ratio may differ between the display 132 of the image forming apparatus 12 and the display 154 of the user terminal 14. For example, a display with an aspect ratio of 4:3 (square screen) may be used as the display 132 of the image forming apparatus 12, and a display with an aspect ratio of 16:9 (wide screen) may be used as the display 154 of the user terminal 14. When the aspect ratios of the display 132 of the image forming apparatus 12 and the display 154 of the user terminal 14 differ in this way, as shown in FIG. 15 , the remote operation screen (second remote operation screen 240) includes an operation area 260 that resembles the input device 130 of the image forming apparatus 12, and a margin area 262 that is the portion other than the operation area 260.

[0110] 15, the aspect ratio of the display 132 of the image forming device 12 is 4:3, and the aspect ratio of the display 154 of the user terminal 14 is 16:9. In this case, a blank area 262 is generated at one end in the longitudinal direction of the display 154 of the user terminal 14. On the other hand, although not shown, if the aspect ratio of the display 132 of the image forming device 12 is 16:9 and the aspect ratio of the display 154 of the user terminal 14 is 4:3, a blank area is generated at one end in the lateral direction of the display 154 of the user terminal 14.

[0111] A message indicating the existence of additional functions (additional function notification section) and an extended operation section 264 including additional function keys 264a to 264c are displayed in the margin area 262. Therefore, the additional function keys can be operated without changing the layout of the operation area 260 that imitates the input device 130 of the image forming apparatus 12.

[0112] 16, an enlarged image 266 obtained by enlarging a portion of the operation area 260 may be displayed in the margin area 262. That is, the remote operation screen of the second embodiment includes a first display mode in which an extended operation section 264 is displayed in the margin area 262, and a second display mode in which an enlarged image 266 is displayed in the margin area 262. However, the first display mode is set as the initial setting (default), and the display mode is switched when a predetermined user operation (display mode switching operation) is received. Furthermore, because the display mode switching operation is not an operation for remotely operating the image forming apparatus 12, an operation instruction for the display mode switching operation is not input from the user terminal 14 to the image forming apparatus 12.

[0113] In the second embodiment, in order to realize the above-described operations, the information processing program for the user terminal 14 includes an enlarged image generation program for generating image data (enlarged image data) of an enlarged image 266 obtained by enlarging a part of the operation area 260 in the margin area 262. The display program 322c is also a program for outputting the enlarged image data to the display 154.

[0114] Below, the remote control processing in the second embodiment will be explained using a flow chart. The same processes as those in the remote control processing explained in the first embodiment will be given the same reference symbols, and explanations of overlapping content will be omitted or will be explained briefly.

[0115] 17 is a flow diagram showing an example of the remote control processing of the second embodiment. As shown in FIG. 17, when the CPU 142 of the user terminal 14 starts the remote control processing, if "YES" in step S19, it determines in step S31 whether the display mode is to be switched, that is, whether a display mode switching operation has been accepted. If "NO" in step S31, that is, if it is determined that a display mode switching operation has not been accepted, it proceeds to step S21. On the other hand, if "YES" in step S31, that is, if it is determined that a display mode switching operation has been accepted, it switches the display mode in step S33 and returns to step S17.

[0116] The process from step S1 to step S25 is the same as that in the first embodiment, so a detailed description will be omitted.

[0117] According to this second embodiment, the additional function keys can be operated without changing the layout of the operation area 260 that mimics the input device 130 of the image forming device 12, thereby improving the operability of the user terminal 14 for remotely operating the image forming device 12.

[0118] [Third Example] The information processing system 10 of the third embodiment is the same as that of the first embodiment except that the operation content on the remote control screen is stored as standard operations. Therefore, we will explain the content that differs from the first embodiment and omit redundant explanations.

[0119] In the third embodiment, the operation contents on the remote control screen may be stored as a standard operation. However, the operation contents on the remote control screen refer to the series of operations from when the type of the remote control screen is selected until the remote control application is terminated.

[0120] Although not shown in the figure, when the remote control application is terminated, the user is presented with an option of whether to store the operation content as a standard operation, and if an operation to store the operation content as a standard operation is performed, the operation content is stored as the standard operation. The remote control screen also has an operation key (standard operation read key) for reading out a standard operation. When the standard operation read key is operated, a list of stored standard operations is displayed, allowing a standard operation to be selectively read out. When the standard operation to be read out is determined, an operation instruction corresponding to the content of the standard operation is transmitted to the image forming apparatus 12.

[0121] In the third embodiment, in order to realize the above-described operations, the information processing program for the user terminal 14 includes a storage program for storing the operation contents on the remote operation screen as standard operations. Note that the data of the standard operations (standard operation data) may be stored in the storage unit 156 of the user terminal 14, or may be stored in a storage unit of an external computer that can communicate with the user terminal 14 (for example, the storage unit 118 of the image forming apparatus 12).

[0122] Below, the remote control processing in the third embodiment will be explained using a flow chart. The same reference symbols will be used for the same processes as those in the remote control processing explained in the first embodiment, and explanations of overlapping content will be omitted or will be explained briefly.

[0123] 18 is a flow diagram showing an example of the remote operation processing of the third embodiment. As shown in FIG. 18, when the CPU 142 of the user terminal 14 starts the remote operation processing, if the answer is "YES" in step S19, the CPU 142 determines in step S51 whether to read out a standard operation. If the answer is "NO" in step S51, that is, if it is determined that the standard operation is not to be read out, the process proceeds to step S21. On the other hand, if the answer is "YES" in step S51, that is, if it is determined that the standard operation is to be read out, the process transmits an operation instruction corresponding to the content of the standard operation to the image forming apparatus 12 in step S53, and the process proceeds to step S23.

[0124] Also, if step S25 is "YES", then in step S55 it is determined whether or not to store the operation contents on the remote control screen as a standard operation. If step S55 is "YES", that is, if it is determined that the operation contents on the remote control screen are to be stored as a standard operation, then in step S57 the operation contents on the remote control screen are stored as a standard operation, and the remote control processing is terminated. On the other hand, if step S55 is "NO", that is, if it is determined that the operation contents on the remote control screen are not to be stored as a standard operation, then the remote control processing is terminated without going through step S57.

[0125] The process from step S1 to step S25 is the same as that in the first embodiment, so a detailed description will be omitted.

[0126] In the third embodiment, the operation contents on the remote operation screen can be stored as standard operations, and the standard operations can be read and executed. This simplifies the operation when repeating the same operation, and the same operation can be executed regardless of the type of input device 130 of image forming apparatus 12, thereby improving the operability of user terminal 14 for remotely operating image forming apparatus 12.

[0127] The aspect shown in the third embodiment can also be combined with the second embodiment.

[0128] Furthermore, the numerical values, screens, specific configurations, etc. given in the above-mentioned embodiments are merely examples and can be changed as appropriate depending on the actual product. Furthermore, the order in which each step in the flow chart shown in the above-mentioned embodiments is processed can be changed as appropriate as long as the same results are obtained. [Explanation of symbols]

[0129] 10. Information Processing Systems 12...Image forming device 130...Input device 132...Display 138…Communications Department 14...User terminal 152...Touch panel 154...Display 150...Wireless communication

Claims

1. A control program executed by a control unit of a mobile terminal including a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, the control program including: a first remote operation screen that does not include a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation, but includes a second operation area having an image indicating a function selection key for selecting a function of the information processing device to be operated, and that is capable of accepting predetermined user operations, and the second operation area is displayed on one end side of the display in a longitudinal direction so as to form a margin area that is a portion other than the second operation area; A control program that, when the specified user operation is received while the first remote control screen is displayed, displays the first operation area on the opposite side of the one end in the longitudinal direction of the display.

2. The control program according to claim 1 , further comprising causing the control unit to display the second operation area so as to form the margin area when an aspect ratio of the display differs from an aspect ratio of a display of the information processing device.

3. 3. The control program according to claim 1, wherein when the control unit is caused to display the first remote operation screen, the control unit is caused to display the first remote operation screen having an image showing function selection keys for setting any of the following functions related to a copy job: color setting, document orientation, paper selection, double-sided copying, magnification, copy density, and page combination.

4. 4. The control program according to claim 1, wherein the control unit terminates the remote control function when an end key is operated.

5. A control program as described in any one of claims 1 to 4, wherein, when there is an unavailable function that cannot be executed by the input device of the information processing device, the control unit displays the first remote operation screen that does not include an image indicating a function selection key corresponding to the unavailable function.

6. A control program according to any one of claims 1 to 5, wherein when the control unit is caused to display the first remote operation screen and the first operation area, the control unit is caused to display the first remote operation screen and the first operation area, which have either an operation screen corresponding to a software key type input device, an operation screen corresponding to a hard key type input device, or an operation screen corresponding to a combined type input device.

7. 7. The control program according to claim 1, wherein the first operation area has an image representing a reset key.

8. A method for controlling a mobile terminal including a display, a touch panel provided on the display, and a communication unit that communicates with an information processing device, comprising: a first remote operation screen that does not include a first operation area having an image indicating a start key for causing the information processing device to start a predetermined operation, but includes a second operation area having an image indicating a function selection key for selecting a function of the information processing device to be operated, and that is capable of accepting predetermined user operations, and the second operation area is displayed on one end side of the display in a longitudinal direction so as to form a margin area that is a portion other than the second operation area; A control method in which, when the specified user operation is received while the first remote control screen is displayed, the first operation area is displayed on the opposite side of the one end of the display in the longitudinal direction.

Images