Operation input control method, operation input device, image processing device

Abstract

The ability to operate a virtual keyboard without confusion, even if the key identification marks on some of the virtual keyboard keys are not present on any of the multiple hardware keys on an external keyboard. [Solution] When no external keyboard is connected, the processor 51 displays the first virtual keyboard G2a on the display device 3a. When the external keyboard is connected, the processor 51 displays the second virtual keyboard G2b on the display device 3a, in which the virtual key identification marks on specific virtual keys in the first virtual keyboard G2a are replaced with a combination of a first mark and a second mark. The first mark is the virtual key identification mark or a simplified version of the virtual key identification mark. The second mark represents a key among a plurality of hardware keys included in the external keyboard that corresponds to the specific virtual key.

Description

Technical Field

[0001] The present invention relates to an operation input control method, an operation input device, and an image processing device for controlling operation input through a virtual keyboard displayed on a display device or an external keyboard connected to a connection interface.

Background Art

[0002] An image processing device such as a printer, a copier, a facsimile device, or a multifunction device includes an operation input device that detects operation input related to image processing.

[0003] The operation input device may display a virtual keyboard on a display device and detect operation input through the virtual keyboard.

[0004] For example, the operation input device includes a touch panel that detects a touch operation on a display area of the virtual keyboard. Further, the operation input device may include a trackball device that changes the position of a cursor in the display area of the virtual keyboard and detects a click operation.

[0005] Also, it is known that an image forming apparatus changes the display state of a plurality of software numeric keys displayed on the display device according to operations on a plurality of hardware numeric keys of an input device connected to the image forming apparatus (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Incidentally, there are cases where key identification marks, which are present on some of the virtual keys on the virtual keyboard, are not present on any of the hardware keys on the external keyboard. In this case, the user may be confused when operating the external keyboard.

[0008] The object of the present invention is to provide an operation input control method, an operation input device, and an image processing device that enable the operator to operate an external keyboard without confusion, even when key identification marks on some of the virtual keys on a virtual keyboard are not marked on any of the hardware keys on the external keyboard. [Means for solving the problem]

[0009] An operation input control method according to one aspect of the present invention is a method for controlling an operation input device equipped with a connection interface to which an external keyboard can be connected and a display device. The operation input control method includes a processor determining whether the operation input device is in a disconnected state where the external keyboard is not connected or in a connected state where the external keyboard is connected. Furthermore, if the operation input device is determined to be in the disconnected state, the operation input control method includes the processor causing the display device to display a first virtual keyboard including a plurality of normal virtual keys, each marked with a virtual key identification mark representing a key type. Furthermore, if the operation input device is determined to be in the connected state, the operation input control method includes the processor causing the display device to display a second virtual keyboard in which the virtual key identification marks on specific virtual keys, which are part of the plurality of normal virtual keys on the first virtual keyboard, are replaced with a combination of a first mark and a second mark. Furthermore, the operation input control method includes detecting operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard. The first mark is the virtual key identification mark of the specific virtual key or a simplified version of the virtual key identification mark of the specific virtual key. The second mark is a mark representing the key among the multiple hardware keys included in the external keyboard that corresponds to the specific virtual key.

[0010] An operation input device according to another aspect of the present invention comprises the connection interface, the display device, and the processor that implements the operation input control method.

[0011] An image processing apparatus according to another aspect of the present invention comprises an operation input device and one or more image processing units that perform image processing in accordance with an operation detected by the operation input device. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide an operation input control method, an operation input device, and an image processing device that enable the operator to operate an external keyboard without confusion, even if the key identification marks on some of the virtual keys on the virtual keyboard are not marked on any of the hardware keys on the external keyboard. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a diagram showing the configuration of an image processing apparatus equipped with an operation input device according to an embodiment. [Figure 2] Figure 2 is a flowchart showing an example of the procedure for keyboard input control in the operation input device according to the embodiment. [Figure 3] Figure 3 shows an example of an input screen in an operation input device according to the embodiment. [Figure 4] Figure 4 shows an example of an input screen in which the first virtual keyboard is displayed as a pop-up in the operation input device according to the embodiment. [Figure 5] Figure 5 shows an example of an input screen in which a second virtual keyboard is displayed as a pop-up in the operation input device according to the embodiment. [Figure 6] Figure 6 shows an example of an input screen in which the first virtual keyboard is displayed as a pop-up in a first modified example of the operation input device according to the embodiment. [Figure 7] Figure 7 shows an example of an input screen in which a second virtual keyboard is displayed as a pop-up in a first modified example of the operation input device according to the embodiment. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described below with reference to the drawings. Note that the following embodiments are merely examples of the present invention and do not limit the technical scope of the present invention.

[0015] [Configuration of the image processing device 10] The operation input device 3x according to the embodiment constitutes a part of the image processing device 10.

[0016] The operation input device 3x has a function of detecting operation input non - contact by recognizing the voice of an operator. The image processing device 10 can execute various image processes such as image formation processing, image reading processing, and image transmission processing. For example, the image processing device 10 is a copier, a facsimile device, or a multifunction device, etc.

[0017] In the example shown in FIG. 1, the image processing device 10 includes an image reading device 1, an image forming device 2, a user interface device 3, a communication device 4, a control device 5, and a secondary storage device 6.

[0018] The image reading device 1 executes the image reading process. The image reading process is a process of reading an image from the original document 91 and outputting the data of the read image. The read image is an image read from the original document 91.

[0019] The image forming device 2 executes the image forming process by a predetermined method such as an electrophotographic method or an ink - jet method. The image forming process is a process of forming an image on the sheet 92.

[0020] The image forming device 2 includes a sheet conveyance mechanism for conveying the sheet 92 and a printing device for forming an image on the sheet 92.

[0021] The user interface device 3 includes a display device 3a and a touch panel 3b. The display device 3a has a screen capable of displaying information. For example, the display device 3a includes a display panel such as a liquid crystal display panel.

[0022] The touch panel 3b is formed on the screen of the display device 3a and detects human contact operations. For example, the touch panel 3b is a capacitive or resistive film type panel.

[0023] Communication device 4 is a communication interface device that communicates with other devices, such as a host device, via a network. Control device 5 performs all data transmission and reception with the other devices through communication device 4.

[0024] The image forming apparatus 2 performs the image forming process based on the data of the read image or the received print data. The received print data is data included in the print request received from the host device via the communication device 4.

[0025] Furthermore, the communication device 4 is capable of performing the image transmission process. The image transmission process is the process of transmitting the data of the read image to a specified destination via the network.

[0026] The control device 5 performs various calculations, data processing, and controls various electrical devices provided by the image processing device 10. The control device 5 includes a CPU 51 and RAM (Random Access Memory) 52, among others.

[0027] The secondary storage device 6 is a computer-readable, non-volatile storage device. The secondary storage device 6 can store computer programs and various types of data. For example, either or both of an SSD (Solid State Drive) and / or a hard disk drive may be used as the secondary storage device 6.

[0028] The secondary storage device 6 stores the computer program executed by the CPU 51 and the data referenced by the CPU 51. The CPU 51 is an example of a processor.

[0029] The CPU 51 is a processor that performs various data processing and control operations by executing the computer program stored in the secondary storage device 6.

[0030] It is also conceivable that other processors, such as a DSP, could perform the data processing and control in place of the CPU 51.

[0031] RAM52 is a computer-readable volatile memory device. RAM52 primarily stores the computer program executed by CPU51 and the data that CPU51 outputs and references during the execution of the computer program.

[0032] The CPU 51 includes a plurality of processing modules that are realized by executing the computer program. The plurality of processing modules in the CPU 51 include a UI control unit 5a and a job control unit 5b, etc.

[0033] The UI control unit 5a controls the user interface device 3. The UI control unit 5a controls the display screen of the display device 3a and generates operation events in response to touch operations detected by the touch panel 3b.

[0034] For example, the UI control unit 5a detects an operation input related to the image processing and generates an operation event corresponding to the detection result of the operation input.

[0035] The job control unit 5b controls the image forming apparatus 2, the image reading apparatus 1, and the communication device 4. The job control unit 5b causes some or all of the image forming apparatus 2, the image reading apparatus 1, and the communication device 4 to execute jobs corresponding to requests entered through the touch panel 3b or the communication device 4.

[0036] The job includes some or all of the multiple types of image processing. In this embodiment, the job includes an image reading job, a print job, and a copy job.

[0037] The image reading job includes the image reading process performed by the operation of the image reading device 1. The print job includes the image forming process performed by the operation of the image forming device 2. The copy job includes the image reading process and the image forming process performed by the operation of the image reading device 1 and the image forming device 2. That is, the job includes one or more types of image processing.

[0038] For example, when the operation event related to the image processing occurs, the job control unit 5b executes the image processing corresponding to the operation event.

[0039] The image reading device 1, the image forming device 2, and the communication device 4 are examples of image processing units that perform the aforementioned image processing in accordance with the operation detected by the operation input device 3x.

[0040] In this embodiment, the user interface device 3 further includes a connection interface 3c to which an external keyboard 8 can be connected.

[0041] The external keyboard 8 is a hardware keyboard that includes multiple hardware keys 8a. Each of the multiple hardware keys 8a is marked with a hardware key identification mark 8b that indicates the type of key. For example, the hardware key identification mark 8b may be a letter, string of characters, or symbol.

[0042] For example, connection interface 3c is a serial interface circuit such as a Universal Serial Bus (USB) that allows for wired connection of an external keyboard 8. Alternatively, connection interface 3c may be a Bluetooth interface circuit that allows for wireless connection of the external keyboard 8. Note that Bluetooth is a registered trademark.

[0043] The UI control unit 5a can input text data according to the operations performed on multiple hardware keys 8a detected by the external keyboard 8.

[0044] Furthermore, the UI control unit 5a can display a virtual keyboard on the display device 3a and input text data according to the operation content of the virtual keyboard detected by the touch panel 3b.

[0045] In other words, the UI control unit 5a can perform text data input processing in response to touch operations detected by the touch panel 3b or operations on multiple hardware keys 8a detected by the external keyboard 8 when the virtual keyboard is displayed on the display device 3a.

[0046] In this embodiment, the user interface device 3 and the UI control unit 5a of the CPU 51 constitute the operation input device 3x. The CPU 51 is an example of a processor that controls the operation input device 3x.

[0047] Incidentally, there are cases where the key identification marks on some of the virtual keys on the virtual keyboard are not present on any of the hardware keys 8a on the external keyboard 8. In this case, the user may be confused about how to operate the external keyboard 8.

[0048] In this embodiment, the UI control unit 5a performs keyboard input control, which will be described later (see Figure 2). The keyboard input control includes processing to enable the operator to operate the external keyboard 8 without confusion.

[0049] The UI control unit 5a executes the keyboard input control when a text input request operation is detected by the touch panel 3b. For example, the text input request operation is an operation to select one of several operation icons G1 on the input screen G10.

[0050] [Keyboard Input Control] Below, an example of the procedure for controlling keyboard input will be described with reference to the flowchart shown in Figure 2.

[0051] The keyboard input control is an example of a process that implements an operation input control method for controlling the operation input device 3x. The CPU 51 that executes the keyboard input control is an example of a processor that implements the operation input control.

[0052] In the following description, S1, S2, ... represent identification codes for multiple processes in the keyboard input control. In the keyboard input control, the processing of process S1 is executed first.

[0053] <Process S1> In step S1, the UI control unit 5a determines whether the operation input device 3x is in a disconnected state (without an external keyboard 8 connected) or a connected state (with an external keyboard 8 connected).

[0054] The method by which the UI control unit 5a determines the state of the operation input device 3x will be described later. In this embodiment, the UI control unit 5a determines that the initial state of the operation input device 3x is the disconnected state.

[0055] The UI control unit 5a executes the process of step S2 when it determines that the operation input device 3x is in the disconnected state, and executes the process of step S3 when it determines that the operation input device 3x is in the connected state.

[0056] <Process S2> In step S2, the UI control unit 5a displays the first virtual keyboard G2a and input frame G4 on the display device 3a (see Figure 4).

[0057] The first virtual keyboard G2a includes multiple regular virtual keys G3, each marked with a virtual key identification mark G3a representing the type of key. For example, the virtual key identification mark G3a can be a letter, string of characters, or symbol. In the example shown in Figure 5, "OK" is the virtual key identification mark G3a representing the confirm key.

[0058] Input field G4 is a display area where one or more characters or symbols entered by operations on each of the multiple normal virtual keys G3 are displayed.

[0059] In this embodiment, the UI control unit 5a displays the first virtual keyboard G2a and input frame G4 as a pop-up overlaid on the input screen G10 (see Figure 4).

[0060] The UI control unit 5a executes the process of process S4 after executing the process of process S2.

[0061] <Process S3> Meanwhile, in step S3, the UI control unit 5a displays the second virtual keyboard G2b and input frame G4 on the display device 3a (see Figure 5).

[0062] The second virtual keyboard G2b is a virtual keyboard in which the virtual key identification marks G3a, which are inscribed on specific virtual keys G3x that are part of the multiple normal virtual keys G3 on the first virtual keyboard G2a, are replaced with the accompanying marks G3b (see Figure 5).

[0063] The combined mark G3b includes the combined first mark G3c and second mark G3d. The first mark G3c is either the virtual key identification mark G3a of a specific virtual key G3x or a simplified version of the virtual key identification mark G3a of a specific virtual key G3x.

[0064] The second mark G3d represents a key among the multiple hardware keys 8a included in the external keyboard 8 that corresponds to a specific virtual key G3x. In the example shown in Figure 5, the newline key and the confirm key are the specific virtual key G3x.

[0065] Furthermore, the second mark G3d on the newline key represents the ENTER key on the external keyboard 8. On the other hand, the second mark G3d on the confirm key represents the CONTROL key on the external keyboard 8.

[0066] For example, the first mark G3c and the second mark G3d are each letters, strings of characters, or symbols.

[0067] The input field G4, which is displayed together with the second virtual keyboard G2b, is a display area where one or more characters or one or more symbols are displayed, entered by operating on each of the multiple normal virtual keys G3 or a specific virtual key G3x.

[0068] In this embodiment, the UI control unit 5a displays the second virtual keyboard G2b and input frame G4 as a pop-up overlaid on the input screen G10 (see Figure 5).

[0069] The UI control unit 5a executes the process of process S4 after executing the process of process S3.

[0070] In the following description, the first virtual keyboard G2a or the second virtual keyboard G2b displayed on the display device 3a by the processing of process S2 or process S3 will be referred to as the target virtual keyboard G2 (see Figures 4 and 5).

[0071] <Process S4> In step S4, the UI control unit 5a performs a process to detect operation input to the target virtual keyboard G2 or the external keyboard 8.

[0072] The UI control unit 5a executes the process in step S5 when it detects a virtual key input, which is an operation input to any of the multiple keys on the target virtual keyboard G2. Furthermore, the UI control unit 5a executes the process in step S6 when it detects a hardware key input, which is an operation input to any of the multiple keys on the external keyboard 8.

[0073] On the other hand, the UI control unit 5a executes the process of step S8 if no operation is detected on either the touch panel 3b or the external keyboard 8.

[0074] Furthermore, the UI control unit 5a also executes the process of step S8 when it detects an operation other than the virtual key input, which is an operation on the touch panel 3b.

[0075] <Process S5> In step S5, the UI control unit 5a determines that the operation input device 3x is in the disconnected state. Subsequently, the UI control unit 5a executes the process of step S7.

[0076] <Process S6> Meanwhile, in step S6, the UI control unit 5a determines that the operation input device 3x is in the connected state. Subsequently, the UI control unit 5a executes the process of step S7.

[0077] Specifically, the UI control unit 5a determines that the operation input device 3x is in the connected state when it detects an operation on any of the multiple hardware keys 8a (see step S6). On the other hand, the UI control unit 5a determines that the operation input device 3x is in the disconnected state when it detects an operation on any of the multiple normal virtual keys G3 or a specific virtual key G3x (see step S5).

[0078] <Process S7> In step S7, the UI control unit 5a executes input response processing. This input response processing is the processing that corresponds to the operation input detected in step S4.

[0079] For example, if the operation input detected in step S4 is the input of a character or symbol, the input response process is the process of adding the input character or symbol to the input frame G4.

[0080] If the operation input detected in step S4 is a newline input, the input processing is the process of adding a newline to the input frame G4.

[0081] If the operation input detected in step S4 is a confirmed input, the input corresponding process is a process that sets the string or sequence of symbols displayed in input frame G4 as the value of a predetermined parameter.

[0082] The UI control unit 5a executes the process of process S7, and then executes the process of process S8.

[0083] <Process S8> In step S8, the UI control unit 5a determines whether or not the erasure condition, which is the condition for erasing the target virtual keyboard G2, is met.

[0084] For example, the erasure condition includes a first erasure condition that the other operation detected in step S4 is a touch operation on an area of ​​the touch panel 3b other than the target virtual keyboard G2.

[0085] Furthermore, the erasure condition may also include a second erasure condition in which the other operation detected in step S4 is a swipe operation on the area of ​​the target virtual keyboard G2 on the touch panel 3b.

[0086] Furthermore, the erasure condition may also include a third erasure condition in which no operation is detected on either the touch panel 3b or the external keyboard 8 for a predetermined period of time.

[0087] For example, the elimination condition is the logical OR of the first elimination condition, the second elimination condition, and the third elimination condition.

[0088] The UI control unit 5a executes the process of step S9 when it determines that the erasure condition is not met. On the other hand, the UI control unit 5a executes the process of step S11 when it determines that the erasure condition is met.

[0089] <Process S9> In step S9, the UI control unit 5a selects the following process depending on whether the determination result of the state of the operation input device 3x has changed or not as a result of the determination process in step S5 or step S6.

[0090] The UI control unit 5a executes the process of step S10 if the result of determining the state of the operation input device 3x changes. On the other hand, the UI control unit 5a executes the processes from step S4 onwards if the result of determining the state of the operation input device 3x does not change.

[0091] <Process S10> In step S10, the UI control unit 5a executes a key display change process for a specific virtual key G3x.

[0092] The aforementioned key display change process is a process that changes the mark inscribed on a specific virtual key G3x from one of the virtual key identification mark G3a and the accompanying mark G3b to the other.

[0093] In other words, when the determination result of the state of the operation input device 3x changes from the disconnected state to the connected state, the UI control unit 5a changes the target virtual keyboard G2 to be displayed on the display device 3a from the first virtual keyboard G2a to the second virtual keyboard G2b.

[0094] On the other hand, if the determination result of the state of the operation input device 3x changes from the connected state to the disconnected state, the UI control unit 5a changes the target virtual keyboard G2 to be displayed on the display device 3a from the second virtual keyboard G2b to the first virtual keyboard G2a.

[0095] The UI control unit 5a executes the processes from process S4 onward after executing the process in process S10.

[0096] <Process S11> In step S11, the UI control unit 5a clears the display of the target virtual keyboard G2 and input frame G4 on the display device 3a.

[0097] The UI control unit 5a terminates the keyboard input control after executing the process in step S11.

[0098] In the operation input device 3x, the virtual key identification mark G3a, which is marked on the confirmation key of the first virtual keyboard G2a, is not marked on any of the multiple hardware keys 8a on the external keyboard 8. Even in such a case, the display of the second virtual keyboard G2b allows the operator to operate an alternative key for the confirmation key on the external keyboard 8 without confusion.

[0099] In the example shown in Figure 5, the alternative key for the confirmation key is the CONTROL key on the external keyboard 8.

[0100] [First variation] Next, a first modified example of the operation input device 3x will be described with reference to Figures 6 and 7.

[0101] In step S5 or step S6 of the keyboard input control, the UI control unit 5a in this modified example displays the virtual switching key G3y on the display device 3a along with the first virtual keyboard G2a or the second virtual keyboard G2b.

[0102] In the example shown in Figures 6 and 7, the UI control unit 5a causes the display device 3a to display either the first virtual keyboard G2a or the second virtual keyboard G2b, which includes the virtual switching key G3y.

[0103] Furthermore, the UI control unit 5a changes the determination state of the operation input device 3x when it detects an operation on the virtual switching key G3y. That is, each time the UI control unit 5a detects an operation on the virtual switching key G3y, it changes the determination result of the state of the operation input device 3x from one of the connected state and the disconnected state to the other.

[0104] The UI control unit 5a changes the target virtual keyboard G2 to be displayed on the display device 3a from one of the first virtual keyboard G2a and the second virtual keyboard G2b to the other each time the detection state of the operation input device 3x is changed.

[0105] By adopting this modified version, the operator can select, through their own actions, whether to display the first virtual keyboard G2a or the second virtual keyboard G2b on the display device 3a.

[0106] [Second variation] Next, a second modified example of the operation input device 3x will be described.

[0107] In this modified example, the UI control unit 5a can perform a key assignment setting process to set a key corresponding to a specific virtual key G3x from among the multiple hardware keys 8a of the external keyboard 8.

[0108] In the key assignment setting process, the UI control unit 5a sets a key corresponding to a specific virtual key G3x in response to an operation input to the target virtual keyboard G2 or external keyboard 8.

[0109] For example, in the key assignment setting process, the UI control unit 5a can set the SHIFT key on the external keyboard 8 as the key corresponding to the confirmation key on the target virtual keyboard G2.

[0110] By adopting this modified version, the degree of flexibility in setting the key corresponding to the specific virtual key G3x on the external keyboard 8 is increased.

[0111] [Third variation] Next, a third modified example of the operation input device 3x will be described.

[0112] In this modified example, the UI control unit 5a can execute a process to set the second mark G3d of a specific virtual key G3x in response to an operation input to the target virtual keyboard G2 or the external keyboard 8.

[0113] For example, the UI control unit 5a can set one or more characters that make up the second mark G3d and the color of those characters.

[0114] The adoption of this modified version increases the degree of flexibility in setting the second mark G3d.

[0115] [Fourth variation] Next, a fourth modified example of the operation input device 3x will be described.

[0116] In this modified example, the UI control unit 5a determines whether the operation input device 3x is in the connected state or the disconnected state by detecting the state of the connection interface 3c.

[0117] For example, if the connection interface 3c is the serial interface circuit such as USB, the UI control unit 5a determines whether or not the external keyboard 8 is connected by detecting the state of the serial interface circuit.

[0118] Even when this modified version is adopted, the same effect as when the operation input device 3x is adopted can be obtained.

[0119] [Notes on the invention] The following is an overview of the invention extracted from the above-described embodiments. Note that each configuration and processing function described below can be selected and combined as desired.

[0120] <Note 1> An operation input control method for controlling an operation input device equipped with a connection interface and display device to which an external keyboard can be connected, The processor determines whether the operation input device is in a disconnected state (without the external keyboard connected) or a connected state (with the external keyboard connected), When it is determined that the operation input device is in the disconnected state, the processor causes the display device to display a first virtual keyboard including a plurality of normal virtual keys, each marked with a virtual key identification mark representing a key type. When it is determined that the operation input device is in the connected state, the processor causes the display device to display a second virtual keyboard in which the virtual key identification marks on specific virtual keys that are part of the plurality of normal virtual keys on the first virtual keyboard are replaced with a combination of the first mark and the second mark. This includes detecting operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard, The first mark is the virtual key identification mark of the specific virtual key or a simplified version of the virtual key identification mark of the specific virtual key. An operation input control method wherein the second mark is a mark representing a key among a plurality of hardware keys included in the external keyboard that corresponds to the specific virtual key.

[0121] <Note 2> The operation input control method according to Appendix 1, wherein the processor determines that the operation input device is in the connected state when it detects an operation on any of the plurality of hardware keys, and determines that the operation input device is in the disconnected state when it detects an operation on any of the plurality of normal virtual keys or a specific virtual key.

[0122] <Note 3> The operation input control method according to Appendix 1 or Appendix 2, wherein the processor sets the second mark in response to the operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard.

[0123] <Note 4> The operation input control method according to any one of the appendices 1 to 3, wherein the processor sets a key among the plurality of hardware keys corresponding to the specific virtual key in response to the operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard.

[0124] <Note 5> The processor causes the virtual switching key to be displayed on the display device along with the first virtual keyboard or the second virtual keyboard, The operation input control method according to any one of the appendices 1 to 4, further comprising the processor changing the discrimination state of the operation input device when it detects an operation on the virtual switching key.

[0125] <Note 6> A connection interface that allows connection of an external keyboard, Display device and An operation input device comprising a processor that implements the operation input control method described in any one of the above appendices 1 to 5.

[0126] <Note 7> The operation input device described in Appendix 6 above, An image processing apparatus comprising one or more image processing units that perform image processing in accordance with an operation detected by the aforementioned operation input device. [Explanation of Symbols]

[0127] 3: User Interface Device 3a:Display device 3b: Touch panel 3c: Connection Interface 3x: Operation input device 5: Control device 8: External keyboard 8a: Hardware key 8b: Hardware key identification mark 10: Image processing device G1: Operation icon G10: Input screen G2: Target virtual keyboard G2a: First virtual keyboard G2b: Second virtual keyboard G3: Standard virtual key G3a: Virtual key identification mark G3b: Dual-use mark G3c: Mark 1 G3d: Mark 2 G3x: Specific virtual key G3y: Virtual Switching Key G4: Input field

Claims

1. An operation input control method for controlling an operation input device equipped with a connection interface and display device to which an external keyboard can be connected, The processor determines whether the operation input device is in a disconnected state (without the external keyboard connected) or a connected state (with the external keyboard connected), When it is determined that the operation input device is in the disconnected state, the processor causes the display device to display a first virtual keyboard including a plurality of normal virtual keys, each marked with a virtual key identification mark representing a key type. When it is determined that the operation input device is in the connected state, the processor causes the display device to display a second virtual keyboard in which the virtual key identification marks on specific virtual keys that are part of the plurality of normal virtual keys on the first virtual keyboard are replaced with a combination of the first mark and the second mark. This includes detecting operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard, The first mark is the virtual key identification mark of the specified virtual key or a simplified version of the virtual key identification mark of the specified virtual key. An operation input control method wherein the second mark is a mark representing a key among a plurality of hardware keys included in the external keyboard that corresponds to the specific virtual key.

2. The operation input control method according to claim 1, wherein the processor determines that the operation input device is in the connected state when it detects an operation on any of the plurality of hardware keys, and determines that the operation input device is in the disconnected state when it detects an operation on any of the plurality of normal virtual keys or the specific virtual key.

3. The operation input control method according to claim 1 or 2, wherein the processor sets the second mark in response to the operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard.

4. The operation input control method according to claim 1 or 2, wherein the processor sets a key among the plurality of hardware keys corresponding to the specific virtual key in response to the operation input to the first virtual keyboard, the second virtual keyboard, or the external keyboard.

5. The processor causes the virtual switching key to be displayed on the display device along with the first virtual keyboard or the second virtual keyboard, The operation input control method according to claim 1 or claim 2, further comprising the processor changing the discrimination state of the operation input device when it detects an operation on the virtual switching key.

6. A connection interface that allows connection of an external keyboard, Display device and An operation input device comprising a processor that implements the operation input control method described in claim 1 or claim 2.

7. The operation input device according to claim 6, An image processing apparatus comprising one or more image processing units that perform image processing in accordance with an operation detected by the aforementioned operation input device.

Images