Display device, display method, and display program

Abstract

To provide a display device that can assist users with impaired visual function in switching to an enlarged view of the operation screen without requiring complicated procedures. [Solution] The display device according to the present invention includes a user interface unit 20 that displays a predetermined operation screen M1 and accepts touch operations from the user on the operation screen M1, and a control unit 1 that stores the operation history of touch operations, determines whether the user is a user with impaired visual function based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired visual function, automatically switches the display mode of the operation screen M1 to an enlarged display or provides support for switching to the enlarged display.

Description

Technical Field

[0001] The present disclosure relates to a display device, a display method, and a display program.

Background Art

[0002] In recent years, display devices that use a touch panel as a user interface, such as mobile phones and multifunction printers, have been increasing. For example, refer to Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, this type of display device is configured with a touch panel provided on the display unit, and the user performs a predetermined operation by operating the touch panel on the operation buttons displayed on the display unit. Hereinafter, such an operation is referred to as a "touch operation". Also, hereinafter, when referring to a "touch operation", it means an operation of lightly touching the touch panel with a finger and immediately releasing it (also referred to as a tap operation).

[0005] By the way, in this type of display device, since the display size of the operation screen suitable for a person varies, setting means is provided that allows the user to freely enlarge or reduce the operation screen.

[0006] Typically, users will notice discomfort with the display size of this type of operation screen while actually using it, but stopping their work to review the display size settings is a cumbersome task for them. In particular, with shared devices (e.g., multifunction printers), the display size of the operation screen is not optimized for each user. Therefore, for some users, such screen settings are inconvenient, and they may be forced to operate the screen with poor visibility.

[0007] This invention has been made in view of the above-mentioned problems. Specifically, the present invention aims to provide a display device, a display method, and a display program that can assist users with impaired visual function in switching the operation screen to an enlarged display without complicated procedures. [Means for solving the problem]

[0008] The main present invention that solves the aforementioned problems is: A user interface unit that displays a predetermined operation screen and accepts user touch operations on the operation screen, A control unit that stores the operation history of the touch operations, determines whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and, if the user is a user with impaired vision, automatically switches the display mode of the operation screen to an enlarged display or provides assistance for switching to the enlarged display. It is a display device equipped with [a specific feature / feature].

[0009] Also, in other situations, The process includes displaying a predetermined operation screen and accepting user touch operations on the said operation screen, The process includes storing the operation history of the touch operations, determining whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired vision, automatically switching the display mode of the operation screen to an enlarged display or providing assistance for switching to the enlarged display. This is a display method that includes [something].

[0010] Also, in other situations, On the computer, The process includes displaying a predetermined operation screen and accepting user touch operations on the said operation screen, The process includes storing the operation history of the touch operations, determining whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired vision, automatically switching the display mode of the operation screen to an enlarged display or providing assistance for switching to the enlarged display. This is a display program that executes the following: [Effects of the Invention]

[0011] The display device according to the present invention makes it possible to assist users with impaired visual function in switching the operation screen to an enlarged display without requiring complicated procedures. [Brief explanation of the drawing]

[0012] [Figure 1] A schematic diagram showing an example of the overall configuration of an image forming apparatus. [Figure 2] A schematic diagram showing an example of the overall configuration of an image forming apparatus. [Figure 3] This figure shows an example of an operation screen displayed in the UI section. [Figure 4] This diagram shows how the display switching button is displayed as a pop-up on the operation screen using the display switching support function of the control unit. [Figure 5] Figure 3 shows an example of the operation screen that appears when the operation screen is enlarged. [Figure 6] This figure shows another example of the operation screen that appears when the operation screen in Figure 3 is enlarged. [Figure 7] Figure 3 shows an example of the operation screen that appears when the copy button is selected. [Figure 8] A diagram showing an example of operation history data. [Figure 9]Figure showing another example of operation history data [Figure 10] Figure showing an example of the operation flow (main flow) of the display switching support function by the control unit [Figure 11] Figure showing an example of the first subroutine processing of step S3 in the main flow of FIG. 10 [Figure 12] Figure showing an example of the second subroutine processing of step S3 in the main flow of FIG. 10 [Figure 13] Figure showing an example of the enlarged display switching process by the control unit [Figure 14] Figure showing an example of the operation flow of the display switching support function by the control unit according to Modification 1 [Figure 15] Figure showing an example of an image captured by the camera [Figure 16] Figure showing an example of the operation flow of the display switching support function by the control unit according to Modification 2 [Figure 17] Figure showing an example of the operation flow of the display switching support function by the control unit according to Modification 3 [Figure 18] Figure showing an example of user data

Mode for Carrying Out the Invention

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0014] <Overall Configuration of the Image Forming Apparatus> First, the configuration of the display device according to an embodiment of the present invention will be described. In the present embodiment, an aspect in which the display device of the present invention is applied to an image forming apparatus (hereinafter referred to as "image forming apparatus U") will be described. The image forming apparatus U is, for example, a multifunction device having a plurality of functions such as a copy function, a fax function, a printer function, and a scanner function.

[0015] FIGS. 1 and 2 are diagrams schematically showing an overall configuration example of the image forming apparatus U.

[0016] Image forming apparatus U is a color image forming apparatus that uses an intermediate transfer method utilizing electrophotographic process technology. Specifically, image forming apparatus U first transfers the toner images of each color, Y (yellow), M (magenta), C (cyan), and K (black), formed on the photoreceptor drum 413 to an intermediate transfer belt 421. Then, after superimposing the four toner images on the intermediate transfer belt 421, it secondarily transfers them to the paper S to form a toner image.

[0017] Furthermore, the image forming apparatus U has photoreceptor drums 413 corresponding to the four YMCK colors arranged in series in the direction of travel of the intermediate transfer belt 421. A tandem system is employed in which the toner images of each color are sequentially transferred to the intermediate transfer belt 421 in a single procedure.

[0018] The image forming apparatus U includes an image reading unit 10, a UI (User Interface) unit 20, an image processing unit 30, an image forming unit 40, a paper transport unit 50, a fixing unit 60, a communication unit 71, a storage unit 72, a camera 80, and a control unit 1, etc. The "display device" of the present invention is composed of the UI unit 20 and the control unit 1.

[0019] The control unit 1 includes a CPU 1a, ROM 1c, RAM 1b, etc. The control unit 1 has the CPU 1a read a program corresponding to the processing content from ROM 1c and load it into RAM 1b, and then works in cooperation with the loaded program to centrally control the operation of each block of the image forming apparatus U. At this time, various data stored in the memory unit 72 are referenced.

[0020] The storage unit 72 consists of a storage device such as a non-volatile semiconductor memory or a hard disk, and stores various processing programs and data related to various processes. For example, the storage unit 72 stores layout data D1 of the operation screen displayed on the display unit 21, and operation history data D2 of touch operations performed on the operation unit 22 (see Figures 8 and 9). In addition, the storage unit 72 also stores camera image data D3 (see Figure 15) obtained by capturing the user's face area with the camera 80, and user data D4 (see Figure 18) registered in association with an external terminal.

[0021] The control unit 1 transmits and receives various data to and from external devices (e.g., personal computers) connected to a communication network such as a LAN or WAN via the communication unit 71. For example, the control unit 1 receives image data transmitted from an external device and forms a toner image on the paper S based on this image data. The communication unit 71 is composed of a communication control card, such as a LAN card.

[0022] The image reading unit 10 is comprised of an automatic document feeder 11, also known as an ADF, and a document image scanning device 12, among other components.

[0023] The automatic document feeder 11 transports the documents D placed in the document tray using a transport mechanism and sends them to the document image scanner 12. The automatic document feeder 11 can continuously scan images (including both sides) of multiple documents D placed in the document tray at once.

[0024] The document image scanning device 12 optically scans a document transported from the automatic document feeder 11 onto the contact glass or a document placed on the contact glass. It then images the reflected light from the document onto the light-receiving surface of the CCD sensor 12a and reads the document image. The image reading unit 10 generates input image data based on the reading result by the document image scanning device 12. This input image data is then subjected to predetermined image processing by the image processing unit 30.

[0025] The UI unit 20 is composed of, for example, a liquid crystal display with a touch panel, and functions as a display unit 21 and an operation unit 22. The display unit 21 displays various operation screens (see Figures 3 to 7) according to display control signals input from the control unit 1. The operation unit 22 receives various input operations from the user and outputs operation signals to the control unit 1.

[0026] The operation unit 22 detects the touch position on the operation screen displayed by the display unit 21. The touch panel constituting the operation unit 22 may be any known configuration. The operation unit 22 is configured, for example, with a projected capacitive touch sensor. That is, multiple capacitive sensors are configured in a matrix on the display area of ​​the display unit 21 by X electrodes and Y electrodes arranged in a matrix. The operation unit 22 then uses these capacitive sensors to detect the change in capacitance due to capacitive coupling that occurs between these electrodes and the finger when a finger approaches, and based on this, detects the position where a touch operation was performed.

[0027] The image processing unit 30 includes circuits and other components that perform digital image processing on the input image data according to initial settings or user settings. For example, the image processing unit 30 applies correction processing to the input image data based on color correction data (LUT) stored in the storage unit 72. Based on this color correction data, the image processing unit 30 performs various correction processing on the input image data, such as gradation correction, color correction, and shading correction, as well as compression processing. The image forming unit 40 is then controlled based on the image data that has undergone these processing.

[0028] The image forming unit 40 includes image forming units 41Y, 41M, 41C, 41K, and an intermediate transfer unit 42, etc.

[0029] The image forming units 41Y, 41M, 41C, and 41K for the Y, M, C, and K components have similar configurations. For the sake of illustration and explanation, common components are indicated by the same reference numeral, and when distinguishing them, Y, M, C, or K is added to the reference numeral. In Figure 1, only the components of the image forming unit 41Y for the Y component are given reference numerals, while the components of the other image forming units 41M, 41C, and 41K are omitted.

[0030] The image forming unit 41 includes an exposure device 411, a developing device 412, a photoreceptor drum 413, a charging device 414, and a drum cleaning device 415, etc.

[0031] The photoreceptor drum 413 is a negatively charged organic photoreceptor, for example, made of an aluminum conductive cylindrical body on which an undercoat layer, a charge generation layer, and a charge transport layer are sequentially laminated. The charge generation layer is made of an organic semiconductor in which a charge generation material is dispersed in a resin binder, and generates a pair of positive and negative charges by exposure by the exposure apparatus 411. The charge transport layer is made of a hole transporting material dispersed in a resin binder, and transports the positive charge generated in the charge generation layer to the surface of the charge transport layer.

[0032] The control unit 1 controls the drive current supplied to the drive motor that rotates the photoreceptor drum 413, thereby rotating the photoreceptor drum 413 at a constant peripheral speed.

[0033] The charging device 414 uniformly charges the surface of the photoconductive photoreceptor drum 413 to a negative polarity. The exposure device 411, for example, is composed of a semiconductor laser and irradiates the photoreceptor drum 413 with laser light corresponding to the image of each color component. Positive charges are generated in the charge generation layer of the photoreceptor drum 413 and transported to the surface of the charge transport layer, thereby neutralizing the surface charge of the photoreceptor drum 413. Electrostatic latent images of each color component are formed on the surface of the photoreceptor drum 413 due to the potential difference with the surroundings.

[0034] The developing device 412 is, for example, a two-component developing device, which visualizes the electrostatic latent image by depositing toners of each color component onto the surface of the photoreceptor drum 413 to form a toner image.

[0035] The drum cleaning device 415 has drum cleaning blades and the like that are slid into contact with the surface of the photoreceptor drum 413. It removes any remaining toner after the primary transfer from the surface of the photoreceptor drum 413.

[0036] The intermediate transfer unit 42 includes an intermediate transfer belt 421 as an image carrier, a primary transfer roller 422, a plurality of support rollers 423, a secondary transfer roller 424, and a belt cleaning device 426, etc.

[0037] The intermediate transfer belt 421 is an endless belt and is stretched in a loop around a plurality of support rollers 423. At least one of the plurality of support rollers 423 is a drive roller, and the others are driven rollers. For example, it is preferable that the roller 423A, which is located downstream in the belt travel direction from the primary transfer roller 422 for component K, is the drive roller. This makes it easier to maintain a constant belt travel speed in the primary transfer section. As the drive roller 423A rotates, the intermediate transfer belt 421 travels at a constant speed in the direction of arrow A.

[0038] The primary transfer roller 422 is positioned on the inner circumferential side of the intermediate transfer belt 421, facing the photoreceptor drum 413 for each color component. By pressing the primary transfer roller 422 against the photoreceptor drum 413 with the intermediate transfer belt 421 in between, a primary transfer nip is formed for transferring the toner image from the photoreceptor drum 413 to the intermediate transfer belt 421.

[0039] The secondary transfer roller 424 is positioned on the outer circumferential surface side of the intermediate transfer belt 421, opposite the backup roller 423B which is located downstream of the drive roller 423A in the belt travel direction. By pressing the secondary transfer roller 424 against the backup roller 423B with the intermediate transfer belt 421 in between, a secondary transfer nip is formed for transferring the toner image from the intermediate transfer belt 421 to the paper S.

[0040] As the intermediate transfer belt 421 passes over the primary transfer nip, the toner image on the photoreceptor drum 413 is sequentially superimposed onto the intermediate transfer belt 421 and primary transferred. Specifically, by applying a primary transfer bias to the primary transfer roller 422 and applying a charge with the opposite polarity to the toner to the back side of the intermediate transfer belt 421, the toner image is electrostatically transferred to the intermediate transfer belt 421.

[0041] Subsequently, as the paper S passes through the secondary transfer nip, the toner image on the intermediate transfer belt 421 is transferred to the paper S. Specifically, a secondary transfer bias is applied to the secondary transfer roller 424, and a charge of the opposite polarity to the toner is applied to the back side of the paper S, thereby electrostatically transferring the toner image to the paper S. The paper S, on which the toner image has been transferred, is then transported toward the fuser unit 60.

[0042] The belt cleaning device 426 has a belt cleaning blade that slides against the surface of the intermediate transfer belt 421, and removes any remaining transfer toner on the surface of the intermediate transfer belt 421 after secondary transfer.

[0043] The fixing unit 60 includes an upper fixing unit 60A having a fixing surface side member positioned on the fixing surface side of the paper S, a lower fixing unit 60B having a back-side support member positioned on the back side of the paper S, and a heating source 60C, etc. When the back-side support member is pressed against the fixing surface side member, a fixing nip is formed that grips and transports the paper S.

[0044] The fuser unit 60 heats and pressurizes the paper S, which has been transported after the toner image has been secondarily transferred, with a fuser nip to fix the toner image to the paper S. The fuser unit 60 is arranged as a unit within the fuser unit F. The fuser unit F also has an air separation unit 60D that separates the paper S from the fuser surface side member by blowing air onto it.

[0045] The paper transport unit 50 includes a paper feeding unit 51, a paper discharge unit 52, and a transport path unit 53, etc. The three paper feeding tray units 51a to 51c that make up the paper feeding unit 51 contain paper S identified based on basis weight, size, etc., according to a preset type. The transport path unit 53 has multiple transport roller pairs, such as registration roller pairs 53a.

[0046] The paper sheets S stored in the paper feed tray units 51a to 51c are fed out one by one from the top and transported to the image forming unit 40 by the transport path unit 53. At this time, the registration roller unit, which is equipped with a pair of registration rollers 53a, corrects the tilt of the fed paper sheets S and adjusts the transport timing. Then, in the image forming unit 40, the toner image on the intermediate transfer belt 421 is transferred to one side of the paper sheets S all at once, and a fixing process is performed in the fixing unit 60. The image formed paper sheets S are then discharged from the machine by the paper discharge unit 52, which is equipped with a paper discharge roller 52a.

[0047] Camera 80 is, for example, a general visible light camera, and generates image data by performing AD conversion on the image signal generated by its own image sensor. Camera 80 generates moving images by, for example, continuously generating frame by frame. Camera 80 is installed for the purpose of photographing the user (especially the face area) using the image forming apparatus U (see Modification 1 described later). Camera 80 transmits the image data it has generated to the control unit 1.

[0048] <Detailed configuration of the control unit> The image forming apparatus U (control unit 1) has a display switching support function that allows the operation screen M1 to be switched to an enlarged display in accordance with the visual function of the user (hereinafter abbreviated as "user") using the image forming apparatus U.

[0049] Specifically, the control unit 1 determines whether a user is a user with impaired vision based on the operation history data D2, so that users with impaired vision can easily operate the operation screen M1. If the control unit 1 determines that the user is a user with impaired vision, it automatically switches the display mode of the operation screen M1 to magnified display or provides assistance in switching to magnified display.

[0050] Figures 3 to 7 show an example of screen transitions for the operation screen displayed in the UI unit 20.

[0051] Figure 3 shows an example of an operation screen M1 displayed on the UI unit 20. This operation screen M1 has, for example, 12 function selection buttons A1 for selecting a function to be executed from the various functions of the image forming apparatus U. In addition to the 12 function selection buttons A1, this operation screen M1 also has a home button A2 for moving to the home screen, a back button A3 for returning to the previous screen, etc. The layout information of the function selection buttons A1 etc. displayed on the operation screen M1 is stored in the storage unit 72 as layout data D1.

[0052] In this embodiment, the control unit 1 stores the operation information of the touch operation as operation history data D2 in the storage unit 72 each time the user performs a touch operation on the UI unit 20 (in this case, the operation screen M1).

[0053] Figure 8 shows an example of operation history data D2. Figure 9 shows another example of operation history data D2. Operation history data D2 stores, for example, the operation history of touch operations, associated with the time when the touch operation was performed, in chronological order.

[0054] The operation history of touch operations includes, for example, information related to the touch position on the operation screen M1 and the content of the operation. Information related to the touch position is represented, for example, in a two-dimensional coordinate system on the operation screen M1. Information related to the content of the operation is, for example, information indicating which icon (for example, function selection button A1) the touch operation was performed to select. If the touch operation is not an operation to select any icon (i.e., a touch operation on an area without icons), the operation content is stored as an invalid operation.

[0055] The operation history data D2 in Figure 8 shows a scenario in which two consecutive invalid operations (i.e., touch operations on an icon-free area where no function selection buttons A1 are placed) occurred within the operation screen M1. Furthermore, the operation history data D2 in Figure 9 shows a scenario in which, after the copy button was selected within the operation screen M1, the back button A13 was touched on the copy execution screen M2, and this operation was repeated.

[0056] Furthermore, the configuration shown in Figure 8 corresponds to the operation history when the user is identified as having impaired visual function in the determination process shown in Figure 11, which will be described later. Also, the configuration shown in Figure 9 corresponds to the operation history when the user is identified as having impaired visual function in the determination process shown in Figure 12, which will be described later.

[0057] As described above, the control unit 1 according to this embodiment determines whether the user is a user with impaired visual function based on the operation history data D2. If the user is a user with impaired visual function, the control unit 1 automatically switches the display mode of the operation screen M1 to magnified display or provides assistance for switching to magnified display.

[0058] Furthermore, the criteria for determining whether a user qualifies as a user with impaired visual function include, for example, whether an operation pattern of repeatedly tapping the same area on the operation screen M1 is detected from the operation history data D2 (details will be described later).

[0059] Figure 4 shows the display switching button A4 popping up on the operation screen M1 using the display switching support function of the control unit 1. In this embodiment, if the user is a user with impaired vision, the control unit 1 displays the display switching button A4 popping up on the operation screen M1 shown in Figure 3.

[0060] Figure 5 shows an example of the operation screen M1a (hereinafter also referred to as the "enlarged operation screen M1a") that is displayed when the operation screen M1 in Figure 3 is enlarged. Figure 6 shows another example of the enlarged operation screen M1a. In this embodiment, when the display switching button A4 is touched on the operation screen M1 in Figure 3, the operation screen M1 is switched to the enlarged operation screen M1a.

[0061] The enlarged operation screen M1a in Figures 5 and 6 has an enlargement OFF button A5 and a page switching button A6. When the user touches the enlargement OFF button A5, the control unit 1 ends the enlarged display of the operation screen M1 and returns to the normal-sized operation screen M1. Also, when the user touches the page switching button A6, the control unit 1 changes the list of function selection buttons A1 displayed on the enlarged operation screen M1a. For example, if the page switching button A6 is touched on the enlarged operation screen M1a in the state shown in Figure 5, the control unit 1 switches to the enlarged operation screen M1a shown in Figure 6.

[0062] Figure 7 shows an example of the operation screen M2 that appears when the copy button in the operation screen M1 in Figure 3 is selected.

[0063] The operation screen M2 in Figure 7 is equipped with multiple job selection buttons A11 for selecting the job to be copied from among several accumulated jobs. In addition to the job selection buttons A11, the operation screen M2 also displays a back button A13 to return to the previous screen (i.e., the main operation screen M1), and a print execution button A12, etc. Hereafter, the operation screen M2 will be referred to as the "copy execution screen M2" to distinguish it from the operation screen M1.

[0064] The following is an example of the operation flow of the control unit 1 according to this embodiment.

[0065] Figure 10 shows an example of the operation flow (main flow) of the display switching support function by the control unit 1.

[0066] Figure 11 shows an example of the first subroutine processing in step S3 of the main flow in Figure 10. Figure 12 shows an example of the second subroutine processing in step S3 of the main flow in Figure 10. Figures 10 to 12 only describe the operation related to the display switching support function when the user performs a touch operation on the operation screen M1. S1 to S4 in Figure 10, Sa1 to Sa5 in Figure 11, and Sb1 to Sb6 in Figure 12 represent the processes that the control unit 1 executes sequentially according to the computer program.

[0067] In S1, the control unit 1 first determines whether a touch operation has been performed on the UI unit 20. If a touch operation has been performed (S1: YES), the control unit 1 proceeds to S2. If no touch operation has been performed (S1: NO), the control unit 1 terminates the process shown in the flowchart of Figure 10 without performing any further processing.

[0068] In S2, the control unit 1 stores the operation information of the touch operation performed in S1 as operation history data D2 in the storage unit 72.

[0069] In S3, the control unit 1 refers to the operation history data D2 and performs a process (described later in Figures 11 and 12) to determine whether the user of the image forming apparatus U is a user with impaired visual function, based on the information relating to touch operations performed within a predetermined time. Here, the control unit 1 performs the first subroutine process shown in Figure 11 and the second subroutine process shown in Figure 12.

[0070] In S4, if the control unit 1 determines, based on the judgment result in S3, that the user is a user with impaired visual function, it displays a display switching button A4 as a pop-up on the operation screen M1, allowing the user to switch the display mode of the operation screen M1 to an enlarged display (see Figure 4).

[0071] In this embodiment, the S3 process determines whether the user of the image forming apparatus U is a user with impaired vision by determining whether one of the two operation patterns shown in Figures 11 and 12 is being performed. Generally, when a user feels uncomfortable with the display size, they often perform operation patterns such as repeatedly tapping a specific area to compensate for the difficulty in seeing. That is, if a user is performing such an operation pattern, it can be inferred that the visibility of the operation screen M1 is poor for that user. The determination process shown in Figures 11 and 12 is set based on this observation.

[0072] The visual function determination process shown in Figure 11 determines whether invalid touch operations on areas without icons (in this case, areas where the function selection button A1 is not placed) within the operation screen M1 have been performed consecutively and in close proximity to each other in a predetermined number of consecutive operation histories performed within a predetermined time period.

[0073] Furthermore, the time range for evaluation within the operation history can be set to, for example, 10 seconds to 60 seconds. Also, the number of consecutive operations to be evaluated can be set to, for example, 2 to 4 times.

[0074] In Sa1, the control unit 1 refers to the operation history data D2 to determine whether the previous touch operation (i.e., the touch operation detected in S1) was an invalid touch operation. If the previous touch operation was an invalid touch operation (Sa1: YES), the control unit 1 proceeds to Sa2. On the other hand, if the previous touch operation was not an invalid touch operation (Sa1: NO), the control unit 1 proceeds to Sa5.

[0075] In Sa2, the control unit 1 refers to the operation history data D2 to determine whether the operation preceding the most recent invalid touch operation was also an invalid touch operation. If the preceding operation was also an invalid touch operation (Sa2: YES), the control unit 1 proceeds to Sa3. On the other hand, if the preceding operation was not an invalid touch operation (Sa2: NO), the control unit 1 proceeds to Sa5.

[0076] In Sa3, the control unit 1 refers to the operation history data D2 and determines whether the touch position of the invalid touch operation referenced in Sa1 is near the touch position of the invalid touch operation referenced in Sa2. If both are near each other (Sa3: YES), the control unit 1 proceeds to Sa4. On the other hand, if they are not near each other (Sa3: NO), the control unit 1 proceeds to Sa5.

[0077] In this case, the distance between the touch positions of two touch operations is calculated based on the touch positions stored in the operation history data D2, for example, and is expressed as a distance in a two-dimensional coordinate system on the operation screen M1. In Sa2, if this distance is below a predetermined threshold, it is determined that the touch positions of the two touch operations are close together.

[0078] In Sa4, the control unit 1 determines that the user is a user with impaired vision, sets a flag in the memory unit 72 to indicate this, and returns to the flow shown in Figure 10. In other words, in this case, the control unit 1 displays a pop-up of the display switching button A4 in S4 (Figure 10) to enable the display mode of the operation screen M1 to be switched to an enlarged display.

[0079] In Sa5, the control unit 1 determines that the user does not qualify as a user with impaired visual function, sets a flag in the storage unit 72 to indicate this, and returns to the flow shown in Figure 10. In other words, in this case, the control unit 1 terminates the flow shown in Figure 10 without performing any special processing.

[0080] The visual function determination process shown in Figure 12 is a process that determines whether the following operations (1) and (2) have been repeated a predetermined number of times consecutively in a history of multiple consecutive operations performed within a predetermined time period. (1) A second touch operation: Tap the area where any function selection button A1 is located on the operation screen M1. (2) A third touch operation: tapping the back button A13 to return to the previous screen from the state after the screen transition following the second touch operation.

[0081] This determination process repeatedly checks whether the same function selection button A1 is being touched incorrectly. For example, this process may involve a scenario where the copy button A1 is selected on the operation screen M1 shown in Figure 3, followed by the selection of the back button A13 on the operation screen M2 shown in Figure 7, and this process is repeated.

[0082] Furthermore, the time range for evaluation within the operation history can be set to, for example, 10 seconds to 60 seconds. Also, the number of repetitions of the operation being evaluated can be set to, for example, 2 to 4 times.

[0083] In Sb1, the control unit 1 refers to the operation history data D2 and determines whether the previous touch operation (i.e., the touch operation detected in S1) was a touch operation of the back button A3 (i.e., a selection operation). If the previous touch operation was a touch operation of the back button A3 (Sb1: YES), the control unit 1 proceeds to Sb2. On the other hand, if the previous touch operation was not a touch operation of the back button A3 (Sb1: NO), the control unit 1 proceeds to Sb6.

[0084] In Sb2, the control unit 1 refers to the operation history data D2 and determines whether there was a touch operation (i.e., a selection operation) of any function selection button A1 on the operation screen M1 before the selection operation of the back button A3. If there was a touch operation of a function selection button A1 before the touch operation of the back button A3 (Sb2: YES), the control unit 1 proceeds to Sb3. On the other hand, if there was no touch operation of a function selection button A1 before the touch operation of the back button A3 (Sb2: NO), the control unit 1 proceeds to Sb6.

[0085] Furthermore, in Sb1 and Sb2, the control unit 1 may perform these determination processes based on the touch position of the touch operation stored in the operation history data D2, or it may perform these determination processes based on the icon information selected by the touch operation.

[0086] In Sb3, the control unit 1 refers to the operation history data D2 and determines whether the back button A3 was touched before the function selection button A1 was touched. If the back button A3 was touched before the function selection button A1 was touched (Sb3: YES), the control unit 1 proceeds to Sb4. On the other hand, if the back button A3 was not touched before the function selection button A1 was touched (Sb3: NO), the control unit 1 proceeds to Sb6.

[0087] In Sb4, the control unit 1 refers to the operation history data D2 and determines whether there was a touch operation to the same function selection button A1 as in Sb2 on the operation screen M1 before the selection operation of the back button A3. If there was a touch operation to the same function selection button A1 as in Sb2 before the touch operation of the back button A3 (Sb4: YES), the control unit 1 proceeds to Sb5. On the other hand, if there was no touch operation to the same function selection button A1 as in Sb2 before the touch operation of the back button A3 (Sb4: NO), the control unit 1 proceeds to Sb6.

[0088] In Sb5, the control unit 1 determines that the user is a user with impaired vision, sets a flag in the memory unit 72 to indicate this, and returns to the flow shown in Figure 10. In other words, in this case, the control unit 1 displays a pop-up of the display switching button A4 in S4 (Figure 10) to enable the user to switch the display mode of the operation screen M1 to an enlarged display.

[0089] In Sb6, the control unit 1 determines that the user does not qualify as a user with impaired vision, sets a flag in the storage unit 72 to indicate this, and returns to the flow shown in Figure 10. In other words, in this case, the control unit 1 terminates the flow shown in Figure 10 without performing any special processing.

[0090] Through the above process, if the user is a user with impaired vision, the control unit 1 provides support to enable the user to switch the display mode of the operation screen M1 to an enlarged display. Here, the control unit 1 displays a "display switching button A4" as a pop-up on the operation screen M1 to enable the user to switch the display mode of the operation screen M1 to an enlarged display (see Figure 4). Then, the control unit 1 makes the user's display switching operation possible on the operation screen M1 via the display switching button A4.

[0091] Here, function selection button A1 is shown as an example of an icon for operation, but when determining the user's visual function, operations on other types of icons (for example, volume control icons) may also be referenced.

[0092] Figure 13 shows an example of the process by the control unit 1 to switch to an enlarged display.

[0093] In S11, the control unit 1 accepts the user's display switching operation on the operation screen M1 via the display switching button A4. If the display switching button A4 is touched (S11: YES), the control unit 1 proceeds to S12. If the display switching button A4 is not touched (S11: NO), the control unit 1 terminates the process shown in the flowchart of Figure 13 without performing any further processing.

[0094] In S12, the control unit 1 sets the display items in the enlarged operation screen M1a that will be displayed immediately after switching, based on the operation history data D2.

[0095] In S13, the control unit 1 displays the enlarged operation screen M1a on the display unit 21 according to the display items set in S12 (see Figures 5 and 6).

[0096] Here, we will describe in detail the process of setting the display items in the enlarged operation screen M1a of S12.

[0097] Users identified as having impaired visual function are presumed to touch a location slightly off-center from their desired touch position on the operation screen M1 due to poor visibility of the screen. Therefore, when switching from the normal operation screen M1 to the enlarged operation screen M1a, it is desirable to enlarge the area of ​​the operation screen M1 that the user wants to see.

[0098] From this perspective, the control unit 1 sets the display items in the enlarged operation screen M1a to be displayed immediately after switching, based on, for example, the touch position of the touch operation indicated by the previous operation history data D2 and the layout data D1 of the operation screen M1.

[0099] For example, if the touch position of the previous operation is near the "FAX button" on the operation screen M1, the control unit 1 displays the "FAX button," "Copy button," and "Scan button" on the enlarged operation screen M1a immediately after the display switch (see Figure 5). That is, at this time, the control unit 1 extracts the function selection button A1 that is close to the "FAX button" from the layout data D1 of the operation screen M1 and sets it as a display item on the enlarged operation screen M1a immediately after the display switch.

[0100] On the other hand, for example, if the touch position of the previous operation is near the "language button" on the operation screen M1, the control unit 1 displays the "language button," "universal button," and "counter button" on the enlarged operation screen M1a immediately after the display switch (see Figure 6). That is, at this time, the control unit 1 extracts the function selection button A1 that is close to the "language button" from the layout data D1 of the operation screen M1 and sets it as a display item on the enlarged operation screen M1a immediately after the display switch.

[0101] In this case, the control unit 1 may use, in place of / also in conjunction with / the touch position of the touch operation indicated by the previous operation history data D2, the content of the touch operation indicated by the previous operation history data D2 (i.e., the type of function selection button A1 selected immediately before).

[0102] Through the above processing, the control unit 1 provides support to enable users with impaired visual function to easily switch the operation screen M1 to an enlarged display.

[0103] Furthermore, it is desirable for the control unit 1 to return to the normal-sized operation screen M1 after switching to the enlarged display and the user has finished using the image forming apparatus U. In other words, it is desirable for the control unit 1 to switch the enlarged display of the operation screen M1 back to the normal size display when the execution of a job (for example, executing a print job on the copy execution screen M2) is completed by touch operation. This is because the image forming apparatus U is shared by multiple users, and it is desirable to return to the normal-sized display, which is more convenient for the next user, after the current user has completed their desired job.

[0104] [effect] As described above, in this embodiment, A user interface unit that displays a predetermined operation screen and accepts user touch operations on the operation screen, A control unit records the operation history data of the touch operation, determines whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired vision, automatically switches the display mode of the operation screen to an enlarged display or provides support for switching to the enlarged display. A display device equipped with [a specific feature] was disclosed.

[0105] According to the display device of this embodiment, it is possible to assist users with impaired visual function in switching the operation screen to an enlarged display without requiring complicated procedures.

[0106] (Variation 1) This modified version differs from the above embodiment in that the control unit 1 determines whether the user is a user with impaired visual function based on camera image data D3 in addition to operation history data D2. The camera image data D3 is image data obtained by capturing the face area of ​​the user using the image forming apparatus U with the camera 80 (the same applies hereinafter).

[0107] As described above, the image forming apparatus U relating to this disclosure assists users with impaired vision in easily switching the operation screen M1 to a magnified display. However, unnecessarily switching the operation screen M1 to a magnified display automatically, or displaying a pop-up of the display switching button A4 that causes a temporary pause in user operation, could conversely result in a user interface that is difficult to use.

[0108] Therefore, in the image forming apparatus U according to this modified version, in order to prevent misjudgment, in addition to the operation history data D2, the estimated age of the user being above a certain level is set as an AND condition for determining whether the user is a user with impaired visual function. It is generally known that in humans, the ability to focus visually deteriorates from around the age of 40, making it difficult to distinguish small characters. In this modified version, this universal phenomenon is used to prevent misjudgment.

[0109] Specifically, the control unit 1 in this modified version performs image recognition processing on the camera image data D3 obtained by the camera 80 to estimate the age of the user using the image forming apparatus U. Then, it sets the estimated age being above a certain level (for example, 40 years or older) as an AND condition for determining whether the user is a user with impaired visual function.

[0110] Figure 14 shows an example of the operation flow of the display switching support function by the control unit 1 according to this modified example.

[0111] Figure 15 shows an example of an image captured by camera 80 (i.e., camera image data D3). Camera 80 is positioned near the display unit 21, for example, as shown in Figure 1, so that it can capture the face area of ​​the user using the image forming apparatus U. That is, as shown in Figure 15, the image captured by camera 80 shows the face area of ​​user p1 operating the operation screen M1.

[0112] The operation flow in Figure 14 differs from the operation flow in Figure 10 only in that the step SX related to age determination is performed between S3 and S4 in Figure 10.

[0113] In step SX, the control unit 1 determines whether the estimated age of the user identified by image processing of the camera image data D3 is above a predetermined level. It should be noted that the method for estimating the user's age from the camera image is possible using conventionally known image recognition processing. For example, the control unit 1 performs the process of detecting the user's face region image from the camera image, and then uses a trained classifier model to extract the features of the user's facial morphology from the face region image and estimate the user's age. This determines whether the user's age is above a predetermined level.

[0114] Generally, skin texture refers to factors such as roughness and pore size, while wrinkles are lines and grooves that appear around the eyes, mouth, and forehead, and tend to deepen and increase in number with age. In other words, in a judgment process using a trained classifier model, if the depth of specific wrinkles or changes in skin texture exceed a predetermined threshold, the facial image will be judged to be of a certain age or older.

[0115] Then, in step SX, if the estimated age of the user is below a predetermined level (SX:NO), the control unit 1 terminates the flow shown in Figure 14 without performing any further processing.

[0116] As described above, the image forming apparatus U according to this modified version makes it possible to more accurately determine whether or not a user of the image forming apparatus U is a user with impaired visual function. Therefore, the image forming apparatus U according to this modified version is useful in that it can suppress unnecessary display switching, etc.

[0117] (Modification 2) In this modified version, similar to Modified Version 1, in order to prevent misjudgment, the control unit 1 determines whether or not a user of the image forming apparatus U is a user with impaired visual function, based on camera image data D3 in addition to operation history data D2.

[0118] In the modified control unit 1, the physical distance between the user's face area and the operation screen M1 is set as an additional condition for determining whether the user is a user with impaired vision, such that the physical distance between the user's face area and the operation screen M1 is less than or equal to a predetermined distance. That is, the control unit 1 determines whether the user is nearsighted based on whether the physical distance between the user's face area and the operation screen M1 is less than or equal to a predetermined distance.

[0119] Figure 16 shows an example of the operation flow of the display switching support function by the control unit 1 according to this modified example.

[0120] In the modified image forming apparatus U, to prevent misjudgment, the user's myopia is set as an AND condition for determining whether the user is a user with impaired visual function. The operation flow in Figure 16 differs from the operation flow in Figure 10 only in that step SY related to myopia determination is performed between S3 and S4 in Figure 10.

[0121] In step SY, the control unit 1 estimates the physical distance between the user's face region and the operation screen M1 by performing image processing on the camera image data D3. For example, the control unit 1 can detect the user's face region image from the camera image and estimate the physical distance between the user's face region and the operation screen M1 based on the size of the face region image.

[0122] Then, in step SY, if the user's physical distance is less than or equal to a predetermined distance (SY:NO), the control unit 1 terminates the flow shown in Figure 16 without performing any further processing.

[0123] As described above, the image forming apparatus U according to this modified version makes it possible to more accurately determine whether or not a user of the image forming apparatus U is a user with impaired visual function. Therefore, the image forming apparatus U according to this modified version is useful in that it can suppress unnecessary display switching, etc.

[0124] (Variation 3) In this modified version, similar to Modified Version 1, in order to prevent misjudgment, the control unit 1 uses other information in addition to the operation history data D2 to determine whether or not the user of the image forming apparatus U is a user with impaired visual function.

[0125] However, the control unit 1 in this modified example uses user data D4 registered in association with the external terminal that issued the job execution command to the image forming apparatus U to estimate the age of the user using the image forming apparatus U, and uses the fact that the estimated age is above a predetermined level as an additional condition.

[0126] Figure 17 shows an example of the operation flow of the display switching support function by the control unit 1 according to this modified example.

[0127] Figure 18 shows an example of user data D4. Note that user data D4 can include various types of information besides age information itself that can be used to estimate a user's age. For example, Figure 18 shows an employee number registered in conjunction with an external terminal. Since employee numbers are generally associated with the date of joining the company, it is often possible to estimate age from the employee number. This modified example adopts a configuration that estimates the user's age from the employee number, based on this premise.

[0128] The operation flow in Figure 17 differs from the operation flow in Figure 10 only in that step SZ, which is related to age determination, is performed between steps S3 and S4 in Figure 10.

[0129] In step SZ, the control unit 1 estimates the age of the user using the image forming apparatus U based on the user data D4, and determines whether the user's age is above a predetermined level.

[0130] Then, in step SZ, if the estimated age of the user is below a predetermined level (SZ:NO), the control unit 1 terminates the flow shown in Figure 17 without performing any further processing.

[0131] As described above, the image forming apparatus U according to this modified version makes it possible to more accurately determine whether or not a user of the image forming apparatus U is a user with impaired visual function. Therefore, the image forming apparatus U according to this modified version is useful in that it can suppress unnecessary display switching, etc.

[0132] (Other embodiments) The present invention is not limited to the above embodiments and can be applied to various modified forms.

[0133] The above embodiment shows an application of the display device according to the present invention to an image forming apparatus. However, the display device of the present invention can be applied to various devices other than image forming apparatuses, such as mobile phones and personal computers.

[0134] Furthermore, in the above embodiment, the control unit 1 displays a display switching button A4 as a pop-up on the operation screen M1 when the user of the image forming apparatus U is a user with impaired vision. However, instead of this configuration, the control unit 1 may automatically switch the operation screen M1 to an enlarged display when it determines that the user is a user with impaired vision.

[0135] Furthermore, in the above embodiment, when the display switching button A4 is pressed and the operation screen M1 is switched to an enlarged display, the control unit 1 is shown to enlarge only the operation screen M1. However, instead of this embodiment, the control unit 1 may also enlarge other operation screens (for example, the copy execution screen M2) after switching the operation screen M1 to an enlarged display.

[0136] Although specific examples of the present invention have been described in detail above, these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples illustrated above. [Industrial applicability]

[0137] The display device according to the present invention makes it possible to assist users with impaired visual function in switching the operation screen to an enlarged display without requiring complicated procedures. [Explanation of symbols]

[0138] U Image Forming Apparatus 1 Control Unit 10 Image reading unit 11. Automatic document feeder 12. Document Image Scanning Device 20 UI section 21 Display section 22 Control section 30 Image Processing Unit 40 Image forming unit 50 Paper transport section 60 Fixing section 71 Communications Department 72 Memory section 80 Cameras D1 Layout Data D2 Operation History Data D3 camera image data D4 User Data

Claims

1. A user interface unit that displays a predetermined operation screen and accepts user touch operations on the operation screen, A control unit that stores the operation history of the touch operations, determines whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and, if the user is a user with impaired vision, automatically switches the display mode of the operation screen to an enlarged display or provides assistance for switching to the enlarged display. A display device equipped with the following features.

2. The control unit determines that the user is a user with impaired vision if, in a predetermined number of consecutive operation histories performed within a predetermined time period, invalid first touch operations on areas without icons on the operation screen are performed consecutively and in close proximity to each other. The display device according to claim 1.

3. The control unit determines that the user is a user with impaired vision if, within a predetermined time, a second touch operation on the icon placement area on the operation screen and a third touch operation on the back button to return to the previous screen from the state obtained by the second touch operation are repeated a predetermined number of times in a series of operations. The display device according to claim 1.

4. When the control unit switches the display mode of the operation screen to the enlarged display, it sets the display items in the enlarged operation screen to be displayed immediately after the switch, based on the operation history data and the layout data of the operation screen. The display device according to claim 1.

5. Applied to image forming apparatus, The aforementioned operation screen is an operation screen for selecting a job to be executed by the image forming apparatus through the touch operation on the function selection button. When the execution of the job is completed by the touch operation, the control unit switches the enlarged display of the operation screen to the normal size display. The display device according to claim 1.

6. The control unit determines whether the user is a user with impaired vision based on the operation history of multiple consecutive operations within a predetermined time period, as well as camera image data obtained by capturing the user's face. The display device according to claim 1.

7. The control unit sets the estimated age of the user, identified by image processing of the camera image data, as being above a predetermined level as an additional condition when determining that the user is a user with impaired vision. The display device according to claim 6.

8. The image processing applied to the camera image data is as follows: A process for detecting the user's face region image from the camera image data, The process includes: extracting the facial features of the user from the facial region image using a pre-trained classifier model and estimating the user's age. The display device according to claim 7.

9. The control unit sets the physical distance between the user's face region, detected by image processing of the camera image data, and the operation screen as an additional condition for determining whether the user is a user with impaired vision. The display device according to claim 6.

10. Applied to image forming apparatus, The control unit determines whether the user is a user with impaired vision based on the operation history of multiple consecutive operations within a predetermined time period, as well as user information registered in association with the external terminal that issued the job execution command to the image forming apparatus. The display device according to claim 1.

11. The process includes displaying a predetermined operation screen and accepting user touch operations on the said operation screen, The process includes storing the operation history of the touch operations, determining whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired vision, automatically switching the display mode of the operation screen to an enlarged display or providing assistance for switching to the enlarged display. Display methods that include this.

12. On the computer, The process includes displaying a predetermined operation screen and accepting user touch operations on the said operation screen, The process includes storing the operation history of the touch operations, determining whether the user is a user with impaired vision based on a series of consecutive operations performed within a predetermined time, and if the user is a user with impaired vision, automatically switching the display mode of the operation screen to an enlarged display or providing assistance for switching to the enlarged display. A display program that executes the command.

Images