Display device and image forming device

The display device uses hand gesture recognition to enlarge or reduce images through non-contact operations, addressing infection risks and improving user interface intuitiveness.

JP7589620B2Active Publication Date: 2024-11-26KYOCERA DOCUMENT SOLUTIONS INC
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2021055933
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-29
Publication Date
2024-11-26
Estimated Expiration
2041-03-29

AI Technical Summary

Technical Problem

Existing image enlargement technologies in display devices pose infection risks due to touch operations and lack intuitive non-contact operation methods.

Method used

A display device equipped with a detection system that recognizes a user's hand gestures, specifically a ring shape and eye position, to intuitively enlarge or reduce images through non-contact operations.

Benefits of technology

Enables intuitive and infection-free image scaling via sensuous non-contact gestures, enhancing user operability and accessibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589620000001
    Figure 0007589620000001
  • Figure 0007589620000002
    Figure 0007589620000002
  • Figure 0007589620000003
    Figure 0007589620000003
Patent Text Reader

Abstract

To provide a display apparatus with excellent operability which allows a displayed image to be enlarged and reduced with a sensuous non-contact operation.SOLUTION: When a ring shape 51 and a right eye 52 positioned in the ring shape 51 are detected based on a detection result of a gesture sensor 24 and a plurality of captured images generated by an imaging unit 25, a control unit 10 calculates a relative size obtained upon the detection as a reference size. When a change in relative size is detected based on the plurality of captured images and the relative size after the change exceeds the reference size, the control unit 10 controls a display unit 15 to enlarge a home screen 40 according to the amount of change of the relative size from the reference size. When a change in relative size is detected based on the plurality of captured images and the relative size after the change is smaller than the reference size, the control unit 10 controls the display unit 15 to reduce in size the home screen 40 according to the amount of change of the relative size from the reference size.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a display device and an image forming device, and more particularly to a technique for enlarging or reducing a displayed image. [Background technology]

[0002] There are known techniques for enlarging a displayed image. For example, Patent Document 1 discloses a technique for enlarging a part of a displayed image when an operator presses a specific operation switch on an operation panel and the size of the detected finger is larger than a predetermined value.

[0003] Furthermore, Patent Document 2 discloses a technology in which, when the dwell time of a fingertip detected by a proximity operation is equal to or longer than a predetermined threshold, an icon corresponding to the position of the fingertip is enlarged and displayed, but when the dwell time is less than the predetermined threshold, the enlarged display is not performed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-87380 [Patent Document 2] JP 2012-226691 A Summary of the Invention [Problem to be solved by the invention]

[0005] With the spread of the novel coronavirus, devices equipped with a display are also required to take measures against infection. The technology disclosed in Patent Document 1 requires a touch operation to enlarge a displayed image, which entails the risk of infection. The technology disclosed in Patent Document 2 allows the display image to be enlarged by a non-contact operation, but has the problem that the operation for enlarging the displayed image is difficult to understand intuitively.

[0006] The present invention has been made in consideration of the above circumstances, and has an object to provide a display device with excellent operability, which allows a displayed image to be enlarged or reduced by an intuitive non-contact operation. [Means for solving the problem]

[0007] A display device according to an aspect of the present invention includes a display unit, a detection unit that detects a user's gesture, and a control unit that causes a predetermined image to be displayed in a display area of ​​the display unit. When the control unit detects a predetermined ring shape and a user's eye located within the ring shape based on a detection result from the detection unit, the control unit calculates a relative size of the user's eye with respect to the size of the ring shape at the time of the detection as a reference size, and when the control unit detects that the relative size has changed based on the detection result from the detection unit and the relative size after the change exceeds the reference size, the control unit enlarges the predetermined image on the display unit in accordance with an amount of change in the relative size from the reference size, and when the control unit detects that the relative size has changed based on the detection result from the detection unit and the relative size after the change is less than the reference size, the control unit reduces the predetermined image on the display unit in accordance with an amount of change in the relative size from the reference size.

[0008] An image forming apparatus according to another aspect of the present invention includes the display device described above, and an image forming section that forms an image on a recording medium. Effect of the Invention

[0009] According to the present invention, a user can enlarge or reduce a displayed image by forming a ring shape with his / her hand and moving his / her eye closer or farther away from the ring shape, a non-contact operation similar to the action of looking through a hole. Therefore, a display device with excellent user operability can be provided. [Brief description of the drawings]

[0010] [Figure 1] 1 is a front cross-sectional view showing a structure of an image forming apparatus according to an embodiment of the present invention; [Diagram 2]1 is a block diagram showing an internal configuration of an image forming apparatus; [Diagram 3] 13 is a flowchart showing a zoom process. [Figure 4] FIG. 13 is a diagram illustrating an example of a home screen. [Diagram 5] FIG. 13 is a diagram illustrating an example of a user gesture. [Figure 6] FIG. 11 is a diagram showing another example of a user gesture. [Figure 7] FIG. 13 is a diagram showing a state when the home screen is enlarged. [Figure 8] FIG. 11 is a diagram showing another example of a user gesture. [Figure 9] FIG. 13 is a diagram showing a state when the home screen is reduced. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an image forming apparatus including a display device according to one embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a front cross-sectional view showing the structure of the image forming apparatus according to one embodiment of the present invention. Fig. 2 is a block diagram showing the internal configuration of the image forming apparatus.

[0012] 1 and 2, image forming apparatus 1 is a multifunction device having multiple functions such as a copy function, a transmission function, a printer function, and a facsimile function. A housing of image forming apparatus 1 contains multiple devices for realizing the various functions of image forming apparatus 1. For example, the housing contains an image reading unit 11, an image forming unit 12, a fixing unit 13, a paper feeding unit 14, etc.

[0013] The image forming apparatus 1 includes a control unit 100. The control unit 100 includes a processor, a random access memory (RAM), a read only memory (ROM), etc. The processor is, for example, a central processing unit (CPU), a micro processing unit (MPU), or an application specific integrated circuit (ASIC).

[0014] The control unit 100 functions as a control unit 10 by the processor executing a control program stored in the ROM or HDD 18. The control unit 10 is responsible for the overall control of the image forming apparatus 1. More specifically, the control unit 10 controls the operation of each part of the image forming apparatus 1 and communication with a PC (Personal Computer) 23 and the like connected via a network.

[0015] The control unit 10 also operates according to a zoom program described later, and when it detects a predetermined ring shape formed by the user's hand and the user's eye located within the ring shape, it calculates the relative size of the user's eye to the size of the ring shape at the time of detection as a reference size, and when it detects that the relative size has changed and the relative size after the change exceeds the reference size, it executes a zoom process to enlarge the operation screen on the display unit 15, and when it detects that the relative size has changed and the relative size after the change is less than the reference size, it executes a zoom process to reduce the operation screen on the display unit 15. Note that the control unit 10 may be configured by a logic circuit, regardless of the operation based on the control program.

[0016] The control unit 100 is electrically connected to the document transport section 6, the image reading section 11, the image forming section 12, the fixing section 13, the paper feeding section 14, the display section 15, the operation section 16, the transport section 17, the HDD 18, the image processing section 19, the image memory 20, the facsimile communication section 21, the communication section 22, the gesture sensor 24, the imaging section 25, and the human presence sensor 26. The display section 15, the gesture sensor 24, the imaging section 25, the human presence sensor 26, and the control unit 100 constitute the display device 2.

[0017] The image reading unit 11 is an ADF (Auto Document Feeder) that includes an original transport unit 6 that transports an original placed on an original table, and a scanner that optically reads an original transported by the original transport unit 6 or an original placed on a platen glass 7. The image reading unit 11 reads the original by illuminating the original with a light irradiation unit and receiving the reflected light with a CCD (Charge-Coupled Device) sensor, thereby generating image data indicative of the original image.

[0018] The image forming unit 12 includes a photoconductor drum, a charging device, an exposure device, a developing device, and a transfer device. The image forming unit 12 forms an image composed of a toner image on the recording paper P conveyed along the conveying path T by the conveying unit 17 based on the image data generated by the image reading unit 11 or the image data input via the communication unit 22.

[0019] The fixing unit 13 heats and pressurizes the recording paper P on which the toner image has been formed by the image forming unit 12, thereby fixing the toner image to the recording paper P. The recording paper P on which the toner image has been fixed by the fixing unit 13 is discharged to the discharge tray 8.

[0020] The paper feed unit 14 includes a manual feed tray and multiple paper feed cassettes. The paper feed unit 14 uses a pickup roller to pull out recording paper P stored in one of the multiple paper feed cassettes or recording paper placed on the manual feed tray one sheet at a time and feeds the paper to the transport path T.

[0021] The display unit 15 is a display device configured with a liquid crystal display, an organic EL (Organic Light-Emitting Diode) display, etc. The display unit 15 displays various screens and the like for each function that can be executed by the image forming apparatus 1 in a display area according to the control of the control unit 10.

[0022] The operation unit 16 is provided on the front side of the image forming apparatus 1. The operation unit 16 includes a plurality of hard keys such as a start key for instructing the start of execution of each process. The operation unit 16 also includes a touch panel arranged over the display area of ​​the display unit 15. A user inputs various information such as instructions for each function that can be executed by the image forming apparatus 1 via the operation unit 16.

[0023] The transport unit 17 includes a transport roller pair 17A, a discharge roller pair 17B, etc., and a transport motor connected to the transport roller pair 17A and the discharge roller pair 17B, etc. The control unit 10 drives the transport motor to rotate the transport roller pair 17A and the discharge roller pair 17B, etc., thereby transporting the recording paper P fed by the paper feed unit 14 along the transport path T toward the image forming unit 12 and the discharge tray 8.

[0024] The HDD 18 is a large-capacity storage device for storing various data such as image data generated by the image reading unit 11. The HDD 18 stores various control programs for implementing general operations of the image forming apparatus 1. The HDD 18 stores, as one of the various control programs, a zoom-in / zoom-out program for executing a zoom-in / zoom-out process according to one embodiment of the present invention.

[0025] The image processing unit 19 performs image processing as necessary on the image data generated by the image reading unit 11. The image memory 20 includes an area for temporarily storing the image data generated by the image reading unit 11. The facsimile communication unit 21 connects to a public line and transmits and receives image data via the public line.

[0026] The communication unit 22 includes a communication module such as a LAN (Local Area Network) board, etc. The image forming apparatus 1 performs data communication via the communication unit 22 with a PC 23 and the like that are connected via a network.

[0027] The gesture sensor 24 is provided in the operation unit 16. The gesture sensor 24 has a detection area in a direction perpendicular to the display area of ​​the display unit 15. As the gesture sensor, any general sensor that can detect the shape of the user's hand can be used, and for example, an infrared camera can be used.

[0028] The imaging unit 25 is a digital camera configured with an imaging element, a lens, etc. The imaging unit 25 is provided in the operation unit 16. The imaging unit 25 captures an image of the user in the detection area of ​​the gesture sensor 24 at predetermined time intervals, and generates a plurality of image data representing the captured images (hereinafter, simply referred to as "captured images").

[0029] The control unit 10 detects a gesture formed by the user's hand and the user's eye based on the detection result of the gesture sensor 24 and the imaging result of the imaging unit 25. The control unit 10, the gesture sensor 24, and the imaging unit 25 constitute a detection unit in the claims.

[0030] The human presence sensor 26 is provided on the front side of the image forming apparatus 1. The human presence sensor 26 detects people present in a predetermined area near the image forming apparatus 1. There are no particular limitations on the human presence sensor 26 as long as it is a commonly used sensor, but for example, an infrared sensor can be used.

[0031] In this embodiment, the control unit 10 sets, in accordance with a user's instruction, either a contact mode for inputting an instruction according to a touch operation on the hard keys or touch panel of the operation unit 16, or a non-contact mode for inputting an instruction according to a gesture detected by the gesture sensor 24. When the non-contact mode is set, for example, when the gesture sensor 24 detects a gesture of the user bringing a finger close to the operation screen displayed on the display unit 15, the control unit 10 accepts an instruction corresponding to a key image according to the position of the user's finger.

[0032] Each part of the image forming apparatus 1 is connected to a power source, and each part of the image forming apparatus 1 operates when power is supplied from the power source.

[0033] [Operation of image forming device 1] Fig. 3 is a flowchart showing the enlargement / reduction process. Fig. 4 is a diagram showing an example of a home screen. Fig. 5 is a diagram showing an example of a user's gesture. Fig. 6 is a diagram showing another example of a user's gesture. Fig. 7 is a diagram showing a state when the home screen 40 is enlarged. Fig. 8 is a diagram showing another example of a user's gesture. Fig. 9 is a diagram showing a state when the home screen 40 is reduced.

[0034] Hereinafter, the operation of the image forming apparatus 1 when performing the enlargement / reduction process will be described with reference to FIGS.

[0035] It is assumed that a user turns on the power of the image forming apparatus 1. When the power of the image forming apparatus 1 is turned on, the control unit 10 displays the home screen 40 shown in Fig. 4 as an operation screen in the display area of ​​the display unit 15. In this case, since the control unit 10 cannot display the entire home screen 40 in the display area, it displays only a part of the home screen 40 in the display area.

[0036] The control unit 10 causes the display unit 15 to display a plurality of key images for inputting different instructions on the home screen 40. Here, the control unit 10 causes the home screen 40 to display a key image 41 for inputting an instruction to select a copy function, a key image 42 for inputting an instruction to select a facsimile function, and the like.

[0037] In this situation, it is assumed that a user approaches the front of image forming apparatus 1. When human presence sensor 26 detects the user within a predetermined area, control unit 10 causes gesture sensor 24 to perform a detection operation for detecting the shape of the user's hand until human presence sensor 26 no longer detects the user, and also causes imaging unit 24 to capture an image of the user within the detection area at predetermined time intervals to generate a plurality of captured images.

[0038] In this situation, the user forms a ring shape 51 with the thumb and index finger of the right hand within the detection area of ​​the gesture sensor 24, as shown in Fig. 5. When the gesture sensor 24 detects the ring shape 51 as a predetermined ring shape, the control unit 10 starts the execution of the enlargement / reduction process shown in Fig. 3. When the execution of the enlargement / reduction process starts, the control unit 10 repeats the process of determining that the user's eyes have not been detected (NO in step S11) until it detects an image showing the user's eyes located within the image showing the detected ring shape, based on the multiple captured images that have been generated.

[0039] In this situation, it is assumed that the user has moved his / her face so that right eye 52 is positioned within ring shape 51. When control unit 10 analyzes multiple captured images and detects an image showing the user's eye (right eye 52 in this case) located within the image showing the detected ring shape (ring shape 51 in this case) (YES in step S11), control unit 10 calculates the relative size of the image showing the user's eye to the size of the image showing the ring shape at the time of detection (hereinafter simply referred to as "relative size") as a reference size (step S12).

[0040] The method of calculating the relative size is not particularly limited, but the control unit 10 calculates, for example, as the relative size, the ratio of the area of ​​the region surrounded by the contour of the image showing the right eye 52 to the area of ​​the region surrounded by the contour of the image showing the ring shape 51. Alternatively, the control unit 10 calculates, as the relative size, the ratio of the width of the image showing the right eye 52 (the distance from the inner corner to the outer corner of the eye) to the diameter of the image showing the ring shape 51.

[0041] After the process of step S12, the control unit 10 sequentially calculates the relative size based on the multiple captured images, and determines whether or not the calculated relative size has changed (step S13).

[0042] (1) When the relative size exceeds the reference size In this situation, the user moves right eye 52 even closer to ring shape 51 from the position shown in Fig. 5. As a result, as shown in Fig. 6, the relative size of the image showing right eye 52 with respect to the image showing ring shape 51 becomes larger than that shown in Fig. 5, and control unit 10 calculates a relative size indicating a value larger than the reference size based on the multiple captured images.

[0043] In this case, the control unit 10 determines that the relative size has changed (YES in step S13), and determines that the changed relative size exceeds the reference size (YES in step S14), and causes the display unit 15 to enlarge the home screen 40 according to the amount of change in the relative size from the reference size (step S15), as shown in Fig. 7. After the process of step S15, the control unit 10 returns to the process of step S13.

[0044] (2) When the relative size is less than the reference size Meanwhile, the user moves right eye 52 further away from ring shape 51 from the position shown in Fig. 5. As a result, as shown in Fig. 8, the relative size of the image showing right eye 52 with respect to the image showing ring shape 51 becomes smaller than that shown in Fig. 5, and control unit 10 calculates a relative size indicating a value smaller than the reference size based on the multiple captured images.

[0045] In this case, control unit 10 determines that the relative size has changed (YES in step S13) and that the changed relative size does not exceed the reference size, that is, the changed relative size is less than the reference size (NO in step S14), and causes display unit 15 to reduce home screen 40 according to the amount of change in the relative size from the reference size (step S16), as shown in Fig. 9. After the process of step S16, control unit 10 returns to the process of step S13.

[0046] In step S13, when the control unit 10 determines that the relative size has not changed (NO in step S13), it determines whether or not the user's eye located within the detected ring shape has been detected (step S17) in the same manner as in step S11. If the user maintains the gesture state, the control unit 10 determines that an image showing the user's eye (in this case, the right eye 52) located within the image showing the detected ring shape (in this case, the ring shape 51) has been detected based on the multiple captured images (YES in step S17), and returns to the process of step S13.

[0047] After enlarging or reducing the home screen 40, the user moves his / her face away from the ring shape 51 and returns his / her thumb and index finger to their original positions to release the ring shape 51. In this case, the control unit 10 is unable to calculate the relative size based on multiple captured images, so determines that the relative size has not changed (NO in step S13) and determines that the user's eyes located within the detected ring shape have not been detected (NO in step S17), and ends the enlargement / reduction process.

[0048] According to the above embodiment, when the control unit 10 detects the ring shape 51 and the right eye 52 located within the ring shape 51 based on the detection result of the gesture sensor 24 and the multiple captured images generated by the imaging unit 25, the control unit 10 calculates the relative size at the time of the detection as a reference size. When the control unit 10 detects that the relative size has changed based on the multiple captured images and the relative size after the change exceeds the reference size, the control unit 10 causes the display unit 15 to enlarge the home screen 40 in accordance with the amount of change in the relative size from the reference size. When the control unit 10 detects that the relative size has changed based on the multiple captured images and the relative size after the change is less than the reference size, the control unit 10 causes the display unit 15 to reduce the home screen 40 in accordance with the amount of change in the relative size from the reference size.

[0049] This allows the user to enlarge or reduce the home screen 40 by forming a ring shape with their hand and moving their eye closer or farther away from the ring shape, a non-contact operation (gesture) that is similar to the action of looking through a hole. Therefore, a display device with excellent user operability can be provided.

[0050] Furthermore, according to the above embodiment, the control unit 10 causes the display unit 15 to display the home screen 40 including a plurality of key images 41, 42, etc. for inputting different instructions. Therefore, the user can easily input instructions corresponding to the key images by enlarging the home screen 40 through the above-mentioned intuitive non-contact operation.

[0051] Furthermore, according to the above embodiment, the control unit 10 displays a part of the home screen 40 in the display area of ​​the display unit 15. Therefore, the user can easily check other parts that were not displayed in the display area by reducing the home screen 40 through the above-mentioned intuitive non-contact operation.

[0052] According to the embodiment, the image forming apparatus 1 includes the display device 2 and the image forming unit 12. This allows the user to enlarge or reduce the operation screen for inputting instructions for executing image formation, etc., by performing the above-mentioned intuitive non-contact operation. Therefore, an image forming apparatus with excellent user operability can be provided.

[0053] (Other variations) In the above embodiment, the home screen 40 is exemplified as the operation screen, but the present invention is not limited to such an embodiment. For example, the operation screen may be a setting screen for setting a copy function, a facsimile function, or the like.

[0054] In the above embodiment, the user forms the ring shape 51 with the thumb and index finger, but the present invention is not limited to such an embodiment. For example, the user may form the ring shape with the thumb and the other four fingers.

[0055] The present invention is not limited to the configuration of the above embodiment, and various modifications are possible. For example, in the above embodiment, the image forming apparatus 1 is a color multifunction machine, but this is merely an example, and other image forming apparatuses such as a monochrome multifunction machine, a copy machine, or a facsimile machine may be used.

[0056] In the above embodiment, the display device 2 provided in the image forming apparatus 1 is used as the display device, but this is merely an example, and other display devices such as a tablet terminal or a smartphone may also be used.

[0057] In the above embodiment, the image forming unit 12 etc. forms an image on the recording paper P, but the present invention is not limited to such an embodiment. The image forming unit 12 etc. may form an image on other recording media, not limited to recording paper. An example of the other recording media is an overhead projector (OHP) sheet.

[0058] Moreover, the configuration and processing of the embodiment described above with reference to FIGS. 1 to 9 are merely one embodiment of the present invention, and it is not intended that the present invention be limited to these configurations and processing. [Explanation of symbols]

[0059] 1. Image forming device 2 Display device 10 Control section 12 Image forming section 15 Display 24 Gesture Sensor 25 Imaging unit

Claims

1. A display unit; A detection unit for detecting a user gesture; A control unit that displays a predetermined image in a display area of ​​the display unit, The control unit is When a predetermined ring shape and the user's eye located within the ring shape are detected based on the detection result of the detection unit, a relative size of the user's eye with respect to the size of the ring shape at the time of the detection is calculated as a reference size; when it is detected that the relative size has changed based on a detection result of the detection unit, and when the relative size after the change exceeds the reference size, the display unit is caused to enlarge the predetermined image in accordance with an amount of change in the relative size from the reference size; A display device that, when it detects that the relative size has changed based on the detection result of the detection unit, and when the relative size after the change is less than the reference size, causes the display unit to reduce the predetermined image in accordance with the amount of change in the relative size from the reference size.

2. The display device according to claim 1 , wherein the control unit causes the display unit to display, as the predetermined image, an operation screen including a plurality of key images for inputting different instructions.

3. The display device according to claim 1 , wherein the control unit causes a part of the predetermined image to be displayed in a display area of ​​the display unit.

4. A display device according to any one of claims 1 to 3, An image forming apparatus comprising: an image forming unit that forms an image on a recording medium.

Citation Information

Patent Citations

  • Display method and device based on eye state, display equipment and storage medium

    CN111913561A

  • Operating body adaptive console panel device

    JP1996087380A

  • Terminal equipment and control program for terminal equipment

    JP2004348429A

  • Image forming apparatus

    JP2008250823A

  • Display control device and display control method

    JP2012226691A