Non-contact input device

The non-contact input device uses a camera to process movements for input operations, addressing the limitations of conventional touch-based devices by allowing individuals to communicate and select items without physical contact, improving accessibility for those with mobility issues.

JP7595532B2Active Publication Date: 2024-12-06MITSUBISHI ELECTRIC ENG CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2021105837
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-06-25
Publication Date
2024-12-06
Estimated Expiration
2041-06-25

AI Technical Summary

Technical Problem

Conventional input devices for communicating using fingers or touch panels are limited by the need for direct contact, making it difficult for individuals with mobility issues or in situations where physical proximity is challenging, such as operating a vending machine or communicating from a bed.

Method used

A non-contact input device utilizing a camera to capture and process movements, allowing for input operations through image processing, enabling selection and communication without physical contact by interpreting hand or eye movements, foot movements, or simple gestures.

Benefits of technology

Enables non-contact input operations over a wide range, facilitating communication and selection for individuals with mobility limitations or in constrained spaces, enhancing usability and accessibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595532000001
    Figure 0007595532000001
  • Figure 0007595532000002
    Figure 0007595532000002
  • Figure 0007595532000003
    Figure 0007595532000003
Patent Text Reader

Abstract

To provide a non-contact input device capable of performing non-contact input operations based on a simple action of an intention-expressing person.SOLUTION: A non-contact input device according to the present disclosure includes an imaging device that captures the movement of an intention-expressing person as a moving image; a display unit that switches any one of a plurality of display screens for specifying the content of the intention expressed by the intention-expressing person; and a control unit that image-processes the moving image of the intention-expressing person viewing contents of the switched display to extract a predetermined action of the intention-expressing person, performs display switching processing in accordance with a result of the extraction to switch display contents of the display screen to be displayed on the display screen, performs confirmation processing to confirm the intention to select the current display contents, and specifies the contents of the intention that the intention-expressing person wants to convey.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a non-contact input device that enables a person to input a selection without touching a display unit. [Background technology]

[0002] Conventional input devices for communicating using fingers mainly have the following structure. A desired input operation is performed by touching a switch with a finger to turn an electrical contact on and off. A touch panel that uses an optical sensor to detect where a finger touches the screen and perform the desired input operation. - A device that uses a non-contact optical sensor or touch panel to perform desired input operations.

[0003] As a specific example, there is a conventional technique for sequentially detecting the position of a fingertip inserted into a finger input space in which the finger can be moved (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-22673 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional techniques have the following problems. Conventional input devices that use a finger or the like to communicate include the above-mentioned switches and touch panels, but the distance between the operator and the input device is limited.

[0006] For example, when considering a situation where a person wishes to communicate non-contact input using a vending machine, it may be difficult to move a finger to a desired button. Also, when considering a situation where a person requiring care lying in bed wishes to communicate non-contact input, it may be difficult to move a finger to a specific area. Therefore, there is a demand for a non-contact input device that enables non-contact input operation based on a simple action by the operator who is the person expressing his / her intention.

[0007] The present disclosure has been made to solve such problems, and has an object to provide a non-contact input device that enables non-contact input operation based on a simple action by a person expressing his / her intention. [Means for solving the problem]

[0008] The non-contact input device according to the present disclosure includes an imaging device that captures the movement of a person expressing his / her intention as a moving image, a display unit that switches between and displays one of a plurality of display screens in order to identify the content of the person expressing his / her intention, and a control unit that executes a display switching process that extracts a predetermined action by the person expressing his / her intention by image processing the moving image of the person viewing the switched and displayed display content, and switches the display content of the display screen to be displayed on the display unit in accordance with the extraction result, and a confirmation process that confirms the intention to select the current display content, thereby identifying the content of the person expressing his / her intention to convey. The imaging device and the display unit are arranged separately, the imaging device is installed at a position where it can image the movement of the person expressing his / her intention, the display unit is installed at a position where the person expressing his / her intention can be seen, and the control unit can use the movement of the eyes or the movement of the feet as well as the movement of the hands or fingers by the person expressing his / her intention as the predetermined movement. This is what we have in mind. Effect of the Invention

[0009] According to the present disclosure, it is possible to obtain a non-contact input device that enables a non-contact input operation based on a simple action by a person expressing his / her intention. [Brief description of the drawings]

[0010] [Figure 1] 1 is an overall configuration diagram of a non-contact input device according to a first embodiment of the present disclosure applied to a vending machine. [Diagram 2]FIG. 11 is an overall configuration diagram of a non-contact input device according to a second embodiment of the present disclosure, when applied to an expression of intent by a person in need of care. [Diagram 3] FIG. 11 is an overall configuration diagram of a non-contact input device according to a third embodiment of the present disclosure, when applied to an expression of intent by a person in need of care. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of a non-contact input device according to the present disclosure will be described with reference to the drawings. The non-contact input device according to the present disclosure has a technical feature of enabling a non-contact input operation based on a specific action of a person expressing an intention that is identified from a moving image captured by an imaging device.

[0012] Embodiment 1 In the first embodiment, a non-contact input device according to the present disclosure is applied to a vending machine to enable non-contact selection of a desired product. Fig. 1 is a diagram showing the overall configuration of a vending machine to which the non-contact input device according to the first embodiment of the present disclosure is applied.

[0013] The non-contact input device according to the present embodiment 1 shown in FIG. 1 is configured to include a camera 10 that captures an image of a person expressing his / her intention 1, a video memory 20 that stores video images acquired by the camera 10, a control unit 30 that performs image processing and display control, and a display unit 40 that switches and displays the display screen based on the display control by the control unit 30.

[0014] One feature of the non-contact input device according to the present disclosure is the use of a camera 10 equivalent to an imaging device, and the advantages of using the camera 10 will be described first. As described in the prior art, when a non-contact type touch panel, optical sensor, or the like is used as an input device, a specific action needs to be performed near the input device, and the position where the specific action is recognized is extremely limited. In addition, there is a risk of accidentally touching the device.

[0015] In contrast, when non-contact is achieved using the camera 10, specific actions can be captured over a wide range, making it possible to make selective input while maintaining a sufficient distance between the input device and the person expressing their intention 1.

[0016] Another possible configuration for enabling non-contact selection input is to use voice recognition technology. However, when using voice recognition technology, there are psychological issues such as the embarrassment of speaking out loud when outdoors where there are many people. Also, when using voice recognition technology, it is necessary to remove noise other than that of the person expressing his / her intention.

[0017] In contrast, when using camera 10 to achieve non-contact, there is no need to speak, and since the user standing in front of camera 10 is considered to be the person expressing his / her intention 1, there is no need to consider anyone other than the user.

[0018] Next, each component will be described. The display unit 40 is provided on the front of the vending machine, and a display screen that the intention expresser 1 visually confirms when purchasing a product is displayed on the display unit 40. In Fig. 1, six types of products, products A to F, can be selected, and the current display screen shows that product E is selected.

[0019] Conventional vending machines are provided with a switch for selecting one of the products. In contrast, the vending machine according to the first embodiment does not have a switch, but has an identification display unit 41 for identifying which product is selected. In Fig. 1, the identification display unit 41 corresponding to product E among the six identification display units 41 is displayed with a black circle, and the other five identification display units 41 are displayed with circles, thereby illustrating an example in which product E is selected.

[0020] Such a display section 40 can be configured as a display panel as a whole, or the six distinguishable display sections 41 can be configured as lamps.

[0021] The intention expresser 1, who is a user of the vending machine, stands in front of the camera 10 installed in the vending machine and performs a predetermined movement instruction action to face the camera 10 and move the position of the identification display unit 41, which is currently located at the location of the selected product, to the location of the desired product. One example of the predetermined movement instruction action is the action of moving a finger in any of the four directions of up, down, left, and right. In other words, the intention expresser 1 does not need to touch the vending machine, but only needs to be in a position where the display content displayed on the display unit 40 can be identified and where the finger can be imaged by the camera 10.

[0022] The input information detection unit 31 performs image processing on the video captured by the camera 10, and when it detects the intention expresser 1 moving his / her finger in any of the four directions (up, down, left, right), extracts this as a predetermined movement instruction motion. Then, the display control unit 32 moves the display position of the ● to any of the directions (up, down, left, right) according to the extraction result by the input information detection unit 31, and changes the currently selected and displayed product.

[0023] That is, the input information detection unit 31 performs image processing on the moving image to extract a predetermined movement instruction action by the intention expressing person 1. Furthermore, the display control unit 32 executes a display switching process to switch the display content of the display screen to be displayed on the display unit 40 according to the extraction result by the input information detection unit 31.

[0024] When the desired product is displayed with a circle, the person 1 who intends to use the vending machine performs a predetermined confirmation instruction action to confirm the currently selected product. One example of the predetermined confirmation instruction action is the action of making a hand shape such as scissors or rock.

[0025] The input information detection unit 31 performs image processing on the video captured by the camera 10, and when it detects the intention expresser 1 making a hand shape such as scissors or rock, it extracts this as a predetermined confirmation instruction motion. Then, the input information detection unit 31 confirms the currently selected product according to the extraction result.

[0026] That is, the input information detection unit 31 extracts a predetermined confirmation instruction action by the person expressing his / her intention 1 by performing image processing on the moving image, and executes a confirmation process to confirm the expression of intention to select the current display content on the display unit 40 based on the extraction result.

[0027] In this way, the control unit 30 can specify the content of the intention expression that the intention expresser 1 wants to convey by executing image processing, display switching processing, and confirmation processing. That is, the intention expresser 1 who uses a vending machine can select a desired product by performing a predetermined specific action while maintaining a certain distance from the vending machine without touching the vending machine.

[0028] Embodiment 2 In the second embodiment, a case will be described in which the non-contact input device according to the present disclosure is applied to the expression of will of a person requiring care, etc. When the person expressing his / her will 1 is a patient with medical equipment such as an artificial respirator or an IV attached to his / her body, or a physically disabled person, it is possible that he / she may be in a situation in which he / she can only move a part of his / her body, or in which he / she cannot make a sound.

[0029] For this reason, it is difficult to use a non-contact type touch panel, optical sensor, voice recognition, etc. as an input means. In contrast, when using a camera 10 as in the present disclosure, selective input is possible by utilizing slight movements of a part that the will expresser 1 can move. Therefore, even when the will expresser 1 is a patient or a physically disabled person as described above, the non-contact input device according to the present disclosure can be easily applied.

[0030] FIG. 2 is a diagram showing an overall configuration of a non-contact input device according to a second embodiment of the present disclosure when applied to an expression of intent by a person in need of care.

[0031] The non-contact input device according to the second embodiment shown in FIG. 2 is configured to include a camera 10 that captures an image of a person expressing his / her intention 1 lying on a bed 2, a video memory 20 that stores video images acquired by the camera 10, a control unit 30 that performs image processing and display control, and a display unit 40 that switches and displays the display screen based on the display control by the control unit 30.

[0032] The display unit 40 is provided at a position where it can be seen by the will expresser 1, who is a patient lying on the bed 2, and a display screen that the will expresser 1 sees when selecting a message to convey his / her intention is displayed on the display unit 40. Fig. 1 shows a state where either the display screen 1 showing the message "I'm in pain" or the display screen 2 showing the message "I want some water" is displayed.

[0033] That is, the non-contact input device according to the second embodiment has a configuration capable of switching between and displaying predetermined communication sentences 42, such as "painful" or "I want water." Note that, while Fig. 2 illustrates an example of a configuration in which two display screens can be selected, namely, display screen 1 displaying the message "painful" and display screen 2 displaying the message "I want water," it is also possible to provide further display screens with different messages.

[0034] Furthermore, it is also possible to display a plurality of communication sentences 42 on one screen and then distinguishably display them using the distinguishable display unit 41 as described in the first embodiment above.

[0035] Such a display unit 40 can be configured as a display panel having a plurality of display screens, and the identification display unit 41 can be configured as a lamp.

[0036] The person expressing their intention 1 on the bed 2 performs a predetermined movement instruction action so as to change the currently selected and displayed display screen to a desired display screen from the bed 2 toward the camera 10. One example of the predetermined movement instruction action is the action of moving a finger in either of two directions, up or down.

[0037] The input information detection unit 31 performs image processing on the moving image captured by the camera 10, and when it detects the intention manifester 1 moving his / her finger in either of the two directions, up or down, it extracts this as a predetermined movement instruction motion. Then, the display control unit 32 switches the display between the display screen 1 and the display screen 2 according to the extraction result by the input information detection unit 31.

[0038] That is, the input information detection unit 31 performs image processing on the moving image to extract a predetermined movement instruction action by the intention expressing person 1. Furthermore, the display control unit 32 executes a display switching process to switch the display content of the display screen to be displayed on the display unit 40 according to the extraction result by the input information detection unit 31.

[0039] When the desired communication sentence 42 is displayed, the patient 1 performs a predetermined confirmation instruction action to confirm the display screen currently displaying the desired communication sentence 42. One example of the predetermined confirmation instruction action is the action of making a hand shape such as scissors or rock.

[0040] The input information detection unit 31 performs image processing on the video captured by the camera 10, and when it detects the intention expresser 1 making a hand shape such as scissors or rock, it extracts this as a predetermined confirmation instruction motion. Then, the input information detection unit 31 confirms the currently selected and displayed communication sentence 42 according to the extraction result.

[0041] That is, the input information detection unit 31 extracts a predetermined confirmation instruction action by the person expressing his / her intention 1 by performing image processing on the moving image, and executes a confirmation process to confirm the expression of intention to select the current display content on the display unit 40 based on the extraction result.

[0042] In this way, the control unit 30 executes image processing, display switching processing, and confirmation processing to specify the intention expression content that the intention expresser 1 wants to convey as the desired communication sentence 42. In other words, the intention expresser 1 can select the desired communication sentence 42 by performing a predetermined specific action without touching the input device and while maintaining a certain distance from the input device.

[0043] In other words, in the non-contact input device according to the second embodiment, the main body with the display unit 40 as a display can be separated from the camera 10. The main body with the display unit 40 is placed in a position within the field of view of the person expressing his / her intention 1 who is a patient, and the camera 10 is placed in a position where the hand of the person expressing his / her intention 1 who is a patient can be seen.

[0044] Furthermore, multiple communication sentences 42 are prepared in advance, and the user can select the desired communication sentence 42 with just a few movements. As a result, communication can be made easier and faster than when characters are directly input, and this is also useful in emergencies.

[0045] Embodiment 3 In the above-mentioned second embodiment, when the non-contact input device of the present disclosure is applied to the expression of will of a person requiring care or the like, a predetermined communication sentence 42 is used as shown in Fig. 2. In contrast to this, in the present embodiment 3, a non-contact input device having a configuration that enables the person expressing their will 1 to express their will using any character string will be described.

[0046] Fig. 3 is a diagram showing the overall configuration of a non-contact input device according to a third embodiment of the present disclosure when applied to the expression of intent of a person requiring care. Compared to the configuration of Fig. 2 in the second embodiment, the configuration of Fig. 3 in the third embodiment differs in the display content of the display unit 40 and in the control content of the control unit. Therefore, the following description will focus on these differences.

[0047] 3, a selection display section 44 for selecting one of the Japanese syllabaries is displayed together with a list 43 of the Japanese syllabaries. In response to such a display content, the person expressing their intention 1 performs a predetermined movement instruction action so as to move the position of the selection display section 44 currently displayed toward the camera 10 to the position of a desired Japanese syllabary. One example of the predetermined movement instruction action is the action of moving a finger in any of the four directions, up, down, left, and right.

[0048] The input information detection unit 31 performs image processing on the moving image captured by the camera 10, and when it detects the intention expresser 1 moving his / her finger in any of the four directions, up, down, left, or right, it extracts this as a predetermined movement instruction motion. Then, the display control unit 32 moves the display position of the selection display unit 44 to any of the directions, up, down, left, or right, depending on the extraction result by the input information detection unit 31, and changes the sound currently surrounded by the selection display unit 44.

[0049] When the display position of the selection display unit 44 reaches a state where the desired sound can be selected, the intention expresser 1 performs a predetermined selection instruction action in order to select the sound currently displayed in the display position of the selection display unit 44. One example of the predetermined selection instruction action is the action of making a scissors shape with one's hand.

[0050] The input information detection unit 31 performs image processing on the video captured by the camera 10, and when it detects the intention manifester 1 making a scissors shape with his / her hand, it extracts it as a predetermined selection instruction motion. Then, the input information detection unit 31 confirms the one sound currently surrounded by the selection display unit 44 according to the extraction result.

[0051] The person expressing his / her intention then repeats the movement instruction action and the selection instruction action to sequentially select one syllable, thereby being able to select a character string corresponding to time-series data in which a plurality of one syllables are arranged.

[0052] When the selection of the desired character string is completed, the intention expresser 1 performs a predetermined confirmation instruction motion toward the camera 10. One example of the predetermined confirmation instruction motion is the motion of making a fist with one's hand.

[0053] The input information detection unit 31 performs image processing on the video captured by the camera 10, and when it detects the intention expresser 1 making a fist, extracts it as a predetermined confirmation instruction motion. Then, the input information detection unit 31 can specify the character string finally confirmed by the intention expresser 1 according to the extraction result.

[0054] In other words, when the display control unit 32 in this embodiment 3 extracts a movement instruction action as a predetermined action by image processing the moving image, it changes the display position of the selection display unit 44 to be displayed on the display unit according to the extracted movement instruction action, thereby switching the display content on the display screen and executing display switching processing.

[0055] On the other hand, when the input information detection unit 31 extracts a selection instruction action as a predetermined action by image processing the moving image, it detects as selection data a sound at the display position of the selection display section 44 displayed on the display unit at the time when the selection instruction action was performed.

[0056] Then, the input information detection unit 31 selects the desired character string by sequentially repeating the detection process based on the selection instruction action in conjunction with the display switching process by the display control unit 32 until it can extract a confirmation instruction action as a predetermined action.

[0057] Furthermore, when the input information detection unit 31 is able to identify the timing of the confirmation instruction action, it executes a confirmation process by confirming the character string consisting of the time-series data of the sequentially detected selection data as an intention expression. Therefore, by utilizing a display screen such as that shown in Figure 3, the intention expression person 1 can communicate using any character string consisting of the Japanese syllabary.

[0058] In addition, in the non-contact input device according to the third embodiment, the main body with the display unit 40 as a display can be separated from the camera 10. The main body with the display unit 40 is placed in a position within the field of view of the person expressing his / her intention 1 who is a patient, and the camera 10 is placed in a position where the hand of the person expressing his / her intention 1 who is a patient can be seen.

[0059] By using the camera 10 arranged in this manner, sick people and people requiring care who are able to move their limbs etc. can easily communicate without contact.

[0060] Instead of the list of the Japanese syllabary shown in FIG. 3, alphabets, pictograms, etc. may also be used.

[0061] In the above-mentioned first embodiment, the case where the movement instruction action, the selection instruction action, and the confirmation instruction action are each performed by hand or finger movements has been described, but the present disclosure is not limited to this. For example, eye movements, foot movements, etc. can also be used depending on the constraints of the person expressing his / her intention. [Explanation of symbols]

[0062] 10 camera, 20 video memory, 30 control unit, 31 input information detection unit, 32 display control unit, 40 display unit, 41 identification display unit, 42 communication sentence, 43 list of the Japanese syllabary, 44 selection display unit.

Claims

1. An imaging device that captures the movement of the person expressing his / her intention as a moving image; A display unit that switches between and displays one of a plurality of display screens in order to identify the content of the intention expressed by the person expressing his / her intention; a control unit that extracts a predetermined action by the person expressing his / her intention by performing image processing on the moving image of the person viewing the display content that is switched and executes a display switching process that switches the display content to be displayed on the display unit in accordance with the extraction result, and a confirmation process that confirms the intention to select the current display content, thereby specifying the expression of intention that the person expressing his / her intention wants to convey; Equipped with the imaging device and the display unit are disposed separately, the imaging device is installed at a position where it can image the movement of the person expressing his / her intention, and the display unit is installed at a position where it can be seen by the person expressing his / her intention, The control unit can use eye movements or foot movements as well as hand or finger movements by the person expressing his / her intention as the predetermined movements. Non-contact input device.

2. The control unit is a display control unit that, when extracting a movement instruction action as the predetermined action by performing image processing on the moving image, executes the display switching process by switching display content of a display screen to be displayed on the display unit in accordance with the extracted movement instruction action; an input information detection unit that, when extracting a confirmation instruction action as the predetermined action by performing image processing on the moving image, executes the confirmation process by confirming the display content displayed on the display unit at the timing when the confirmation instruction action was performed as the expression of intention; The non-contact input device according to claim 1 , further comprising:

3. The person expressing his / her intention is a patient lying in bed, The display unit has a configuration capable of switching between and displaying predetermined communication sentences, The control unit executes the image processing, the display switching processing, and the determination processing to specify the content of the intention expression that the patient wants to convey as a desired communication sentence. The non-contact input device according to claim 1 or 2.

Citation Information

Patent Citations

  • Interface device

    JP2006209563A

  • Interface device and its method

    JP2008052590A

  • Finger input device and game device using finger input device

    JP2010022673A

  • Advanced camera-based input

    JP2010534895A

  • Communication system and blink determination method

    JP2015202133A