Gesture interface robot

Abstract

A gesture interface robot (GIR) automatically measures a work zone to establish virtual puzzle cells located in a user gesture comfortable area. The GIR allows the user to easily move hands to press selected virtual keys so as to prevent injury. The GIR draws virtual keyboard images that display commands on virtual puzzle cells. The GIR uses video vision sensing to detect user's hand location on the virtual keyboard images. Special unique gesture hand sings are designed for enhancing hand control virtual key selections. A push-hand-z-dimensional distant is divided into 3 selectable zones. A real-time display highlights a selected key graphic image to be visually seen by the user. Using a UIRT cable to send IR signal to a remote control computer. The GIR instantly generates a virtual puzzle cell keyboard or controller that the user selected. The GIR introduces a new gesture interface method, a touch screen mouse combined with puzzle cell virtual keys in sandwich layers gesture zones and a method for operating computer keys and mouse operations.

Description

BACKGROUND OF THE INVENTION[0001]Field of the Invention[0002]This invention relates to an intelligent gesture interface robot (GIR) equipped with a video vision sensor to read user hand gestures, to operate computers, machines, and intelligent robots. The unique gesture reading method is that the GIR uses a puzzle cell mapping dynamic multiple sandwich layers work zone of a virtual touch screen mouse, keyboard, and control panel in the user's comfortable gesture action area. The GIR allows the user to easily move hands and push to click. It is easy to operate. It does not require the user to make hands swings action or abnormal body posting actions that the user could get hurt and hit object or others around them. The best gesture solution is my invented method using a puzzle cell mapping gesture method which is considered as a safe and efficient way. The user uses simple gesture actions to control all kind of computer machines all together. It does not require the user to remember ...

AI Technical Summary

Benefits of technology

[0013]The above problems are solved by the present invention of a GIR. To solve problems and improve a better way for human controlling computers, machines and intelligent robots, I have purposed the GIR that allows a user to use hands to move in a comfortable area to select a virtual puzzle cell keyboard, a mouse, or a control panel key and to push out the hand toward a selection as a click select action. A video vision of the GIR recognizes a user selection gesture action and its location according to the center point assigned on user. The GIR uses the relative distant on hands location with the center point to determine the X, Y position of a corresponding puzzle cell position. In addition, the GIR can also recognize user push hands locations in a Z direction from a distant change between hands and user body distant. For example, if a hand location being pushed out, the distant between the hand and the user body will increase. The maximum of this push distant is a total length of the hand and arms that normal human can push the hand out.

[0019]On the other hand, my proposed GIR is using puzzle cell mapping method. The GIR is acting like a graphic image (Picasso) painter. The GIR program draws the graphic picture of virtual control panels keys. On the display monitor, the puzzle cell virtual control panel keys can be drawn as a grid image of rows and columns cells and be tilted with TextBlock field, then fill in text word to TextBlock field on each grid row and column cells on the graphic image as command. Therefore, inside the program function those text command words can be coded and arranged into a two-dimensional array text strings, then loading each text word into row and column cells, so that display on the graphic puzzle cell image and virtually have assign on user's working space zone. The user can freely work around, sit in chair, the GIR provides user tracking in video view of the user, assign virtual center point on user and create work zone and establish virtual control panel keys in front of user and plus intelligent motor module that can physical rotate video sensor to aim vision tracking on user accordingly when if use walk out of it video view. The GIR can assign virtual center point on user such as prefer work space virtual center point is user's shoulder's center point where join connected with throat neck and establish the work zone size by width, and height. The prefer work space width is each shoulder length×1.5 on each side, so total workspace zone width prefer 1.5+1.5=3 and the prefer work space height is the shoulder's center point up to the user's head center times 2 to 3. Additional virtual points to be assigned on user body if special need for handicap disability users that require special assignment that could be anywhere of user body location. User without arms, could use mouth to hold pointer stick, or water color painting pen to make gesture selection. The left and right shoulder's edge points can be added into program for enhancement of the selection X, Y value reading accuracy, and the hands' palm size value (open and hold fingers) can be added in the program to enhance the click selection reading accuracy. Therefore the GIR creates a prefect comfortable work space zone for the user, and the user can move the user's hands in comfortable space, and in all directions without difficulty, and prevent problems such as injury of themselves, or hitting any others, or objects around. Because the GIR uses puzzle cell mapping method, the GIR can graphically draw any virtual mouse, keyboard, and control panels that the user wanted instantly. The gesture video sensor requires the user to do simple hand moving, and click action. the GIR can be built by using a regular computer, laptop with video camera, lower system electricity consumption, low building equipment cost. The GIR can be used by everyone in convenience, waling moving, sitting and everywhere.

[0020]The GIR can be used in all areas on the Earth. Furthermore, under zero gravity environment, physical motion is difficult, the GIR is useful in spaceship that astronaut can use gesture action to move their hands in front of them to control computer, machine, and intelligent robot. The GIR also frees the room space and reduce spaceship weight.

[0021]In addition to unique gesture continuous click action, the GIR vision enables the user to move the user's hand in front of the user like a fish swimming using fins smoothly and to softly move each fingers UP and Down like waving fin to control a continuous click action in 3rd click selection zone. In 3rd selection zone user's hand palm makes the fish fin waving swimming gesture action as hand sign and the GIR vision program can detect the distance changing, hand's palm center visible blinking like night sky star view, each wave makes a blinking and the GIR automatically blinks to continue click action without requiring the user pulling hand back to 1st selection zone to unlock and push out to reselect action. This unique gesture of fish fin waving swimming hand palm sign makes the user very easy to control machines when continuous clicks are required such as TV volume UP / Down, or computer Mouse moving UP, Down, Left, Right, etc.

[0023]In summary, the TouchScreen Mouse is combined with the Virtual Puzzle Cell keys control panels in sandwich layers functions as an advanced gesture system that includes all current computer interface device methods to be the one true universal computer interface device and enable the user to perform gesture control of all machine functions together, and easy gesture to control computer, without need to physical built mouse, keyboard, remote controller, control interface on equipments, machines, robots. The GIR will replace the need for building physical control panels, interface devices, reduce high tech device pollution and save the material resource usage on the Earth.

[0024]The GIR can be equipped with output display devices options, such as display monitor, visual image projector on any surface, wireless monitor glass that user can wear and see the project monitor screen in the lances. The GIR can control a wireless BLUETOOTH card attached with micro controller board or a smart phone to control LED light on and OFF to display MORSE code of text command select on, or to generate a vibration long and short signals of MORSE code of text command too. User can wear the wireless display MORSE code text command device on their palm's back and LED lights face direction to themselves, or like a watch. When the user's hand moves on puzzle cell, then the GIR program will send a command to wireless micro controller boards to blink LED light ON / OFF Long and short to indicate which command select on and / or motor vibration long and short signal for silent reading text command. So, the user doesn't need to watch display monitor, and this feature is especially useful for poor eyesight user and blind users so they can perform gesture selection like normal people do.

Problems solved by technology

However, there is limited computer keys that can be assigned to particular commands, and also particular physical key will be assigned and can not be used for normal typing function any more.

Images