Method for guiding route by using hand tracking

Abstract

The present invention relates to a method for guiding a route by using hand tracking and, more particularly, to a method for guiding a route by using hand tracking, wherein a route guide program is controlled by recognizing a user's hand gesture and motion, thereby enabling a physically handicapped user to request and receive necessary information more easily by using the route guide program.

Description

How to guide directions using hand tracking

[0001] The present invention relates to a route guidance method using hand tracking, and more specifically, to a route guidance method using hand tracking that recognizes a user's hand gestures and movements to control a route guidance program, thereby enabling even physically challenged users to more easily request and receive necessary information using the route guidance program.

[0002] Gestures are movements of the hands or body with a certain intention, and are usually used as an auxiliary function to voice, which is the main means of human communication, and perform the function of assisting smooth communication.

[0003] These gestures sometimes have a power of expression that goes beyond that of speech, and sign language and hand signals are used as independent communication tools with linguistic characteristics, and are expected to be utilized as next-generation interfaces that can replace difficult input interfaces such as keyboards and mice.

[0004] In particular, hand movements are the most frequently used gestures, and can serve as an interface that provides convenience to users even through simple applications in areas such as general home life or presentations.

[0005] That is, it is expected to be able to provide an interface role for controlling the Windows system, adjusting TV channels, and making announcements using a computer, and it is expected to be used as an interface role for a virtual environment.

[0006] Meanwhile, the development of the display panel industry has driven significant growth in the enterprise display market, known as Digital Information Displays (DID), beyond the consumer electronics market, including TVs. As the content displayed on these DIDs has become more diverse and complex, digital signage devices, which comprehensively combine software and platforms to control the displayed content, have emerged.

[0007] Digital signage devices are communication tools that can be used for corporate marketing, advertising, training, and customer experience enhancement. They can be installed in various public places, such as department stores, shopping malls, public institutions, bus stops, and subway stations, to provide advertising and information content to customers and pedestrians.

[0008] More specifically, digital signage devices are a medium that displays various contents and commercial advertisements by installing display devices in a certain outdoor location or on devices such as street furniture, and are currently installed in places where people move, such as apartment elevators, subway stations, subways, buses, schools, banks, and department stores.

[0009] Digital signage devices are mainly installed along public movement lines, so they have the advantage of being able to provide advertisements or information to a large number of people. However, they have the problem of making it difficult to provide accurate information to users who need specific information.

[0010] In particular, in crowded public places such as department stores or shopping centers, it is quite difficult for users who need route guidance to obtain information about the route from their current location to their destination through digital signage devices.

[0011] In addition, in the case of general digital signage devices or touch displays installed in public places, since users must directly touch the screen to use them, there is a problem that the devices or displays are frequently damaged due to physical touch, and in the case of physically disabled people, there are many limitations in using the devices in large crowds.

[0012] Therefore, there is a need for technology development for a method to detect the user's hand gestures and movements, thereby enabling the user to easily obtain necessary information without directly touching the device or display, and to provide various types of information by utilizing the user's hand tracking so that accurate detection of a specific user can be performed even when installed in a public place.

[0013] The present invention was created to solve the above-described problem, and more specifically, it provides a route guidance method using hand tracking that recognizes the user's hand gestures and movements to control the route guidance program, thereby enabling even physically challenged users to more easily request and receive necessary information using the route guidance program.

[0014] A route guidance method using hand tracking according to one embodiment of the present invention is a method for providing various information and route guidance information requested by a user using hand tracking, the method comprising: a user detection step of detecting a user through a camera and defining the detected user as a route guidance target; a hand gesture recognition step of recognizing hand gestures and movements of a route guidance target using a camera; a program control step of controlling a route guidance program based on information about the recognized hand gestures and movements of the route guidance target; and an information output step of outputting information selected or requested by the route guidance target through an audiovisual output device based on the controlled route guidance program.

[0015] More specifically, the user detection step is characterized by recognizing the user's face and determining whether to define the user as a target of route guidance based on the recognized user's face location and recognition maintenance time.

[0016] More specifically, the user detection step is characterized by recognizing the user's hand and determining whether to define the user as a guidance target based on the recognized user's hand position and recognition maintenance time.

[0017] More specifically, the user detection step is characterized by imaging the recognized user's hand position and shape and displaying it on a visual output device.

[0018] More specifically, the user detection step is characterized by defining the user as a target of route guidance when it is detected that the user has maintained the position and posture of the hand for a predetermined period of time after displaying the imaged position and shape of the user's hand.

[0019] More specifically, the hand gesture recognition step is characterized by detecting a change in hand gesture due to movement of the guidance subject based on the hand position and shape of the guidance subject.

[0020] More specifically, the hand gesture recognition step is characterized by distinguishing and recognizing the hand gestures and movements of the route guidance target based on the type, number of times, and speed of the gesture, and the program control step is characterized by controlling the route guidance program to output provided information according to various pre-stored hand gestures and movements.

[0021] More specifically, the information output step is characterized by providing detailed information on the direction and destination of the route selected by the route guidance target through hand gestures and movements.

[0022] More specifically, the detailed information is characterized by including information about a building located in the direction of the path selected by the person receiving the route guidance, a path to the destination, and a building located on the path.

[0023] More specifically, the route guidance program is provided in multiple languages ​​through an audiovisual output device, and the hand gesture recognition step is characterized in that the route guidance target selects an applicable language, a route direction, or a destination using hand gestures and movements.

[0024] More specifically, the route guidance program provides a zoom in and zoom out function when visually outputting information, and the program control step is characterized in that, when the hand gesture recognition step recognizes that the route guidance target has made a hand gesture and movement corresponding to the zoom in or zoom out function, the information output on the visual output device is output in a zoomed-in or zoomed-out manner.

[0025] More specifically, the route guidance method using the hand tracking is characterized in that the hand gesture recognition step, the program control step, and the information output step are repeatedly performed until a defined route guidance target is not detected.

[0026] A route guidance method using hand tracking according to one embodiment of the present invention provides the effect of enabling even physically challenged users to more easily request and receive necessary information using a route guidance program by recognizing a user's hand gestures and movements and controlling the route guidance program.

[0027] In addition, the route guidance method using hand tracking according to one embodiment of the present invention includes a user detection step (S100) that defines a route guidance target based on the position and recognition maintenance time of the user's face or hand, thereby providing an effect of preventing problems that may occur in which nearby pedestrians or other people who do not need route guidance are defined as route guidance targets, and thus the user is unable to receive necessary information or is provided with different information, causing the user to feel uncomfortable when using the route guidance program.

[0028] In particular, a route guidance method using hand tracking according to one embodiment of the present invention provides an effect of improving convenience of use by allowing the user to more easily identify whether his or her hand has been correctly recognized and normal use of the route guidance program is possible by displaying the recognized position and shape of the user's hand as an image.

[0029] In addition, the route guidance method using hand tracking according to one embodiment of the present invention includes a hand gesture recognition step (S200) that recognizes hand gestures and movements by distinguishing them based on the type, number of times, and speed of the gestures, thereby enabling more accurate detection of the intention or purpose of the user's hand gestures, and in particular, when selecting different functions such as selecting an applicable language or selecting a road direction, the program is configured to be executed based on different movements for each, thereby providing an effect of preventing malfunction of the program.

[0030] FIG. 1 is a block flowchart schematically illustrating the procedure of a route guidance method using hand tracking according to one embodiment of the present invention.

[0031] Figure 2 is a drawing showing an example of a state in which a route guidance program is output through a visual output device according to the user's current location and direction, and a state in which the imaged position and shape of the user's hand are displayed.

[0032] Figure 3 is a drawing showing an example of a state in which a route guidance program being used by a route guidance target is output through a visual output device.

[0033] Figure 3a is a diagram illustrating an example of a state in which a person receiving guidance selects an applicable language through hand gestures and movements.

[0034] Figure 3b is a drawing showing an example of a state in which a person receiving guidance selects a direction of the road through hand gestures and movements.

[0035] Figure 3c is a drawing showing an example of a state in which detailed information about the direction of the road selected by the person receiving guidance is output.

[0036] The detailed description of the present invention, which follows, refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention.

[0037] Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the present invention. Therefore, the following detailed description is not intended to be limiting, and the scope of the present invention, if properly described, is defined solely by the appended claims, along with the full scope equivalents thereof. Similar reference numerals in the drawings designate the same or similar functions throughout.

[0038] Hereinafter, in order to enable a person having ordinary skill in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0039] First, a route guidance method using hand tracking according to one embodiment of the present invention will be described with reference to FIGS. 1 to 3.

[0040] FIG. 1 is a block flowchart illustrating the progress sequence of a route guidance method using hand tracking according to one embodiment of the present invention, FIG. 2 is a diagram illustrating an example of a state in which a route guidance program is output through a visual output device according to a user's current location and direction and a state in which an image of the user's hand location and shape is displayed, and FIG. 3 is a diagram illustrating an example of a state in which a route guidance program being used by a route guidance target is output through a visual output device.

[0041] In addition, FIG. 3a is a drawing showing an example of a state in which a person receiving guidance selects an applicable language through hand gestures and actions, FIG. 3b is a drawing showing an example of a state in which a person receiving guidance selects a direction of a road through hand gestures and actions, and FIG. 3c is a drawing showing an example of a state in which detailed information about the direction of a road selected by a person receiving guidance is output.

[0042] First, the route guidance method using hand tracking of the present invention corresponds to a method for providing various information and route guidance information requested by a user using hand tracking.

[0043] As illustrated in FIG. 1, the route guidance method (S1000) using hand tracking of the present invention includes a user detection step (S100), a hand gesture recognition step (S200), a program control step (S300), and an information output step (S400).

[0044] Here, the user detection step (S100) is performed to detect a user through a camera and define the detected user as a target of route guidance, as illustrated in FIG. 2.

[0045] More specifically, the user detection step (S100) is performed to recognize the user's face and determine whether to define the user as a target of route guidance based on the recognized user's face location and recognition maintenance time.

[0046] As another example, the user detection step (S100) may be performed to recognize the user's hand and determine whether to define the user as a target of route guidance based on the recognized user's hand position and recognition maintenance time.

[0047] Here, the camera may be installed on top of the visual output device for more accurate detection, as shown in Fig. 2, and may be provided in the form of a device that is automatically driven through recognition of the user's face or hand.

[0048] In addition, the above visual output device may be provided in the form of signage, which is a visual structure that displays images or various information to convey specific information to an unspecified number of people, such as a signboard.

[0049] Accordingly, the hand tracking-based route guidance method of the present invention includes a user detection step (S100) that defines a route guidance target based on the position and recognition maintenance time of the user's face or hand, thereby providing an effect of preventing problems that may occur in which nearby pedestrians or other people who do not need route guidance are defined as route guidance targets, thereby failing to provide the actual user with the information he or she needs or providing different information, which may cause the user to feel uncomfortable when using the route guidance program.

[0050] Here, the user detection step (S100) is performed to image the recognized user's hand position and shape and display them on a visual output device, as illustrated in FIG. 2.

[0051] In addition, the user detection step is performed to define the user as a target of route guidance when it is detected that the user has maintained the position and posture of the hand for a predetermined period of time after displaying the imaged position and shape of the user's hand.

[0052] As described above, the route guidance method using hand tracking of the present invention can improve convenience of use by allowing the user to more easily identify whether his or her hand has been correctly recognized and normal use of the route guidance program is possible by displaying the recognized hand position and shape as an image, and by defining the route guidance target based on the hand position and recognition duration as well as the face, it can prevent the phenomenon of other people in the vicinity being mistakenly detected as the target.

[0053]

[0054] Next, the hand gesture recognition step (S200) is performed to recognize the hand gestures and movements of the person receiving guidance using a camera.

[0055] More specifically, the hand gesture recognition step (S200) is characterized by detecting a change in hand gesture due to movement of the guidance target based on the hand position and shape of the guidance target.

[0056] Next, the above program control step (S300) is performed to control the route guidance program based on information about the hand gestures and movements of the recognized route guidance target.

[0057] In addition, after the above program control step (S300), an information output step (S400) is performed to output information selected or requested by the route guidance target through an audiovisual output device based on the controlled route guidance program.

[0058] More specifically, the above information output step (S400) is performed to provide detailed information on the direction and destination of the route selected by the route guidance target through hand gestures and movements.

[0059] At this time, the detailed information is characterized in that it includes information about a building located in the direction of the path selected by the person receiving guidance, a path to the destination, and a building located on the path, as shown in FIG. 3c.

[0060] In addition, the hand tracking-based route guidance method (S1000) of the present invention is performed by repeatedly performing the hand gesture recognition step (S200), program control step (S300), and information output step (S400) until a defined route guidance target is not detected.

[0061] Through this configuration, the hand tracking-based route guidance method of the present invention recognizes the user's hand gestures and movements to control the route guidance program, thereby providing the effect of enabling even physically challenged users to more easily request and receive necessary information using the route guidance program.

[0062]

[0063] Next, with reference to FIG. 3, each function of a route guidance program performed through a route guidance method using hand tracking according to one embodiment of the present invention will be described.

[0064] First, the above-mentioned route guidance program is provided in multiple languages ​​through an audiovisual output device, as illustrated in Fig. 3.

[0065] For example, the above route guidance program may be configured to enable the route guidance recipient to use a function to change the applicable language on the output screen by using hand gestures and movements to activate buttons shaped like national or language flags on the visual output device.

[0066] Accordingly, the hand gesture recognition step (S200) described above is characterized in that the person receiving guidance selects the applicable language, direction of the road, or destination using hand gestures and movements.

[0067] In addition, the above-mentioned route guidance program provides zoom in and zoom out functions when visually outputting information, as shown in Fig. 3.

[0068] For example, the above-described route guidance program may be configured to enable the route guidance subject to use a "+" or "-" button on a visual output device using hand gestures and movements to use a zoom-in or zoom-out function on the output screen.

[0069] Accordingly, in the above-described program control step (S300), if it is recognized that the person receiving guidance has made a hand gesture and movement corresponding to the zoom in or zoom out function in the hand gesture recognition step (S200), the information output on the visual output device is output by zooming in or out.

[0070] Here, in the hand gesture recognition step (S200), the hand gestures and movements of the person receiving guidance are distinguished and recognized based on the type, number of gestures, and speed.

[0071] And, in the above program control step (S300), the route guidance program is controlled to output information provided according to various pre-stored hand gestures and movements.

[0072] For example, as illustrated in FIG. 3a, when a person receiving route guidance changes or selects the applicable language of a route guidance program, the function may be configured to be executed only when the person receiving route guidance moves his or her hand at a fast speed and performs a touch-related action more than twice.

[0073] In addition, as illustrated in FIG. 3b, when a person receiving route guidance selects a destination or a direction of a road on a route guidance program, the function may be configured to be executed only when the person receiving route guidance moves his or her hand at a slow speed and performs a touch-related action once.

[0074] As described above, the route guidance method using hand tracking of the present invention includes a hand gesture recognition step (S200) that recognizes hand gestures and movements by distinguishing them based on the type, number of times, and speed of the gestures, thereby enabling more accurate detection of the intention or purpose of the user's hand gestures, and in particular, when selecting different functions such as selecting an applicable language or selecting a road direction, the program is configured to be executed based on different movements, thereby providing an effect of preventing malfunction of the program.

[0075]

[0076] Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from this description.

[0077] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and all things that are modified equally or equivalently to the following claims as well as the claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. A method for providing various information and route guidance information requested by a user using hand tracking, A user detection step (S100) of detecting a user through a camera and defining the detected user as a target of route guidance; A hand gesture recognition step (S200) for recognizing hand gestures and movements of a person receiving guidance using a camera; A program control step (S300) for controlling a route guidance program based on information about the hand gestures and movements of a recognized route guidance target; and A route guidance method using hand tracking, characterized by including an information output step (S400) of outputting information selected or requested by the route guidance target through an audiovisual output device based on a controlled route guidance program.

2. In paragraph 1, The above user detection step (S100) is A route guidance method using hand tracking, characterized by recognizing a user's face and determining whether to define the user as a route guidance target based on the recognized user's face position and recognition maintenance time.

3. In paragraph 1, The above user detection step (S100) is A route guidance method using hand tracking, characterized by recognizing a user's hand and determining whether to define the user as a route guidance target based on the recognized user's hand position and recognition maintenance time.

4. In paragraph 3, The above user detection step (S100) is A route guidance method using hand tracking, characterized in that the recognized user's hand position and shape are visualized and displayed on a visual output device.

5. In paragraph 4, The above user detection step (S100) is A route guidance method using hand tracking, characterized in that after displaying the imaged user's hand position and shape, if it is detected that the user has maintained the hand position and posture for a predetermined period of time, the user is defined as a route guidance target.

6. In paragraph 1, The above hand gesture recognition step (S200) is A route guidance method using hand tracking, characterized in that it detects changes in hand gestures due to movement of a route guidance target based on the hand position and shape of the route guidance target.

7. In paragraph 1, The above hand gesture recognition step (S200) is Recognizes and distinguishes hand gestures and movements of the person receiving guidance based on the type, number, and speed of gestures. The above program control step (S300) is: A route guidance method using hand tracking, characterized in that the route guidance program is controlled to output provided information according to various pre-stored hand gestures and movements.

8. In paragraph 1, The above information output step (S400) is A route guidance method using hand tracking, characterized in that the route guidance target provides detailed information about the direction and destination of the selected route through hand gestures and movements.

9. In paragraph 8, The above details are, A route guidance method using hand tracking, characterized in that it includes information about a building located in the direction of the route selected by the person receiving the route guidance, a route to a destination, and information about a building located on the route.

10. In paragraph 1, The above route guidance program is, It is provided in multiple languages through audiovisual output devices, The above hand gesture recognition step (S200) is A route guidance method using hand tracking, characterized in that the route guidance target selects an applicable language, a route direction, or a destination using hand gestures and movements.

11. In paragraph 1, The above route guidance program is, When displaying visual output of information, it provides zoom in and zoom out functions. The above program control step (S300) is: A route guidance method using hand tracking, characterized in that when it is recognized that the route guidance target has made a hand gesture and movement corresponding to a zoom in or zoom out function in the above hand gesture recognition step, information output on a visual output device is output by zooming in or out.

12. In paragraph 1, The above method of guiding people using hand tracking is as follows: A route guidance method using hand tracking, characterized in that the hand gesture recognition step, the program control step, and the information output step are repeatedly performed until a defined route guidance target is not detected.

Images