Barrier-free kiosk device to which hand tracking function is applied and information providing method using same

Abstract

According to one embodiment of the present invention, provided is a barrier-free kiosk (1000) to which a hand tracking function is applied, comprising: a main body unit (100) including a face recognition camera (110) for photographing a person or an object located in front and transmitting the photographing result to a control unit, a hand gesture recognition camera (120) for photographing and sensing the position, speed, and shape of a hand and transmitting same to the control unit, and a display unit (130) for outputting necessary information and generating a hand tracking cursor upon receiving a command from the control unit; a control unit (200) receiving content photographed and sensed by the main body unit to form a command; and a moving unit (300) formed in the lower portion of the main body unit to move the main body unit. According to the present invention, it is possible to provide a barrier-free kiosk device to which a hand tracking function is applied to prevent frequent damage to a device or a display by a user's direct hand contact, and an information providing method using same.

Description

Barrier-free kiosk device with hand tracking function and information provision method using the same

[0001] The present invention relates to a barrier-free kiosk device applying a hand tracking function and an information providing method using the same, and more particularly, to a barrier-free kiosk device applying a hand tracking function that enables even physically disabled users to use the device more easily by recognizing the movement and shape of the user's hand and operating the device, and an information providing method using the same.

[0002] In general, the term kiosk is used to refer to a device installed in a fixed location that receives user input and outputs various information desired by the user.

[0003] Kiosks can be broadly categorized into unmanned ordering machines that automatically order products based on user input, and interactive kiosks that display the information desired by the user based on user input.

[0004] Existing kiosks were mainly installed inside businesses to reduce labor costs within stores, but due to COVID-19, non-face-to-face services have increased significantly, and as the content displayed on digital information displays in addition to businesses selling products has become more diverse and complex, digital signage devices, which are a comprehensive combination of software and platforms for controlling the displayed content, have emerged.

[0005] Digital signage devices are communication tools that can be used for corporate marketing, advertising, training, and customer experience enhancement. They can be installed in various public places, such as department stores, shopping malls, public institutions, bus stops, and subway stations, to provide advertising and information content to customers and pedestrians.

[0006] However, since digital signage devices are mainly installed along the movement paths of the public, they have the advantage of being able to provide advertisements or information to a large number of people, but there was a problem in that it was difficult to provide accurate information to users who needed specific information.

[0007] Additionally, there are concerns that the kiosks are not safe from COVID-19 infection because the public operates them by touching them with their hands, that physical contact frequently causes damage to the devices or displays, and that people with disabilities who have physical difficulties have many limitations in using the devices in large crowds.

[0008] As an example of solving this problem, Korean Patent Publication No. 10-2023-0094808 (June 28, 2023) discloses an intelligent barrier-free kiosk that can operate appropriately in response to a user's approach and a control method thereof.

[0009] However, even in this case, only the height adjustment function is included, and there are still limitations such as frequent damage to the device or display due to direct hand contact by certain users, and inconvenience in use by physically disabled people.

[0010] In order to solve these problems, if a barrier-free kiosk device with a hand tracking function and an information provision method using the same are provided, the problem of frequent damage to the device or display due to direct hand contact by the user through the kiosk with the hand tracking function can be solved.

[0011] In addition, the inventors have come up with the present invention by noting the possibility that even physically disabled people can conveniently use the kiosk by using hand gestures.

[0012]

[0013] Therefore, the purpose of the present invention is to solve all of the above-described problems.

[0014] Another purpose is to provide a barrier-free kiosk device with a hand tracking function that prevents frequent damage to the device or display due to direct hand contact by the user, and a method for providing information using the same.

[0015] In addition, another purpose of the present invention is to provide a barrier-free kiosk device with a hand tracking function that allows a physically disabled person to conveniently use the kiosk using hand gestures, and a method for providing information using the same.

[0016] According to one embodiment of the present invention, a barrier-free kiosk device (1000) applying a hand tracking function is provided, which includes a face recognition camera (110) that photographs a person or an object located in front and transmits the photographing result to a control unit, a hand gesture recognition camera (120) that photographs and detects the position, speed, and movement of a hand and transmits the same to the control unit, a main body (100) including a display unit (130) that receives a command from the control unit and generates a hand tracking cursor and outputs necessary information, a control unit (200) that receives the content photographed and detected by the main body and forms a command, and a moving unit (300) formed at a lower portion of the main body so as to enable the kiosk device to be moved.

[0017] More specifically, a barrier-free kiosk device (1000) is provided that applies a hand tracking function, characterized in that the facial recognition camera (110) is located on the upper part of the display unit (130), recognizes a user located in front, and photographs the user.

[0018] More specifically, the hand gesture recognition camera (120) is located at the bottom of the display unit (130), and a barrier-free kiosk device (1000) is provided that applies a hand tracking function characterized by detecting changes in the user's hand gestures based on the user's hand position and movement.

[0019]

[0020] More specifically, a barrier-free kiosk device (1000) is provided, which applies a hand tracking function characterized in that the hand gesture recognition camera (120) distinguishes and recognizes the user's hand gestures and movements based on the type, number and speed of the gesture, and movement changes.

[0021] More specifically, a barrier-free kiosk device (1000) with a hand tracking function is provided, characterized in that the main body (100) includes an input device (140) that converts information input by a user into an electric signal and transmits it to a control unit (200), and a speaker (150) that converts result data processed by the control unit (200) into an audio signal and outputs it.

[0022] More specifically, a barrier-free kiosk device (1000) with a hand tracking function is provided, characterized in that the input device unit (140) is configured with at least one selected from a Braille pad, a QWERTY keyboard, a mouse, and a microphone.

[0023] More specifically, a barrier-free kiosk device (1000) with a hand tracking function is provided, characterized in that the main body (100) further includes a light sensor unit (160) that recognizes people and objects and activates or terminates the screen of the display unit (130).

[0024] More specifically, a barrier-free kiosk device (1000) is provided with a hand tracking function in which the light sensor unit (160) adjusts the brightness of the display unit (130) when a person or object is not recognized for a reference time.

[0025] According to one embodiment of the present invention, an information providing method (S1000) using a hand tracking kiosk is provided, which includes a user detection step (S100) of detecting and recognizing a user through a camera, a hand gesture recognition step (S200) of recognizing a hand gesture and motion of the user using a camera, a kiosk program control step (S300) of controlling a kiosk program based on information about the recognized hand gesture and motion of the user, and an information output step (S400) of outputting information or a command requested by the user through an audiovisual output device based on the controlled kiosk program.

[0026] More specifically, the user detection step (S100) provides an information provision method (S1000) using a hand tracking kiosk, characterized in that the user is recognized using a light sensor unit (160) and the user's shape and image are transmitted to the control unit (200) using a facial recognition camera (110).

[0027] More specifically, the hand gesture recognition step (S200) provides an information provision method (S1000) using a hand tracking kiosk characterized in that it detects the user's hand position verification and action, speed, and movement using a hand gesture recognition camera (120).

[0028] More specifically, the kiosk program control step (S300) is characterized in that the control unit receives information generated about the user's shape and hand gesture in the user detection step and the hand gesture recognition step, defines user information and hand gesture commands, and controls the kiosk program to output commands according to the set hand gestures and movements, thereby providing an information providing method (S1000) using a hand tracking kiosk.

[0029] More specifically, the information output step (S400) provides an information provision method (S1000) using a hand tracking kiosk, characterized in that it provides detailed information on menu movement and menu selection selected by a user through hand gestures and actions.

[0030] More specifically, the kiosk program includes a zoom in and zoom out function when outputting information, and the kiosk control step (S300) is characterized in that, when the hand gesture recognition step recognizes that the user has made a hand gesture and movement corresponding to the zoom in or zoom out function, information output on the display unit is output in a zoomed-in or zoomed-out manner. An information providing method (S1000) using a hand tracking kiosk is provided.

[0031] More specifically, the information providing method (S1000) using the hand tracking kiosk is provided, characterized in that the hand gesture recognition step, the program control step, and the information output step are repeatedly performed until the user is no longer detected.

[0032] According to the present invention, a barrier-free kiosk device with a hand tracking function that prevents frequent damage to a device or display due to direct hand contact by a user and a method for providing information using the same can be provided.

[0033] In addition, a barrier-free kiosk device with a hand tracking function that allows a physically disabled person to conveniently use the kiosk using hand gestures and a method of providing information using the same can be provided.

[0034] FIG. 1 is a front perspective view of a barrier-free kiosk applying a hand tracking function according to one embodiment of the present invention.

[0035] FIG. 2 is a rear perspective view of a barrier-free kiosk applying a hand tracking function according to one embodiment of the present invention.

[0036] Figure 3 is a complete flowchart of an information provision method using a hand tracking kiosk according to one embodiment of the present invention.

[0037]

[0038] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the positions or arrangements of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the present invention is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly described. Like reference numerals in the drawings designate the same or similar functionality throughout the several aspects.

[0039] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

[0040]

[0041] FIG. 1 is a front perspective view of a barrier-free kiosk applying a hand tracking function according to one embodiment of the present invention.

[0042] FIG. 2 is a rear perspective view of a barrier-free kiosk applying a hand tracking function according to one embodiment of the present invention.

[0043] Referring to FIGS. 1 and 2, a barrier-free kiosk (1000) with a hand tracking function may be composed of a main body (100), a control unit (200), and a moving unit (300).

[0044] More specifically, the main body (100) may be composed of a facial recognition camera (110), a hand gesture recognition camera (120), and a display unit (130), and may additionally include an input device unit (140), a speaker (150), and a light sensor unit (160).

[0045] First, the facial recognition camera (110) is located at the top and can capture a person or object located in front, and in the case of a user, capture the height and hand position of the user and transmit the capture result to the control unit (200).

[0046] The above hand gesture recognition camera (120) may be able to capture and detect the position, speed, and shape of the hand and transmit the captured images to the control unit, and may be able to detect changes in the user's hand gestures based on the position and shape of the user's hand.

[0047] In addition, the hand gesture recognition camera (120) may be able to distinguish and recognize the user's hand gestures and movements based on the type, number of times, and speed of the gestures.

[0048] The display unit (130) may output necessary information and generate a hand tracking cursor by receiving commands from the control unit.

[0049] The above display unit (130) may be implemented in a form that displays information composed of images or photos to convey specific information to an unspecified number of people, such as the internal structure of a building or advertisements.

[0050] The above input device unit (140) may be additionally configured and may convert information input by the user into an electrical signal and transmit it to the control unit (200).

[0051] In addition to the hand-tracking cursor, this could consist of one or more input devices selected from a Braille pad, QWERTY keyboard, mouse, and microphone, allowing users to access information using the kiosk in their preferred manner.

[0052] The above speaker (150) may be able to convert the result data processed by the control unit (200) into a sound signal and output it, and in the case of a visually impaired person, the selected content may be heard as a voice to select the menu of the kiosk.

[0053] The above light sensor unit (160) can adjust the brightness of the display unit (130) when a person or object is not recognized for a reference time or longer. This is done by setting a reference time timer, and when a user is recognized, the brightness of the display increases. When a user is not recognized by the light sensor unit (160) within the reference time timer, the brightness of the display decreases, thereby saving power and making it possible to confirm whether a user has been recognized.

[0054] Figure 3 is a complete flowchart of an information provision method using a hand tracking kiosk according to one embodiment of the present invention.

[0055] Referring to FIG. 3, the information provision method (S1000) using a hand tracking kiosk may be composed of a user detection step (S100), a hand gesture recognition step (S200), a kiosk program control step (S300), and an information output step (S400).

[0056] The above user detection step (S100) may recognize the user using the light sensor unit (160) and transmit the user's shape and image to the control unit (200) using the facial recognition camera (110).

[0057] The above hand gesture recognition step (S200) may recognize the user by detecting the position and shape of the user's hand using a hand gesture recognition camera (120).

[0058] In addition, the hand gesture recognition step (S200) may capture elements such as the shape, position, and speed of the hand taken by the user and transmit them to the control unit.

[0059] The hand gesture recognition step (S200) will enable the user to be identified based on the position and shape of the hand as well as the face, preventing other people in the vicinity from being mistakenly identified as the user.

[0060] The above kiosk program control step (S300) is a step for controlling a kiosk program based on information about the recognized user's hand gestures and movements, and the control unit receives information generated about the user's shape and hand gestures in the user detection step and hand gesture recognition step, defines user information and hand gesture commands, and controls the kiosk program to output commands according to the set hand gestures and movements.

[0061] The above information output step (S400) is a step of outputting information or a command requested by a user based on a controlled kiosk program through an audiovisual output device, which may transmit one or more visual information through a display unit (130) and auditory information through a speaker, and may provide detailed information on menu movement and menu selection selected by the user through hand gestures and actions.

[0062] Additionally, the above kiosk program has a zoom in and zoom out function when outputting information.

[0063] It will be possible to equip it.

[0064] In addition, if it is recognized that the user has made a hand gesture and movement corresponding to the enlargement or reduction function in the hand gesture recognition step in the above kiosk control step (S300), the information output on the display unit (130) can be enlarged or reduced and output.

[0065] In addition, when the brightness of the display unit (130) is increased through the light sensor unit (160), the hand gesture recognition step, the program control step, and the output steps can be repeatedly performed until the brightness of the display unit (130) is decreased because the user is not detected.

[0066]

[0067] Although the invention has been described with reference to specific details such as specific components and limited embodiments and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains may make modifications and variations based on these descriptions.

[0068] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and all things that are modified equally or equivalently to the following claims as well as the claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. In a barrier-free kiosk device (1000) with a hand tracking function, A main body (100) including a face recognition camera (110) that photographs a person or object located in front and transmits the photographing results to a control unit, a hand gesture recognition camera (120) that photographs and detects the position, speed, and movement of a hand and transmits the results to the control unit, and a display unit (130) that receives a command from the control unit, generates a hand tracking cursor, and outputs necessary information; A control unit (200) that receives and forms commands from the contents captured and detected by the main body; A moving part (300) formed at the lower part of the main body to enable the kiosk device to be moved; A barrier-free kiosk device (1000) having a hand tracking function.

2. In paragraph 1, The above facial recognition camera (110) It is located at the upper part of the above display part (130), It is characterized by recognizing a user located in front and photographing the user. Barrier-free kiosk device (1000) with hand tracking function.

3. In paragraph 1, The above hand gesture recognition camera (120) A barrier-free kiosk device (1000) having a hand tracking function, which is located at the lower portion of the display unit (130) and detects changes in the user's hand gestures based on the user's hand position and movement.

4. In paragraph 1, The above hand gesture recognition camera (120) A barrier-free kiosk device (1000) having a hand tracking function, characterized by distinguishing and recognizing a user's hand gestures and movements based on the type, number and speed of the gestures, and changes in movement.

5. In paragraph 1, The above main body (100) is An input device unit (140) that converts information input by the user into an electrical signal and transmits it to the control unit (200); A speaker (150) that converts the result data processed in the above control unit (200) into a sound signal and outputs it; A barrier-free kiosk device (1000) having a hand tracking function, characterized by including:

6. In paragraph 5, The above input device unit (140) is A barrier-free kiosk device (1000) having a hand tracking function, characterized by comprising at least one selected from a Braille pad, a QWERTY keyboard, a mouse, and a microphone.

7. In paragraph 1, The above main body (100) is A light sensor unit (160) that recognizes people and objects and activates or turns off the screen of the display unit (130); A barrier-free kiosk device (1000) having a hand tracking function characterized by further including:

8. In paragraph 7, The above light sensor part (160) is A barrier-free kiosk device (1000) having a hand tracking function, characterized by adjusting the brightness of a display unit (130) when a person or object is not recognized for a reference period of time.

9. In the method of providing information using a hand tracking kiosk (S1000), A user detection step (S100) of detecting a user through a camera and recognizing the user; A hand gesture recognition step (S200) for recognizing a user's hand gestures and movements using a camera; A kiosk program control step (S300) for controlling a kiosk program based on information about recognized user's hand gestures and movements; An information output step (S400) for outputting information or commands requested by a user through an audiovisual output device based on a controlled kiosk program; Information providing method using a hand tracking kiosk including (S1000).

10. In paragraph 9, The above user detection step (S100) is An information providing method (S1000) using a hand tracking kiosk characterized by recognizing a user using a light sensor unit (160) and transmitting the user's shape and image to a control unit (200) using a facial recognition camera (110).

11. In paragraph 9, The above hand gesture recognition step (S200) is An information providing method (S1000) using a hand tracking kiosk characterized by detecting the user's hand position, action, speed, and movement using a hand gesture recognition camera (120).

12. In paragraph 9, The above kiosk program control step (S300) is: An information providing method (S1000) using a hand tracking kiosk, characterized in that the control unit receives information generated about the user's shape and hand gestures in the user detection step and the hand gesture recognition step, defines user information and hand gesture commands, and controls the kiosk program to output commands according to the set hand gestures and movements.

13. In paragraph 9, The above information output step (S400) is An information providing method (S1000) using a hand tracking kiosk characterized in that it provides details on menu movement and menu selection selected by a user through hand gestures and movements.

14. In paragraph 9, The above kiosk program is, When outputting information, it includes zoom in and zoom out functions. The above kiosk control step (S300) is An information providing method (S1000) using a hand tracking kiosk, characterized in that when it is recognized that a user has made a hand gesture and movement corresponding to a zoom in or zoom out function in the above hand gesture recognition step, information output on a display unit is output by zooming in or out.

15. In paragraph 9, The information provision method (S1000) using the above hand tracking kiosk is as follows: Repeating the above hand gesture recognition step, program control step and information output step until the user is no longer detected. Information provision method using a hand tracking kiosk featuring (S1000).

Images