Kiosk system capable of discriminating digital illiterate, and operation method thereof

Abstract

The present invention relates to a kiosk system capable of screening digitally illiterate people and a method of operating the same. A kiosk system capable of screening digital illiterate people according to the present invention comprises: a main body portion forming the overall body of the kiosk system; a display portion that displays an initial screen having various function buttons, including a menu selection button, according to a program for operating the kiosk system, and displays a corresponding result on the screen according to touch input by a kiosk user; a camera that captures the face of the kiosk user and tracks the pupils by eye tracking; and a control portion that estimates the age of the user and determines the emotional state of the user using a deep learning algorithm based on data captured and tracked by a plurality of cameras, and if the user's eye movement is not in a certain pattern or stops for a certain period of time or longer, or if the menu selection time is delayed for a certain period of time or longer, identifies the user as digitally illiterate and provides a kiosk UI dedicated to digitally illiterate people. According to the present invention, even digitally illiterate people can use the kiosk (signage) system without much difficulty.

Description

Technology Field

[0001] The present invention relates to a kiosk system capable of screening digitally illiterate individuals, and more specifically, to a kiosk system capable of screening digitally illiterate individuals and providing services that show consideration for digitally illiterate individuals, and a method of operating the same. Background Technology

[0003] Kiosks are unmanned automated systems installed in stores or public places for information delivery or transactions, and their application is on the rise due to reduced labor costs and the alleviation of the burden of face-to-face service. Currently, kiosks have expanded from service sectors such as banks, subways, and airports to restaurants, shopping malls, movie theaters, hospitals, and unmanned stores.

[0004] However, digitally illiterate individuals often become confused by kiosks that offer a variety of menu options, leading to a problem where they avoid kiosk stores.

[0005] In particular, when there is a long line, digitally illiterate people become conscious of the gaze of those behind them, and if they fail to place an order, they develop trauma regarding kiosks and tend to dislike kiosks very much.

[0006] While the adoption of kiosks is naturally expected to expand due to their ability to reduce the burden of face-to-face service, cut labor costs, and save time and money for operators, widespread adoption may be difficult without consideration for digitally illiterate individuals.

[0007] Meanwhile, Korean Registered Patent Publication No. 10-2253279 (Patent Document 1) discloses "a kiosk-based unmanned payment system using artificial intelligence and a method thereof." The kiosk-based unmanned payment system according to this invention comprises: a kiosk that displays a menu through a display, receives an order input from a user via touch or voice, and displays the order details; and a server that analyzes order information input from a user through the kiosk and transmits the analysis results to the kiosk. The server is characterized by being configured to detect whether a user is present in front of the kiosk based on human recognition artificial intelligence, predict the age group of the user detected through the human recognition artificial intelligence based on age prediction artificial intelligence, analyze the user's voice based on voice recognition artificial intelligence, and engage in two-way conversation with the user via voice through the kiosk.

[0008] In the case of Patent Document 1 described above, although it has the advantage of providing an efficient and accurate unmanned payment system by being configured to have two-way conversations with the user via voice through a kiosk, it also contains the problem of not sufficient consideration for digitally illiterate people. Prior art literature

[0009] Korean Registered Patent Publication No. 10-2253279 (Published on May 18, 2021) The problem to be solved

[0010] The present invention was created by comprehensively taking into account the above-mentioned matters, and aims to provide a kiosk system capable of screening digitally illiterate individuals and a method of operating the same, which can screen digitally illiterate individuals and provide services that show consideration for them. means of solving the problem

[0012] To achieve the above objective, the kiosk system capable of screening digital illiterate people according to the present invention is,

[0013] A main body part that forms the overall body of a kiosk system (device) and is configured in the form of a rectangular upright structure;

[0014] A display unit provided on the upper front half of the main body, which displays an initial screen having various function buttons including a menu selection button according to a program for operating a kiosk system (device), and displays a corresponding result on the screen according to touch input by a kiosk user;

[0015] A plurality of cameras are installed on a predetermined external portion of the main body, and the cameras capture the face of the kiosk user and track the pupils using eye tracking; and

[0016] The invention is characterized by including a control unit that is installed inside the main body and, based on data captured and tracked by the plurality of cameras, estimates the age of the user using a deep learning algorithm, determines the emotional state of the user, identifies the user as digitally illiterate if the user's eye movement is not in a certain pattern or stops for a certain period of time or if the menu selection time is delayed for a certain period of time or longer, and provides a kiosk UI (user interface) dedicated to digitally illiterate users.

[0017] Here, two cameras are installed on the upper part of the main body, and among the two cameras, the first camera captures the user's face, and the second camera is configured to track the pupils using eye tracking with dlib (distribution library).

[0018] In addition, the control unit can estimate the user's age and determine the user's emotional state by applying deep face technology to face recognition.

[0019] In addition, the control unit identifies the user as digitally illiterate and provides a kiosk UI (user interface) dedicated to digitally illiterate users. The UI may be configured to allow digitally illiterate users to select a menu in a question-and-answer or multiple-choice format.

[0020] In addition, to achieve the above objective, the method of operating a kiosk system capable of screening digital illiterate people according to the present invention is,

[0021] A method of operating a kiosk system in which a plurality of cameras are installed on the exterior of the main body and a deep learning algorithm (program) is installed in a control unit installed inside the main body,

[0022] a) A step in which a control unit operates the deep learning algorithm (program);

[0023] b) A step in which the control unit determines whether the user's age is above a preset age range based on the user's face captured by the camera;

[0024] c) a step in which, if the user’s age is above a preset age range in the determination of step b) above, the control unit determines whether the user’s face is in a preset emotional state;

[0025] d) If, in the determination of step c) above, the user's face is in a preset emotional state, the control unit determines whether the pattern of the user's reading eye movements is not a preset pattern based on data tracked by the camera;

[0026] e) a step in which, if the pattern of the eye movement for reading text in the determination of step d) above is not a preset pattern, the control unit determines whether the user's eye movement has stopped for a certain period of time or longer; and

[0027] f) The invention is characterized by including a step in which, if the user's eye movement remains stationary for a certain period of time or longer during the determination in step e) above, the control unit determines the user to be digitally illiterate and provides a kiosk UI (user interface) dedicated to digitally illiterate users.

[0028] Here, in step c), the preset emotional state may be an emotional state of surprise and / or anger.

[0029] In addition, in step f) above, the kiosk UI (user interface) dedicated to digital literacy may be a method that allows digital literacy to select a menu by using a question-and-answer or multiple-choice method. Effects of the invention

[0031] According to the present invention, by using a camera to photograph the face of a kiosk user, estimating the user's age and emotional state based on the photographed face image, and tracking the user's eye movements to identify digitally illiterate users, and then switching to a question-and-answer or multiple-choice menu selection function to select a menu, there is an advantage that even digitally illiterate users can use the kiosk (signage) system without much difficulty. Brief explanation of the drawing

[0033] FIG. 1 is a drawing showing the exterior of a kiosk system capable of screening digital illiterate people according to the present invention. FIG. 2 is a schematic diagram showing the configuration of a kiosk system capable of screening digital illiterate people according to the present invention. FIG. 3 is a flowchart illustrating the execution process of an operation method of a kiosk system capable of screening digital illiterate people according to the present invention. Figure 4 is a diagram showing an overview of determining age and emotion by photographing a user's face to detect digital illiteracy. Figure 5 is a diagram showing an eye tracking pattern and the user's eye tracking results. Figure 6 is a diagram showing an example of checking whether digital illiteracy is detected. Figure 7 is a diagram showing the experimental results regarding the concordance rate of surprise and anger emotions in people aged 50 and older. Specific details for implementing the invention

[0034] Terms and words used in this specification and claims should not be interpreted as being limited to their ordinary or dictionary meanings, and should be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor can appropriately define the concept of the terms to best describe his invention.

[0035] Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "…part," "…unit," "module," and "device" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0036] Embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0037] FIG. 1 is a drawing showing the exterior of a kiosk system capable of screening digital illiterate people according to an embodiment of the present invention, and FIG. 2 is a drawing schematically showing the configuration of a kiosk system capable of screening digital illiterate people according to an embodiment of the present invention.

[0038] Referring to FIGS. 1 and 2, a kiosk system (100) capable of screening digital illiterate people according to the present invention may be configured to include a main body part (110), a display part (120), a camera (130), and a control part (140).

[0039] The main body (110) is a part that forms the entire body of the kiosk system (device) and is configured in the form of an upright structure that is roughly rectangular.

[0040] The display unit (120) is provided in the upper front half of the main body unit (110) and displays an initial screen having various function buttons, including a menu selection button, according to a program for operating the kiosk system (device), and displays a corresponding result on the screen according to touch input by a kiosk user.

[0041] A plurality of cameras (130) (e.g., two) (130a, 130b) are installed on a specific external portion of the main body (110) (e.g., the upper portion of the main body (110)), and the cameras photograph the face of the kiosk user and track the pupils using eye tracking. Here, among the two cameras installed on the upper portion of the main body (110), the first camera (130a) may be configured to photograph the user's face, and the second camera (130b) may be configured to track the pupils using eye tracking with a dlib (distribution library).

[0042] The control unit (140) is installed inside the main body (110) and estimates the user's age and determines the user's emotional state using a deep learning algorithm based on data captured and tracked by the plurality of cameras (130a, 130b). If the user's eye movement does not follow a certain pattern, stops for a certain period of time (e.g., 3 seconds), or if the menu selection time is delayed for a certain period of time (e.g., 5 seconds), the user is identified as digitally illiterate and a kiosk UI (user interface) dedicated to digitally illiterate users is provided. Here, the control unit (140) may be equipped with a program (140p) for face recognition and eye tracking. Additionally, when the control unit (140) estimates the user's age and determines the user's emotional state, the user's age and emotional state can be estimated and determined using face recognition with deep face technology applied. In addition, when the control unit (140) identifies the user as digitally illiterate and provides a kiosk UI (user interface) dedicated to digitally illiterate users, the UI may be configured to allow digitally illiterate users to select a menu in a question-and-answer or multiple-choice format. For example, it may be configured to allow users to select one of two or three menus.

[0043] In FIG. 2, the unexplained reference numeral 150 represents an input device and sensor, which may be configured to include an NFC / RFID reader and a QR code scanner. Additionally, 160 represents a power supply.

[0044] Then, below, a method of operating a kiosk system capable of screening digital illiterate people according to the present invention having the configuration as described above will be explained.

[0045] FIG. 3 is a flowchart illustrating the execution process of an operation method of a kiosk system capable of screening digital illiterate people according to an embodiment of the present invention.

[0046] Referring to FIG. 3, the method of operating a kiosk system capable of screening digital illiterate people according to the present invention is a method of operating a kiosk system (100) in which a plurality of cameras (130a, 130b) are installed on the outside of a main body part (110) and a deep learning algorithm (program) is installed in a control unit (140) installed inside the main body part (110), wherein first, the control unit (140) operates the deep learning algorithm (program) (step S301).

[0047] Then, the control unit (140) determines whether the user's age is older than a preset age range (e.g., 50s) based on the user's face captured by the camera (i.e., the first camera (130a)) (step S302). Here, the preset age range is not necessarily limited to 50s, and may be set to a different age range depending on the case.

[0048] In the determination of step S302 above, if the user's age is above a preset age range, the control unit (140) determines whether the user's face is in a preset emotional state (step S303). Here, the preset emotional state may be an emotional state of surprise and / or anger.

[0049] Here, in addition, in steps S302 and S303, when the control unit (140) estimates the user's age and determines the user's emotional state, the user's age and emotional state can be estimated and determined by face recognition using deep face technology.

[0050] If, in the determination of step S303 above, the user's face is in a preset emotional state, the control unit (140) determines whether the pattern of the user's reading eye movements is not a preset pattern (e.g., an F-pattern or a Z-pattern) (see FIG. 5 (a) and (b)) based on data tracked by the camera (i.e., the second camera (130b)) (step S304).

[0051] If, in the determination of step S304 above, the pattern of the eye movement for reading is not a preset pattern (F-pattern or Z-pattern), the control unit (140) determines whether the user's eye movement has stopped for a certain period of time (e.g., 3 seconds or 5 seconds) or longer (step S305).

[0052] In the determination of step S305 above, if the user's eye movement is stopped for a certain period of time or longer, the control unit (140) determines that the user is digitally illiterate and provides a kiosk question-and-answer type UI (user interface) dedicated to digitally illiterate people (step S306). Here, such a kiosk question-and-answer type UI (user interface) dedicated to digitally illiterate people may be a method that allows digitally illiterate people to select a menu by using a question-and-answer format or a multiple-choice format.

[0053] Below, we will provide further explanation regarding the kiosk system capable of screening digitally illiterate people and the method of operation thereof according to the present invention as described above.

[0054] Figure 4 is a diagram showing an overview of determining age and emotion by photographing a user's face to detect digital illiteracy.

[0055] Referring to FIG. 4, when the kiosk system (100) of the present invention is operated, the user's face is captured by a camera (first camera (130a)) as in (a), and based on this, the control unit (140) estimates the user's age and determines the emotional state (surprise or anger) as in (b).

[0056] Figure 5 is a diagram showing an eye tracking pattern and the user's eye tracking results.

[0057] Referring to FIG. 5, the present invention introduces an F-pattern as shown in (a) and a Z-pattern as shown in (b) for the detection of digital illiterate people. (c) is the result of recognizing the F-pattern for a kiosk user, (d) is the result of recognizing the Z-pattern, and (e) is the result of recognizing other patterns other than the F-pattern or Z-pattern. The control unit (140) determines that a user who shows the F-pattern of (c) or the Z-pattern of (d) is not digitally illiterate, and determines (detectes) that a user who shows other patterns other than the F-pattern or Z-pattern as shown in (e) is digitally illiterate as shown in FIG. 6.

[0058] Figure 7 is a diagram showing the experimental results regarding the concordance rate of surprise and anger emotions in people aged 50 and older.

[0059] Referring to Figure 7, among subjects aged 50 and older, 55 out of 81 (67.90%) were judged to be in a surprised emotional state in the surprise category, and 244 out of 306 (79.74%) were judged to be in an angry emotional state in the angry category. From this, it can be seen that the accuracy of judgment regarding the angry emotional state is higher than that regarding the surprised emotional state.

[0061] As described above, the kiosk system capable of screening digital illiterate people and the method of operation thereof according to the present invention have the advantage that even digital illiterate people can use the kiosk (signage) system without difficulty by using a camera to photograph the face of a kiosk user, estimating the age and emotional state of the user based on the captured face image, and tracking the user's eye movements to screen digital illiterate people, and then switching to a question-and-answer or multiple-choice menu selection function to select a menu.

[0062] Although the present invention has been described in detail through preferred embodiments, the invention is not limited thereto, and it is obvious to those skilled in the art that various modifications and applications can be made within the scope of the technical concept of the invention. Accordingly, the true scope of protection of the present invention should be interpreted by the following claims, and all technical concepts within an equivalent scope should be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0063] 100: (The present invention) Kiosk system capable of screening digitally illiterate people 110: Main body 120: Display unit 130: Camera 130a: First Camera 130b: Second camera 140: Control unit 150: Input devices and sensors 160: Power supply

Claims

Claim 1 A main body portion comprising the overall body of a kiosk system (device) configured in the form of a rectangular upright structure; a display portion provided on the upper front half of the main body portion, which displays an initial screen having various function buttons including menu selection buttons according to a program for operating the kiosk system (device), and displays corresponding results on the screen according to touch input by a kiosk user; and a plurality of cameras installed on a predetermined external part of the main body portion, which photograph the face of the kiosk user and track the pupils using eye tracking. A kiosk system capable of screening digital illiterate individuals, comprising a control unit installed inside the main body and, based on data captured and tracked by the plurality of cameras, using a deep learning algorithm to estimate the age of the user and determine the emotional state of the user, and if the user's eye movement does not follow a certain pattern or stops for a certain period of time or if the menu selection time is delayed for a certain period of time or longer, the control unit identifies the user as digitally illiterate and provides a kiosk UI (user interface) dedicated to digital illiterate individuals; wherein two cameras are installed on the upper part of the main body, and among the two cameras, the first camera captures the user's face, and the second camera is configured to track the pupils using eye tracking with a dlib (distribution library); wherein the control unit is equipped with a program for face recognition and eye tracking; and wherein, in the control unit identifying the user as digitally illiterate and providing a kiosk UI (user interface) dedicated to digital illiterate individuals, the UI is configured to allow the digital illiterate individual to select a menu using a question-and-answer or multiple-choice method. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete

Images