Screen reader-based kiosk system to improve accessibility for the visually impaired

Abstract

This invention is a technology that improves ordering and payment systems utilizing NFC tags by adding accessibility features, thereby enabling visually impaired individuals to place orders more easily. Existing NFC-based ordering and payment systems had problems, such as users with low vision, including the visually impaired, being unable to navigate desired menus or pay for incorrectly selected products due to inappropriate feedback during the operation process, or having their free use restricted due to operation delays. To address this, the present invention provides an improved kiosk system characterized by storing a kiosk service address for the visually impaired in an NFC (Near Field Communication) tag, accessing the kiosk service by reading the service address stored in the user's mobile terminal, and providing voice feedback to the visually impaired by utilizing a screen reader function that reads the screen of the mobile terminal, thereby enabling the user to place an order by selecting a menu in an agreed manner.

Description

Technology Field

[0001] The present invention relates to a kiosk system characterized by storing a kiosk service address for the visually impaired in a Near Field Communication (NFC) tag, accessing the kiosk service by reading the service address stored in the user's mobile terminal, and providing voice feedback to the visually impaired by utilizing a screen reader function that reads the screen of the mobile terminal, thereby enabling the user to place an order by selecting a menu in a pre-arranged manner. In this process, the invention includes a technology that provides a menu interface that enhances ordering efficiency by removing images and using a condensed menu to reduce the number of interactions between the user and the screen reader. Background Technology

[0002] Currently commercialized kiosk systems are designed to facilitate menu navigation, ordering, and payment processes through visual feedback; however, the following issues are emerging. First, users with low vision, including the visually impaired, are unable to navigate their desired menus or end up paying for incorrectly selected items due to inappropriate feedback during operation. Second, delays experienced by the visually impaired during ordering queues cause unwanted inconvenience to other users in the line, which can lead to social stigma that restricts their independent and free use of kiosks. These problems result in restricted access to kiosks for the visually impaired, which is being pointed out as a form of comprehensive discrimination.

[0003] Furthermore, the inconsistent menu ordering across different locations where kiosks are installed causes inconvenience by increasing menu selection time for both visually impaired and non-disabled individuals. This not only lowers user convenience during the kiosk ordering process but also acts as a major factor in increasing waiting times.

[0004] Therefore, to solve these problems, the present invention proposes a kiosk system with enhanced accessibility by providing voice guidance compatible with screen readers, access to the kiosk ordering system via NFC tags, and a menu interface optimized for screen readers. Through this, accessibility for the visually impaired is improved during the kiosk operation process, and efficiency is ensured at the ordering stage, thereby enhancing user convenience for both disabled and non-disabled users.

[0005] [Prior Art Literature]

[0006] Existing kiosk systems have evolved to provide information to users and support payment functions by utilizing NFC tags and QR codes. They operate by attaching NFC tags or QR codes to kiosks or tables, allowing users to scan the tags with mobile devices and place orders and make payments based on the information. While these systems offer the advantage of enhancing ordering convenience through contactless interfaces, most interfaces are still designed around visual feedback, posing difficulties for users, such as the visually impaired.

[0007] In particular, since currently commercialized kiosk systems operate primarily through touchscreen interfaces, visual information must be recognized during menu navigation and ordering processes. Consequently, delays experienced by visually impaired users during operation have caused inconvenience to other users. Due to these limitations, the independent use of kiosks by the visually impaired has been restricted.

[0008] To address these issues, the present invention proposes a kiosk system that applies an NFC tag-based access method and provides screen reader-based voice guidance. This includes a user interface specialized for screen readers that removes pictorial elements and provides condensed menu information to enhance usability for the visually impaired. Through this, visually impaired individuals can use the kiosk independently without separate assistance, thereby providing more comprehensive user accessibility.

[0009] [Patent Literature]

[0010] Korean Patent No. 10-2545739 (June 15, 2023) describes a restaurant ordering system utilizing QR codes. This system is designed to allow customers to scan a QR code attached to a table to access a mobile webpage, where they can view the menu and place an order. However, there is a problem in that visually impaired individuals experience difficulty recognizing the visually provided QR codes.

[0011] Korean Patent No. 10-2207735 (January 20, 2021) describes an NFC-based store order management system designed to allow users to place orders and make payments via smartphones using NFC tags. However, the aforementioned kiosk system did not consider improving accessibility for the visually impaired, and its lack of voice guidance and feedback functions made independent use by the visually impaired difficult. Furthermore, because it is based on visual information provision and the interface elements are configured in a fragmented manner, it suffered from low compatibility with screen readers and an inefficient ordering process.

[0012] [Non-patent literature]

[0013] "It feels like knocking on a silent wall" ... People with disabilities face 'discrimination' with unmanned terminals. A news report (YTN, Oct. 15, 2021) points out the problem that people with disabilities, including the visually impaired, face significant difficulties using unmanned terminals such as kiosks. Kiosks have limitations, such as a lack of voice guidance and screen magnification functions, making them unusable by the visually impaired, and accessibility is further restricted if the buttons are small or broken.

[0014] The news article titled "Mandatory Installation of 10 Million Won Barrier-Free Kiosks... Small Business Owners Face Daunting Future Due to Cost Concerns" (Green Economy Newspaper, August 27, 2024) points out that while barrier-free kiosks have been developed to address the issue of digitally vulnerable groups, such as the visually impaired, having difficulty using kiosks, they are difficult to manage due to high initial and maintenance costs. Additionally, they are larger than existing kiosks, leading to spatial constraints and difficulties in maintenance and management. Consequently, it is realistically difficult to improve kiosk accessibility in all stores through barrier-free kiosks.

[0015] Screen readers are assistive technologies that read text and elements on a screen aloud for the visually impaired, and Apple’s VoiceOver and Google’s Talkback are widely used. A news report titled “Delivering and Shopping... Apps That Are Distant to the Visually Impaired” (MBC, Dec. 24, 2021) suggests that because these screen reader functions are not perfectly integrated with the kiosk system, there are still difficulties for the visually impaired to use the kiosk independently. The problem to be solved

[0016] The objective of the present invention is to provide a kiosk ordering system that utilizes a voice guidance function compatible with screen readers, a menu interface specialized for screen readers, a menu placement algorithm based on in-store order volume or individual preferences, and access via NFC tag functionality to solve the problems of existing kiosk systems. Through this, the aim is to ensure the efficiency of the ordering process by supporting visually impaired individuals to independently browse menus and place orders. means of solving the problem

[0017] In the present invention, the method for accessing a kiosk service from a mobile terminal is designed to be achieved through NFC tag recognition, and by a user recognizing an NFC tag attached to a kiosk or table in a store using a mobile terminal, menu information and service data of the kiosk are transmitted to the user end and provided along with an intuitive interface, which may include a screen reader, user-customized feedback, contactless ordering and payment, an operating server, a database, and an internet communication network.

[0018] In the present invention, a screen reader that reads the content of a screen on a mobile terminal may include a voice output function; a keyboard navigation support function; a web page structure analysis function; a text highlighting and reading order adjustment function; an image and graphic description function; a support function for various languages ​​and voice engines; a support function for application and document accessibility; a screen element navigation function; and shortcut keys and user-defined setting functions.

[0019] In order to provide an efficient menu configuration for the visually impaired, the operation method of a screen reader can be optimized. This may include methods such as removing menu photos and graphic elements to prevent redundant information, optimizing the menu interface to prevent unnecessary movement, displaying units for accurate transmission of quantity increase / decrease information, providing concise text for menus, and prioritizing the placement of frequently accessed menus. Additionally, the design may include optimized voice guidance, support for shortcut keys or voice commands, provision of vibration feedback, and a preferred menu saving function.

[0020] In the present invention, after a user recognizes an NFC tag through a voice guidance function, the process of viewing the menu and placing an order can be guided by voice. When a user scans an NFC tag, the mobile device automatically provides voice guidance on the menu description and selectable items, which may also include a function that allows the user to view and select a menu without providing redundant information through a graphic interface. Effects of the invention

[0021] The present invention provides the following effects: First, it enables visually impaired individuals to use a kiosk independently and overcomes the limitations of existing touchscreen-based kiosks, thereby supporting visually impaired individuals to place orders independently without separate assistance.

[0022] Second, convenient service access is possible through the use of NFC tags. Users can access kiosk services without any additional operations by scanning NFC tags using their mobile devices.

[0023] Third, accessibility for the visually impaired is improved based on voice feedback. Screen reader functions support visually impaired users in navigating menus and completing orders, while increasing the efficiency of the ordering process by eliminating inefficient visual information.

[0024] Fourth, AI-based text-to-speech (TTS) technology is applied. AI voice recognition more accurately recognizes user commands, and TTS is utilized to provide natural voice guidance.

[0025] Fifth, it provides contactless ordering and payment functions. By integrating with a mobile payment system, it enables ordering and payment without using a touchscreen, creating a hygienic ordering environment.

[0026] Sixth, it provides multilingual support and user-customized services. By supporting various languages, it enhances accessibility for foreign users and improves ordering convenience by offering personalized menu recommendations.

[0027] It maintains compatibility with existing kiosk systems. This system can be integrated with existing kiosk hardware and software, allowing for the application of accessibility enhancement features without the need for additional equipment replacement.

[0028] Through this, the present invention will enable various user groups, including the visually impaired, to use existing kiosk systems more conveniently and independently, and will contribute to maximizing accessibility and ease of use. Brief explanation of the drawing

[0029] Figure 1 is a configuration diagram of a voice response accessibility-based kiosk system. Figure 2 is a configuration diagram of a voice response accessibility-based kiosk application. Figure 3 is an example of the accessibility and touch mode menu selection process. Figure 4 is an example of voice response accessibility. Figure 5 shows database storage information. Figure 6 shows the procedure for using a voice response accessibility-based kiosk system. Specific details for implementing the invention

[0030] Hereinafter, the present invention will be described in more detail with reference to the attached drawings. In order to clarify the understanding of the present invention in the following description, descriptions of known technologies regarding the features of the present invention are omitted. The following embodiments are detailed descriptions intended to aid in understanding the present invention and are not intended to limit the scope of the rights of the present invention. Accordingly, equivalent inventions that perform the same function as the present invention will also fall within the scope of the rights of the present invention. In the following description, identical identification symbols denote identical configurations, and unnecessary redundant descriptions and descriptions of known technologies are omitted.

[0031] FIG. 1 is a schematic diagram of the configuration of the present invention, comprising a kiosk terminal (100), a mobile terminal (200), a service address storage unit (300), a database (400), an operation server (500), and a payment institution (600).

[0032] FIG. 2 is a block diagram showing the main components of an application of the present invention. The application is used on a mobile terminal (200) and consists of various functional modules that perform ordering and service provision based on the screen reader function. The application can be broadly divided into two main groups.

[0033] The first group consists of modules (210–240) related to user interface and accessibility, and the second group consists of modules (250–280) that manage the order and payment process. The user management unit (210) is responsible for user account management, access permission settings, and providing user-customized services. The service address recognition unit (220) performs the function of recognizing and storing store information stored in the service address storage unit (300), such as NFC tags, for the user's access to the system. The menu management unit (230) is configured to manage the menu of products and services provided by the kiosk and to provide them to the user. In this process, to maintain compatibility between the existing kiosk terminal (100) and the mobile terminal (200), menu information is stored in a cloud-based operation server, and menus are managed considering the characteristics of the terminal. The voice accessibility processing unit (240) processes information selected by the user through screen touch, such as menus and order options, by listening to the voice of a screen reader that reads the screen of a kiosk or mobile terminal, or receives the user's voice input and performs the function of recognizing information such as menus and order options as voice, and performs the function of interpreting and recognizing the user's voice into a designated language by applying AI-based voice recognition and synthesis technology according to the user's settings.

[0034] The order management unit (250) performs the function of processing user orders and storing and managing order history. The order payment unit (260) performs the function of processing user payment requests and supporting payment methods. The order guidance unit (270) is responsible for the function of notifying the user of the order progress status and providing the order number, order completion status, and estimated waiting time. The feedback management unit (280) performs the function of collecting user preferences for services and products, analyzing them, and managing improvements.

[0035] Figure 3 is an example of an interface specialized for screen readers. First, in Step 1, the user can select either accessibility mode or touch mode. If accessibility mode is touched, the voice accessibility module is activated, allowing ordering to be done by voice alone. If touch mode is selected, ordering can be done using motion with a screen reader. Next, in Step 2, the user can select either in-store or takeout, and in Step 3, the user can select the type of menu.

[0036] In this process, images were removed and essential information was used as a single tag to improve compatibility with screen readers. For example, by removing images of the menu in Step 4, duplicate voice guidance caused by image alternative text was prevented. At the same time, the menu name and price information were displayed simultaneously, allowing users to recognize key information about the menu without unnecessary operations.

[0037] Next, the system was simplified by setting each screen to perform only one task. Instead of navigating menus after selecting a category via the top bar on a single screen, a new category selection screen was added, assigning a single task to each screen so that the user could focus on only one task.

[0038] Finally, the clarity of meaning was enhanced by clarifying contextual terminology and feedback regarding user behavior. For example, when selecting the beverage temperature in Step 5, the words "warm" and "cold" were used to clarify the meaning. For steps involving detailed user selection (Steps 7, 8, and 9), statements such as "The selected option is 00" and "The selected quantity is 0" were specified to allow the user to reconfirm their selected option before proceeding to the next step.

[0039] Figure 4 illustrates the process of selecting a menu in a screen reader. (a) sets the focus when a double tap is pressed with one finger. (b) explains the movement of the focus, and (c) explains the screen scrolling function.

[0040] FIG. 5 illustrates the database structure of the present invention. The database (400) includes product information (410), order information (420), payment information (430), management information (440), and member information (450). Product information (410) represents detailed information about products sold at services utilizing the present invention, such as cafes and restaurants, and information regarding sales volume. Order information (420) refers to information ordered by a user through the system, such as order details, order number, and order status, such as '3 cups of iced Americano'.

[0041] Payment information (430) refers to information that manages the user's payment history, such as payment time and payment method, and management information (440) refers to information necessary for managing the system overall, such as store-specific management data, system operation logs, and maintenance history. Finally, member information (450) refers to information about the user who ordered the menu, such as member ID, password, and preference. Orders can be placed as either a member or a non-member, and if the user is a member, information about the menu they ordered is saved and reflected in the menu sorting when they use the service next time.

[0042] FIG. 6 shows a service implementation flowchart of the present invention and includes steps of membership registration (S710), login (S720), service address recognition (S730), menu configuration and display (S740), menu search and ordering (S750), additional ordering (S760), order confirmation (S770), order payment (S780), order number and preparation guidance (S790), and feedback management (S800).

[0043] In FIG. 6 above, membership registration (S710) is a step in which a user joins the service, and the information of the registered user is stored in the member information (450) of the database (400). Login (S720) is a step in which the registered user logs into their service, and if it fails, the procedure is terminated. Service address recognition (S730) is when the user accesses the service through a service address recognition unit (220), such as an NFC tag, to order a menu. Menu configuration and display (S740) sorts and displays the menus stored in the product information (410) in order of individual preference or sales volume within the member information (450) of the database (400). Menu search and ordering (S750) is when the user searches for and selects the displayed menu to place an order.

[0044] Additional Order (S760) is a step to check whether the user proceeds with ordering a menu other than the one selected. If an additional order is placed, the process returns to Menu Configuration and Display (S740) and repeats. Order Confirmation (S770) confirms the order by displaying the quantity, price, and option information of all menus selected by the user. Order Payment (S780) is a step to proceed with payment for the confirmed order through a payment institution, and the order details are stored in the order information (450) of the database (400). Order Number and Preparation Info (S790) displays the order number according to the order guide section (270) and provides information on the menu preparation status. The Feedback Management Section (S800) stores the menus and details ordered by the user in the member information (450) of the database (400) and manages and updates individual preferences.

[0045] [Example]

[0046] Embodiments of the present invention will be described in detail below. Embodiments of the present invention can be modified in various forms and are not limited to specific structures or functions.

[0047] (Voice guidance compatible with screen readers) In this invention, voice guidance is designed to be simple when a user uses a screen reader through a simple menu configuration. Users can receive real-time voice guidance on the currently selected situation, providing a simple and user-friendly environment for the visually impaired.

[0048] (Access to Kiosk Ordering System via NFC Tags) In the present invention, a user accesses an ordering and payment system by utilizing an NFC tag sticker. An order URL is encoded in each NFC tag, ensuring that the user is connected to the same URL upon tagging. When a user recognizes an NFC tag, the present invention enables convenient access to the kiosk ordering system by providing voice guidance for viewing the menu and the ordering process. Through this, the effect of reducing waiting lines and waiting times for kiosk usage can be expected.

[0049] (UI Optimized for Ordering - Minimized GUI) Unlike existing services, this invention does not provide a graphical interface for menus to the user, but instead presents the name and price as a single tag. This eliminates the need for unnecessary scrolling caused by graphical interfaces for the visually impaired, allowing users to receive all information regarding a menu (menu and price details) simultaneously with a single touch. This minimizes excessive operation and unnecessary voice guidance when using screen readers, enabling users to conveniently utilize the service. Furthermore, unnecessary recommendation menus and advertising banners have been removed, utilizing only essential elements so that users can focus solely on the ordering process.

[0050] (UI optimized for ordering - Single task-centered sequential structure) Through this invention, the user is required to perform only a single task on the screen, enabling the user to more clearly recognize which task needs to be performed on that screen. It is designed so that the user selects only one option, such as Ice or Hot, or Store or Takeout, whenever the page changes. This allows for the design of a service structure friendly to digitally vulnerable groups.

[0051] (UI Optimized for Orders - Response Feedback Based on Operation) In this invention, once a menu selection is completed, the user is immediately redirected to the shopping cart to verify the menu. It provides immediate response feedback to user actions without requiring additional user input, thereby preventing incorrect menu selections and payments.

[0052] (UI optimized for ordering - replacement of the top bar with low usability) In this invention, instead of placing categories on the top bar and listing menus at the bottom of a single screen, the screen is separated into two screens for category and menu selection. Users can view menus by selecting a category before entering the menu screen, and can change categories by returning to the previous category selection screen through the 'return to previous screen' action. This increases the efficiency of operation by reducing the repetitive top bar navigation required to change categories when using screen reader functions. Industrial applicability

[0053] The present invention is a screen reader-based kiosk system for improving accessibility for the visually impaired and can be utilized in the following fields. First, the food service and restaurant industry: The present invention can be utilized in various food service industries such as restaurants, cafes, and food courts, providing a user-friendly environment where all users, including the visually impaired, can place orders using the same kiosk. In particular, the present invention ensures accessibility and ease of use through the same kiosk without the need to install separate barrier-free kiosks for non-disabled and visually impaired users. This increases the efficiency of kiosk management and maintenance, while simultaneously creating an environment where all users, including the visually impaired, can equally utilize services, thereby enhancing inclusivity and customer satisfaction in the food service industry.

[0054] Second, public services and public institutions: Based on NFC and voice guidance, this invention supports visually impaired individuals and vulnerable groups facing difficulties in accessing information to use public services more easily and conveniently. By providing equal opportunities for use to diverse users through this technological foundation and enhancing the equity and accessibility of public services, it can contribute to the realization of social value.

[0055] Third, in the retail and distribution sector: By applying the present invention to convenience stores, supermarkets, shopping centers, etc., customers can check product information and proceed with ordering and payment through voice guidance. In particular, it can resolve the problem where visually impaired individuals pay more than their ordered items because they cannot read the product prices or quantities displayed on kiosks in unmanned stores. Through this, an environment is provided in which digitally vulnerable groups, including the visually impaired, can proceed with the payment process with greater trust and security.

[0056] As such, the present invention can be applied in various industrial fields and has a substantial effect in contributing to the activation of accessibility technology and the establishment of an inclusive user environment. Explanation of the symbols

[0057] 100 : Kiosk terminal 110: Voice Input / Output Unit 200 : Mobile device 210 : User Management Department 220 : Service address recognition unit 230 : Menu Management Department 240 : Voice Accessibility Processing Unit 250 : Order Management Department 260 : Order and Payment Department 270 : Order Guide 280 : Feedback Management Department 300: Service address storage 310 : NFC Tag Sticker 400 : Database 410 : Product Information 420 : Order Information 430 : Payment Information 440 : Management Information 450 : Member Information 500 : Production Server 600 : Payment institution 610 : Bank 620 : Credit card company S710 : Sign Up S720 : Login S730: Service Address Recognition S740: Menu Configuration and Display S750: Menu navigation and ordering S760 : Additional Order S770 : Order Confirmed S780 : Order Payment S790: Order Number and Preparation Guide S800: Feedback Management Department

Claims

Claim 1 A kiosk system comprising a kiosk terminal (100), a mobile terminal (200), a service address storage unit (300), a database (400), an operation server (500), and a payment institution (600), wherein the system comprises: a plurality of kiosk terminals (100) installed by a service provider to allow a user to purchase goods or various services; a mobile terminal (200) that provides a function for a user to select a menu and place an order; a kiosk service address storage unit (300), such as an NFC tag containing a kiosk service address; a database (400) that includes sales information, order information, payment information, management information, membership information, etc.; an operation server (500) that supports the operation, maintenance, performance improvement, etc. of services in the form of a web or app on the kiosk terminal or mobile terminal; or a payment institution (600); wherein the user management unit (210) of the kiosk terminal (100) and the mobile terminal (200) provides user account management, access permission settings, and user-customized services in the form of a web or app; The service address recognition unit (220) recognizes and stores store information for the user's access to the system; the menu management unit (230) manages the menu of products and services provided by the kiosk; the voice accessibility processing unit (240) supports user menu selection and multilingual voice recognition based on a screen reader; the order management unit (250) processes the user's order and stores and manages order history; the order payment unit (260) processes the user's payment request and supports payment methods; the order guidance unit (270) guides the user on the order progress status and provides the order number, order completion status, and estimated waiting time; the feedback management unit (280) collects preferences for services and products from the user, analyzes them, and manages UI improvements optimized for orders;A screen reader-based kiosk system characterized by improved accessibility for the visually impaired using the functions of the above-mentioned kiosk terminal (100), mobile terminal (200), service address storage unit (300), database (400), operation server (500), and payment institution (600).; Claim 2 A screen reader-based kiosk system characterized by implementing a function to improve accessibility for the visually impaired using the modules, wherein, in claim 1 above, a service address recognition unit (220) reads information stored in a service address storage unit (300), such as an NFC tag, and connects it to a service address; a menu management unit (230) that supports linkage with existing kiosk hardware and software through cloud-based menu information management; a voice accessibility processing unit (240) that processes information selected by the user via screen touch after listening to the voice of a screen reader that reads the screen of a kiosk or mobile terminal, or performs a function to recognize information such as menu selection and order options as voice by receiving the user's voice input, and applies AI-based voice recognition and synthesis technology according to the user's settings to interpret and recognize the user's voice into a designated language; an order management unit (250) that supports ordering and payment without using a touchscreen by linking with a mobile payment system.

Images