Cloud-based artificial intelligence barrier-free universal kiosk platform

Abstract

The present invention relates to a cloud-based artificial intelligence barrier-free universal kiosk platform and, more specifically, to a cloud-based artificial intelligence barrier-free universal kiosk platform which may identify the type of a user and provide an interface suitable for the type of the user so as to perform communication suitable for user convenience.

Description

Cloud-based AI barrier-free universal kiosk platform

[0001] The present invention relates to a cloud-based artificial intelligence barrier-free universal kiosk platform, and more specifically, to a cloud-based artificial intelligence barrier-free universal kiosk platform that identifies the type of user and provides an interface suited to the type of user, thereby enabling communication suited to the user's convenience.

[0002] Currently, the global kiosk technology market is centered on hardware technology, which accounts for about half of total sales, but the proportion of system integration and kiosk application technology is expected to increase in the future.

[0003] Currently, kiosks providing various services are installed in various locations to provide convenience to users, but kiosk technology that takes the disabled into consideration is very inadequate. In the case of the United States, for example, convenience for the disabled is promoted through regulations on shape rather than improving functional convenience, so barrier-free technology that incorporates welfare ICT technology is minimal.

[0004] In both domestic and international markets, there is a need for a platform through kiosks that provide services for various types of disabilities, including visual, physical, hearing, and complex disabilities, but there are not many specialized kiosk companies that can meet this need.

[0005] Therefore, there is a need for a platform that provides kiosks and services for the disabled, and in particular, a platform that can provide services through kiosks that can provide communication interfaces appropriate for the user's disability type (visual / physical / hearing) is needed.

[0006] The purpose of the present invention is to provide a cloud-based artificial intelligence barrier-free universal kiosk platform that can provide a communication method suited to the user type by taking into account the user type.

[0007] Another purpose of the present invention is to provide a cloud-based artificial intelligence barrier-free universal kiosk platform that can provide various customized information to all citizens, including socially vulnerable groups such as the disabled, the elderly, and children, by utilizing kiosks, and can efficiently provide payment / ticketing / guidance services to all private companies and public institutions.

[0008] Another object of the present invention is to provide a cloud-based artificial intelligence barrier-free universal kiosk platform that provides welfare ICT technology to support the daily lives of persons with disabilities by enabling the expansion and utilization of services for persons with disabilities to kiosks and mobile devices.

[0009] The present invention provides a cloud-based artificial intelligence barrier-free universal kiosk platform, comprising: a cloud server that operates a universal kiosk platform that receives user information from a kiosk, provides a user type interface suitable for a user type, communicates with the user through the corresponding user type interface using artificial intelligence to identify a user request, and processes the user request through linkage with a service agency that performs the user request; and a kiosk that recognizes the user's approach and the user's disability, mediates communication between the cloud server and the user, and provides a user interface suitable for the user type using a multi-interface device to provide a service suitable for the user request.

[0010] Here, the cloud server may be characterized by including a server communication unit that performs communication with the kiosk and other system devices; a communication processing unit that generates and provides communication information suitable for the user type interface and analyzes communication information performed with the user; a service agency linkage unit that performs linkage with a service agency that processes the user request; and a universal operation processing unit that controls the functions of each component of the cloud server, analyzes user information received from the kiosk to determine the type of disability of the user, processes linkage with an external platform that provides an additional method necessary for communication with the user, and obtains a method necessary for communication, and manages and controls the kiosk.

[0011] Here, the communication processing unit may be characterized by including an image algorithm processing unit that recognizes and processes the user's gestures and actions for communication with the user; a sign language algorithm processing unit that recognizes and processes the user's sign language for communication with the user; a voice algorithm processing unit that recognizes and processes the user's voice for communication with the user; and a Braille algorithm processing unit that recognizes and processes Braille for communication with the user.

[0012] Here, the image algorithm processing unit may be characterized in that, if the user is one of a mentally or physically disabled person, an elderly person, or a child, a learning dataset is constructed that classifies the gesture and behavior patterns of the user, and the user's gestures and behaviors are corrected and processed.

[0013] Here, the sign language algorithm processing unit may be characterized in that it generates an avatar suitable for the user by considering the user information and processes sign language with the user.

[0014] Here, the voice algorithm processing unit may be characterized in that it processes the user's voice by performing noise compensation suitable for the environment of the kiosk by considering noise according to the environment of the kiosk.

[0015] Here, the kiosk may be characterized by including a user recognition unit that includes at least one of an image processing device and a proximity sensor to recognize the user's approach and disability; a multi-interface unit that includes at least one of a display device, a sound device, and a Braille block device to perform information exchange with the user through at least one of the devices; a kiosk communication unit that communicates with the cloud server; a height adjustment unit that can adjust the height according to the user's physical condition according to the disability; and a service provision unit that provides a service requested by the user.

[0016] Here, it may be characterized by further including a user terminal communication unit that performs communication with a user terminal and transmits and receives communication information provided by the cloud server to the user terminal.

[0017] The present invention has the effect of improving kiosk accessibility and usability for the disabled and elderly, thereby providing a digital-based barrier-free universal kiosk platform for all citizens, including vulnerable groups.

[0018] The present invention has the effect of contributing to the growth of related industries and the spread of welfare ICT technology by providing open SW and a standard interface that are easy to expand and distribute to private sector kiosks (ordering / ticketing / information provision).

[0019] The present invention has the effect of providing a low-cost and general-purpose universal kiosk service through a customized interface that meticulously considers the physical and psychological inconveniences of the socially disadvantaged by improving information accessibility and digital usability / convenience for the digitally vulnerable based on kiosk.

[0020] FIG. 1 is a diagram showing the configuration of a cloud-based artificial intelligence barrier-free universal kiosk platform as an embodiment of the present invention.

[0021] FIG. 2 is a drawing for explaining the main areas of the cloud-based artificial intelligence barrier-free universal kiosk platform of the present invention.

[0022] FIG. 3 is a drawing showing the configuration of a kiosk according to one embodiment of the present invention.

[0023] FIG. 4 is a diagram illustrating the configuration of a cloud server according to one embodiment of the present invention.

[0024] FIG. 5 is a diagram illustrating the configuration of a communication processing unit according to one embodiment of the present invention.

[0025] FIG. 6 is a diagram illustrating one embodiment of the present invention for processing communication with a user by performing sign language through an avatar.

[0026] FIG. 7 is a diagram illustrating one embodiment of the present invention for processing communication with a user through voice.

[0027] FIG. 8 is a diagram illustrating a method for recognizing a user and determining a user's disability according to one embodiment of the present invention.

[0028] The advantages and features of the present invention and the method for achieving them will become clear with reference to the embodiments described in detail below together with the attached drawings.

[0029] However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms.

[0030] The embodiments in this specification are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention.

[0031] And the present invention is defined only by the scope of the claims.

[0032] Accordingly, in some embodiments, well-known components, well-known operations, and well-known techniques are not specifically described to avoid obscuring the present invention.

[0033] Additionally, like reference numerals throughout the specification refer to like elements, and the terms used (referred to) herein are for the purpose of describing embodiments and are not intended to limit the present invention.

[0034] In this specification, the singular includes the plural unless specifically stated otherwise in the phrase, and the reference to an element or action as “including (or comprising)” does not exclude the presence or addition of one or more other elements or actions.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the sense commonly understood by a person of ordinary skill in the art to which the present invention belongs.

[0036] Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless they are defined otherwise.

[0037] Hereinafter, a preferred embodiment of the present invention will be described with reference to the attached drawings.

[0038]

[0039] FIG. 1 is a diagram showing the configuration of a cloud-based artificial intelligence barrier-free universal kiosk platform as an embodiment of the present invention.

[0040] The cloud-based artificial intelligence barrier-free universal kiosk platform of the present invention is configured to include a kiosk (200), which is a client device, and a cloud server (300) that communicates with a user (10) through the kiosk (200) and provides a service requested by the user (10).

[0041] The kiosk (200) can recognize the user's (10) access and the user's disability, and provides an interface suitable for the user's disability type to mediate communication between the cloud server (300) and the user.

[0042] The kiosk (200) includes a multi-interface device, and provides an interface suitable for the user's (10) type by using a device suitable for the user's (10) type among the multi-interface devices.

[0043] The cloud server (300) receives user information from the kiosk (200) and provides a user interface tailored to the user's type. Furthermore, it communicates with the user through an interface tailored to the user's type, utilizing artificial intelligence. Subsequently, the cloud server identifies the user's needs through communication and, through linkage with a service provider that fulfills the user's request, provides the user with the service they desire.

[0044] Here, the user type refers to the type of disability of the user (10) if the user is disabled. For example, it refers to a visually impaired person, a hearing impaired person, a speech impaired person, a disabled person using a wheelchair, etc.

[0045] Additionally, user type can mean male, female, elderly, children, foreigner, etc.

[0046] Therefore, the user type interface is an interface for communication that is appropriate for the user's disability type. For example, if the user is visually impaired, it may be an interface for communication through Braille. If the user is hearing or speech impaired, it may be an interface for communication through text or sign language. If the user is a foreigner, it may be an interface for communication through a language appropriate for the foreigner.

[0047] That is, the kiosk (200) recognizes the type of user (10) and the cloud server (300) provides a communication means suitable for the user type interface to communicate with the user and provide the service requested by the user.

[0048] Here, communication means suitable for the user type interface include communication SW through Braille, communication SW through sign language, communication SW through voice, SW that recognizes gestures, SW that recognizes images, and communication SW through foreign languages.

[0049]

[0050] FIG. 2 is a drawing for explaining the main areas of the cloud-based artificial intelligence barrier-free universal kiosk platform of the present invention.

[0051] The operation of the cloud-based artificial intelligence barrier-free universal kiosk platform of the present invention can be divided into a kiosk area, a cloud operation area, and a service linkage area.

[0052] The kiosk area is an area that directly provides an interface means suitable for the type of user (10) performing operations at the kiosk (200), and provides the user with an interface for using voice, sign language, gestures, video, text, etc. depending on the type of user, and ultimately provides the user with a service received from an institution that performs the service requested by the user (e.g., guidance, purchase, reception, etc.).

[0053] The cloud operation area is an area where communication is performed through an interface suitable for the user type performed on the cloud server (300). It is an area where communication with the user is performed through various artificial intelligence communication methods such as voice recognition SW, sign language recognition SW, non-contact / gesture recognition SW, and image recognition SW depending on the user type.

[0054] The service linkage area is an area that performs services requested by users by identifying user needs through communication with users and performing linkage with external organizations. Depending on the service requested by the user, linkage with various service providers can be performed to perform the service requested by the user.

[0055]

[0056] FIG. 3 is a drawing showing the configuration of a kiosk according to one embodiment of the present invention.

[0057] The kiosk (200) of the present invention includes a user recognition unit (210), a multi-interface unit (220), a kiosk communication unit (230), a height adjustment unit (240), and a service provision unit (250).

[0058] The user recognition unit (210) includes one or more devices capable of recognizing a user, such as an image processing device, a proximity sensor, or an ultrasonic sensor, and performs the function of recognizing the user's approach and obstacles.

[0059] For example, when a user approaches a kiosk (200), the user's image is recognized, or the user's approach is recognized through an ultrasonic sensor or a proximity sensor, and when the user logs in or performs an action to use the kiosk, it is possible to determine whether the user has a disability through the user's image, and if so, what type of disability the user has.

[0060] The multi-interface unit (220) includes various devices for information exchange for precise communication with the user (10), thereby providing the user with an information exchange device suitable for the user's type of disability. For example, the multi-interface unit (220) includes at least one of a display device, a sound recognition and generation device, and a Braille block device, thereby providing a device for information exchange desired by the user (10) depending on the user's (10's) type of disability.

[0061] The kiosk communication unit (230) performs a function of communicating with the cloud server (300).

[0062] The height adjustment unit (240) is a device that can adjust the height of the kiosk (200), and performs the function of adjusting the height of the kiosk (200) to suit the physical condition of the user's disability.

[0063] The service provider (250) performs the function of providing the service requested by the user (200). Here, the service provider (250) may directly perform the user's request, but if the user's request is a task that can only be performed through an external organization, such as a task through a government agency, a banking task, or a reservation task through a specific organization, the service provider (250) performs the function of simply informing or presenting the task that has been performed and processed by linking with the external organization through a cloud server (300).

[0064] If multiple users wish to use the same kiosk (200), the kiosk (200) may further include a user terminal communication unit (260) to perform communication with the user's terminal and provide the service provided by the present invention to multiple users simultaneously.

[0065]

[0066] FIG. 4 is a diagram illustrating the configuration of a cloud server according to one embodiment of the present invention.

[0067] The cloud server (300) of the present invention includes a server communication unit (310), a communication processing unit (320), a service agency linking unit (330), and a universal operation processing unit (340).

[0068] The server communication unit (310) performs a function of communicating with the kiosk (200) and other system devices.

[0069] The communication processing unit (320) generates communication information suitable for the user type interface and provides it to the user through the kiosk (200).

[0070] In addition, the communication processing unit (320) performs the function of processing algorithms for various communication methods and analyzes communication information performed with the user.

[0071] The service agency linkage unit (330) performs a function of performing a service requested by a user by linking with a selected service agency that processes a user request.

[0072] To this end, the service agency linkage unit (330) can perform linkage with relevant external agencies through APIs (Application Programming Interfaces) provided by various agencies.

[0073] The universal operation processing unit (340) controls the functions of each component of the cloud server and analyzes user information received from the kiosk (200) to determine the user's disability type. Furthermore, if additional methods are required for communication with the user (10), the universal operation processing unit (340) processes linkage with an external platform that provides the corresponding communication method to obtain the method required for communication and performs a control function to communicate with the user through the communication processing unit (320).

[0074] For example, if the user is a foreigner, communication with the user can be performed by linking with an external platform that analyzes and processes speech in the foreign language through the communication processing unit (320), and if the user uses a foreign sign language, communication with the user can be performed by linking with an external platform that analyzes and processes the sign language through the communication processing unit (320).

[0075] The universal operation processing unit (340) can be operated using multi-protocols (MQTT, HTTPS, GRPC, etc.) for optimal interoperability with other platforms.

[0076] In addition, the universal operation processing unit (340) can analyze the user image as described in FIG. 8 to determine the user's disability type, select a user type interface that matches the user's disability type, and control communication with the user through the interface that matches the user type even without a request from the user.

[0077]

[0078] FIG. 5 is a diagram illustrating the configuration of a communication processing unit according to one embodiment of the present invention.

[0079] The communication processing unit (320) of the present invention includes an image algorithm processing unit (321), a sign language algorithm processing unit (322), a voice algorithm processing unit (323), and a Braille algorithm processing unit (324).

[0080] The image algorithm processing unit (321) performs the function of recognizing and processing the user's gestures and actions for communication with the user.

[0081] The sign language algorithm processing unit (322) performs the function of recognizing and processing the user's sign language for communication with the user.

[0082] The sign language algorithm processing unit (322) recognizes the user's sign language and generates and provides sign language that matches the sign language used by the user.

[0083] Here, the sign language algorithm processing unit creates an avatar suitable for the user to facilitate easier communication through sign language and communicates with the user through sign language through the avatar.

[0084] For example, if the user is a child, the child can create an avatar of their favorite character and communicate through sign language.

[0085] The voice algorithm processing unit (323) performs the function of recognizing and processing the user's voice for communication with the user.

[0086] The voice algorithm processing unit (323) considers the installation environment of the kiosk (200) and generates a noise-resistant STT (Speech-To-Text) algorithm through artificial intelligence to learn voice recognition according to the noise data and noise space of the environment, thereby improving the Error Rate to the level of 45%.

[0087] When receiving a user's voice from a kiosk (200), the user's voice includes, in addition to the user's voice, noise generated in the environment where the kiosk (200) is installed.

[0088] If ambient noise is included in addition to the user's voice, there is a very high possibility that the user's voice will be recognized as an incorrect voice and the user's request will be misunderstood and processed.

[0089] To solve these problems, the noise-resistant STT algorithm method of the present invention performs the following steps.

[0090] The artificial intelligence of the voice algorithm processing unit (323) separates the user's voice received from the kiosk (200) into the user's voice part and the non-voice part (noise generated around the kiosk) and stores only the non-voice part.

[0091] Afterwards, the artificial intelligence learns the stored non-speech part as input data and finds out the frequency characteristics of the non-speech part of the kiosk.

[0092] Afterwards, the artificial intelligence learns to create an optimal noise removal frequency filter that can remove the frequency characteristics of non-speech parts.

[0093] Afterwards, when a user's voice is received from the kiosk, a noise removal frequency filter corresponding to the kiosk is immediately used to remove only the non-voice portion from the user's voice received from the kiosk, thereby isolating only the user's voice portion.

[0094] In this way, by generating text using only the separated user voice portion, the user's voice can be accurately interpreted as text without being misinterpreted due to noise.

[0095] By using the noise-resistant STT algorithm method of the present invention described above, there is an advantage in that the user service can be provided more quickly and accurately as the number of times the user uses the service through the voice interface increases.

[0096] The Braille algorithm processing unit (324) performs the function of recognizing and processing Braille for communication with the user.

[0097]

[0098] FIG. 6 is a diagram illustrating one embodiment of the present invention for processing communication with a user by performing sign language through an avatar.

[0099] When a user's sign language output sentence is input from the kiosk (200), the corresponding sign language player command is transmitted to the cloud server (300), and the sign language algorithm processing unit (322) analyzes and preprocesses the corresponding sign language, analyzes the morphemes of the sign language sentence, and searches the sign language DB to find out the meaning of the corresponding sign language. Thereafter, an appropriate sign language animation sentence is generated through the avatar animation generation module and transmitted to the kiosk (200), so that the corresponding sign language animation is output through the avatar through the display device of the kiosk (200).

[0100]

[0101] FIG. 7 is a diagram illustrating one embodiment of the present invention for processing communication with a user through voice.

[0102] When a user (10) makes a service request to a kiosk (200) by voice, the kiosk transmits the user's voice to a cloud server, and the voice algorithm processing unit (323) converts the user's voice into text through a noise-resistant STT (Speech-To-Text) algorithm generated through voice recognition learning according to noise data and noise space of the environment by artificial intelligence considering the installation environment of the kiosk, and performs STT candidate correction based on a dialogue scenario for each service domain to analyze the correct user's request, generates an answer appropriate to the request, and transmits it to the kiosk. Thereafter, the kiosk provides the received answer to the user by voice.

[0103]

[0104] FIG. 8 is a diagram illustrating a method for recognizing a user and determining a user's disability according to one embodiment of the present invention.

[0105] When a user (10) approaches, the kiosk (200) recognizes the user through a user recognition sensor (video, proximity sensor, etc.), creates a user image through a video device, and transmits it to a cloud server (300).

[0106] The cloud server (300) analyzes the user image to detect important objects (e.g., wheelchair, cane, hearing aid, etc.) that can identify the user's disability type in the user image and analyzes them to determine the user's disability type.

[0107] For example, if a wheelchair is detected in a user's video, the user may be identified as having a lower limb disability. If a cane for the visually impaired is detected in the user's video, the user may be identified as having a visual impairment. If a hearing aid is detected in the user's video, the user may be identified as having a hearing impairment. If no object identifying a person with a disability is detected, the user may be identified as having a normal disability.

Claims

1. In the cloud-based artificial intelligence barrier-free universal kiosk platform, A cloud server that operates a universal kiosk platform that receives user information from a kiosk, provides a user type interface that matches the user type, communicates with the user through the corresponding user type interface using artificial intelligence to identify the user's needs, and processes the user's needs through linkage with a service agency that performs the user's needs; and A kiosk that recognizes the user's access and the user's disability, mediates communication between the cloud server and the user, and provides a user interface suitable for the user type using a multi-interface device to provide a service suitable for the user's needs; Cloud-based artificial intelligence barrier-free universal kiosk platform including.

2. In paragraph 1, The above cloud server, A server communication unit that performs communication with the above kiosk and other system devices; A communication processing unit that generates and provides communication information suitable for the above user type interface and analyzes communication information performed with the user; A service agency linkage unit that performs linkage with a service agency that processes the above user requests; and A universal operation processing unit that controls the functions of each component of the cloud server, analyzes user information received from the kiosk to identify the type of disability of the user, processes linkage with an external platform that provides additional methods necessary for communication with the user, obtains methods necessary for communication, and manages and controls the kiosk; Cloud-based artificial intelligence barrier-free universal kiosk platform including.

3. In paragraph 2, The above communication processing unit, An image algorithm processing unit that recognizes and processes the user's gestures and actions for communication with the user; A sign language algorithm processing unit that recognizes and processes the sign language of the user for communication with the user; A voice algorithm processing unit that recognizes and processes the user's voice for communication with the user; and A Braille algorithm processing unit that recognizes and processes Braille for communication with the user; A cloud-based artificial intelligence barrier-free universal kiosk platform characterized by including:

4. In paragraph 2, The above image algorithm processing unit, A cloud-based artificial intelligence barrier-free universal kiosk platform characterized in that it builds a learning dataset that classifies the gesture and behavior patterns of a user when the user is any one of a mentally or physically disabled person, the elderly, or a child, and corrects and processes the user's gestures and actions.

5. In paragraph 2, The above sign language algorithm processing unit, A cloud-based artificial intelligence barrier-free universal kiosk platform characterized by generating an avatar suitable for the user by considering the above user information and processing sign language with the user.

6. In paragraph 2, The above voice algorithm processing unit, A cloud-based artificial intelligence barrier-free universal kiosk platform characterized by processing a user's voice by performing noise compensation suitable for the environment of the kiosk by considering the noise according to the environment of the kiosk.

7. In paragraph 1, The above kiosk is, A user recognition unit including at least one of an image processing device and a proximity sensor, which recognizes the user's approach and obstacles; A multi-interface unit including at least one of a display device, a sound device, and a Braille block device, and performing information exchange with the user through at least one of the devices; A kiosk communication unit that communicates with the above cloud server; A height adjustment unit capable of adjusting the height according to the physical condition of the user according to his / her disability; and A service provider that provides the service requested by the above user; A cloud-based artificial intelligence barrier-free universal kiosk platform characterized by including:

8. In paragraph 7, A cloud-based artificial intelligence barrier-free universal kiosk platform characterized by further including a user terminal communication unit that performs communication with a user terminal and transmits and receives communication information provided by the cloud server to the user terminal.

Images