Method and apparatus for providing image information for physically vulnerable users

Abstract

The present invention relates to a method and device for providing image information for physically disabled persons. A method for providing image information for a physically disabled person according to an embodiment of the present invention is a method in which a terminal provides image information for a physically disabled person, comprising the steps of executing an application, displaying a recognition menu area, checking whether a user gesture is detected, checking whether the user gesture is valid according to a pre-set user gesture standard when the user gesture is detected, analyzing the user gesture to generate a user gesture analysis result when the user gesture is valid according to the pre-set user gesture standard, and setting a specific recognition method pre-set according to the user gesture analysis result.

Description

Technology Field

[0001] The present invention relates to a method and device for providing image information for physically disabled persons. Background Technology

[0002] Conventional smartphone applications are primarily designed for sighted users and employ visual buttons, icons, and complex menu structures. However, physically vulnerable individuals with poor or impaired vision, such as the visually impaired, the elderly, and children, must rely on specific screen readers to use these applications, which leads to the following problems.

[0003] First, navigating the complex menu structure takes a significant amount of time, and users must go through multiple steps to find the desired function. Second, the screen reader's voice guidance is verbose and repetitive, degrading the user experience. Third, the learning burden is high due to the differing interface structures of each application. Fourth, integrated use is difficult because features such as text recognition, object recognition, and facial expression recognition are scattered across individual applications.

[0004] In addition, existing voice command-based systems require accurate pronunciation and a quiet environment, while air gesture recognition systems had problems with low accuracy and caused arm fatigue.

[0005] Therefore, there was a need for a new type of user interface that is intuitive and easy to learn, and that can be used independently by physically vulnerable individuals such as the visually impaired, the elderly, and children without assistive technology. Prior art literature

[0006] Republic of Korea Published Patent No. 10-2021-0125748 (Published on October 19, 2021) The problem to be solved

[0007] The technical problem that the present invention aims to solve is to provide a method and device for providing image information for physically vulnerable individuals, such as the visually impaired, the elderly, and children, so that they can efficiently use various functions using only intuitive gestures without navigating complex menus. means of solving the problem

[0008] A method for providing image information for a physically disabled person according to an embodiment of the present invention is a method in which a terminal provides image information for a physically disabled person, comprising the steps of executing an application, displaying a recognition menu area, checking whether a user gesture is detected, checking whether the user gesture is valid according to a preset user gesture standard when the user gesture is detected, analyzing the user gesture to generate a user gesture analysis result when the user gesture is valid according to the preset user gesture standard, and setting a preset specific recognition method according to the user gesture analysis result.

[0009] A method for providing image information for a physically disabled person according to an embodiment of the present invention further comprises the steps of: providing a pre-set specific recognition method as voice; checking whether a shooting command signal is received; upon receiving the shooting command signal, executing a camera to capture an image and acquiring a specific image; analyzing the specific image using artificial intelligence and generating a recognition analysis result; and providing the recognition analysis result.

[0010] If the result of the above user gesture analysis is a first-direction sliding gesture, the above-previously set specific recognition method is set as a first specific recognition method; if the result of the above user gesture analysis is a second-direction sliding gesture, the above-previously set specific recognition method is set as a second specific recognition method; if the result of the above user gesture analysis is a third-direction sliding gesture, the above-previously set specific recognition method is set as a third specific recognition method; and if the result of the above user gesture analysis is a fourth-direction sliding gesture, the above-previously set specific recognition method is set as a fourth specific recognition method.

[0011] A method for providing image information for a physically disabled person according to another embodiment of the present invention is a method in which a terminal provides image information for a physically disabled person, comprising: a step of detecting a user gesture; a step of obtaining user gesture direction information and user gesture distance information; a step of checking whether the user gesture satisfies a preset user gesture standard; a step of generating a user gesture analysis result by analyzing the user gesture direction and the number of user gestures when the user gesture satisfies the preset user gesture standard; a step of setting a preset specific recognition method according to the user gesture analysis result; a step of providing the preset specific recognition method as voice; a step of checking whether a shooting command signal is received; and a step of setting the camera to a preset camera setting that matches the preset specific recognition method when the shooting command signal is received.

[0012] A method for providing image information for a physically disabled person according to another embodiment of the present invention further includes the steps of capturing an image with the aforementioned preset camera settings and acquiring a specific image, analyzing the specific image using artificial intelligence, and generating a recognition analysis result.

[0013] An image information providing device for a physically disabled person according to one embodiment of the present invention includes a camera for capturing images, a control unit for executing an application, displaying a recognition menu area, checking whether a user gesture is detected, and if the user gesture is detected, checking whether the user gesture is valid according to a preset user gesture standard, and if the user gesture is valid according to the preset user gesture standard, analyzing the user gesture to generate a user gesture analysis result, and controlling to set a preset specific recognition method according to the user gesture analysis result.

[0014] The above control unit provides the above-mentioned preset specific recognition method as voice, checks whether a shooting command signal is received, and if the shooting command signal is received, controls the camera to take the image, acquire a specific image, analyze the specific image using artificial intelligence, generate a recognition analysis result, and provide the recognition analysis result.

[0015] The control unit controls the user gesture analysis result to be a first-direction sliding gesture, the pre-set specific recognition method to be set as a first specific recognition method, the user gesture analysis result to be a second-direction sliding gesture, the user gesture analysis result to be set as a second specific recognition method, the user gesture analysis result to be a third-direction sliding gesture, the user gesture analysis result to be set as a third specific recognition method, and the user gesture analysis result to be a fourth-direction sliding gesture, thereby controlling the user gesture analysis result to be set as a fourth specific recognition method.

[0016] An image information providing device for a physically disabled person according to another embodiment of the present invention includes a camera that captures an image, a control unit that detects a user gesture, obtains user gesture direction information and user gesture distance information, checks whether the user gesture satisfies a preset user gesture standard, and if the user gesture satisfies the preset user gesture standard, analyzes the user gesture direction and the number of user gestures to generate a user gesture analysis result, sets a preset specific recognition method according to the user gesture analysis result, provides the preset specific recognition method as voice, checks whether a shooting command signal is received, and if the shooting command signal is received, controls the camera to set the camera to a preset camera setting that matches the preset specific recognition method.

[0017] The above control unit controls the capture of the image with the above-mentioned preset camera settings, the acquisition of a specific image, the analysis of the specific image using artificial intelligence, and the generation of a recognition analysis result. Effects of the invention

[0018] According to the method and device for providing image information for physically disabled persons according to an embodiment of the present invention, the following effects are achieved.

[0019] First, the present invention allows physically disabled people, such as the visually impaired, the elderly, and children, to access all functions for acquiring image information with only simple four-way gestures without the need to navigate a complex menu structure, thereby drastically reducing accessibility to devices such as functional terminals compared to existing ones.

[0020] Second, the present invention enables completely independent use through its own voice guidance system without the need for screen readers or other assistive technologies, thereby significantly improving the digital independence of physically vulnerable individuals such as the visually impaired, the elderly, and children, and eliminating usage restrictions in various environments.

[0021] Third, the present invention provides various AI-based recognition functions, such as text, color, distance, facial expressions, and objects, through a single integrated interface, thereby providing a consistent user experience without the need for users to switch between multiple applications and maximizing utility in real life.

[0022] Fourth, the present invention provides a UX optimized for the cognitive flow of physically vulnerable people, such as the visually impaired, the elderly, and children, through auditory-centered multi-stage feedback (voice guidance → music playback → result voice guidance), thereby reducing mental fatigue and maintaining comfort even during long-term use.

[0023] Fifth, the present invention applies an algorithm that comprehensively analyzes distance, time, speed, and angle during gesture recognition and a machine learning-based correction system to efficiently filter out incorrect inputs and accurately execute intended functions.

[0024] Sixth, the present invention can significantly improve social inclusion by bridging the digital divide and guaranteeing access to information by enabling physically vulnerable people, such as the visually impaired, the elderly, and children, to utilize the latest digital technology. Brief explanation of the drawing

[0025] FIG. 1 is a drawing for explaining an image information providing device for physically disabled people according to an embodiment of the present invention. FIG. 2 is a drawing illustrating a user interface of an image information providing device for physically disabled people according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a method for providing image information for physically disabled people according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for providing image information for physically disabled people according to another embodiment of the present invention. FIG. 5 is a flowchart illustrating a method for providing image information for physically disabled people according to another embodiment of the present invention. Specific details for implementing the invention

[0026] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0027] Furthermore, the terms used in this specification are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms “comprises” and / or “comprising” as used in this specification do not exclude the presence or addition of other components other than those mentioned. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the invention pertains.

[0028] In the present invention, an “application” is software stored on a computer-readable storage medium and includes instructions and data for executing the operation of the present invention. The application is installed on a terminal, executed on the terminal, and communicates with a designated server via a communication network.

[0029] In the present invention, a visual assistance application for physically vulnerable people, such as the visually impaired, the elderly, and children, is an example of an application.

[0030] In the present invention, a “terminal” is a computing device comprising hardware such as a control unit (or processor), memory, a display unit (or display), a transceiver unit (or communication module), a camera, and an input unit. The processor (or control unit) operates an application in cooperation with the hardware.

[0031] The terminal can be implemented in various forms, and although a mobile terminal is used as an example, it can be implemented in the form of a pad-type terminal such as a smartpad, various types of computers such as a laptop computer, a wearable device, a TV terminal, etc.

[0032] In the present invention, “image information” refers to images or image information obtained by capturing with a camera. For example, image information can be various types of information such as objects, environments, colors, distances, facial expressions, moods, things, situations, banknotes, text, receipts, and text organization.

[0033] In the present invention, regarding the direction of user gestures, the left direction is used with the same meaning as the 9 o'clock direction of a clock, the upward direction as the 12 o'clock direction of a clock, the right direction as the 3 o'clock direction of a clock, and the downward direction as the 6 o'clock direction of a clock.

[0034] Hereinafter, in order to explain the present invention more specifically, embodiments according to the present invention will be described in more detail with reference to the accompanying drawings.

[0036] FIG. 1 is a drawing for explaining an image information providing device for a physically disabled person according to an embodiment of the present invention. FIG. 2 is a drawing for explaining a user interface of an image information providing device for a physically disabled person according to an embodiment of the present invention.

[0037] Referring to FIG. 1 and FIG. 2, an image information providing device for a physically disabled person (hereinafter referred to as a “terminal”) according to one embodiment of the present invention includes a memory (10), a control unit (20), a display unit (30), an input unit (40), a transmission and reception unit (50), and a camera (60). These components communicate through one or more communication buses or signal lines.

[0038] Memory (10) is a part that stores various data.

[0039] The memory (10) can store an application, at least one command signal (or command), a user command signal, a preset user gesture standard, a user gesture analysis result, a preset specific recognition method (a first specific recognition method, a second specific recognition method, a third specific recognition method, a fourth specific recognition method), user gesture direction information, user gesture distance information, a shooting command signal, an image, etc.

[0040] In addition, the memory (10) can store artificial intelligence learned according to a preset algorithm.

[0041] The control unit (20) is a part that controls the terminal according to the present invention. In addition, the control unit (20) is a part that controls each component of the terminal.

[0042] The control unit (20) can control each component of the terminal according to various applications, commands, command signals, etc.

[0043] Additionally, the control unit (20) executes an application, displays a recognition menu area, checks whether a user gesture is detected, and if a user gesture is detected, checks whether the user gesture is valid according to a pre-set user gesture standard, and if the user gesture is valid according to a pre-set user gesture standard, analyzes the user gesture to generate a user gesture analysis result, and controls the setting to a pre-set specific recognition method according to the user gesture analysis result.

[0044] Additionally, the control unit (20) provides a pre-set specific recognition method by voice, checks whether a shooting command signal is received, and if a shooting command signal is received, controls the camera to take a picture, acquire a specific picture, analyze the specific picture using artificial intelligence, generate a recognition analysis result, and provide the recognition analysis result.

[0045] Additionally, the control unit (20) controls the user gesture analysis result to be a first-direction sliding gesture, to be set as a first-direction specific recognition method, to be set as a second-direction specific recognition method, to be set as a third-direction specific recognition method, to be set as a third-direction specific recognition method, and to be set as a fourth-direction specific recognition method if the user gesture analysis result is a fourth-direction sliding gesture.

[0046] Additionally, the control unit (20) detects a user gesture, obtains user gesture direction information and user gesture distance information, checks whether the user gesture satisfies a preset user gesture standard, and if the user gesture satisfies the preset user gesture standard, analyzes the user gesture direction and the number of user gestures to generate a user gesture analysis result, sets a preset specific recognition method according to the user gesture analysis result, provides the preset specific recognition method as voice, checks whether a shooting command signal is received, and if a shooting command signal is received, controls the camera to be set to a preset camera setting that matches the preset specific recognition method.

[0047] In addition, the control unit (20) controls the capture of an image with a preset camera setting, the acquisition of a specific image, the analysis of the specific image using artificial intelligence, and the generation of a recognition analysis result.

[0048] The display unit (30) provides an interface for visual output between the user and the terminal.

[0049] The display unit (30) can display images captured by the camera (60) or various information.

[0050] The display unit (30) can display a recognition menu area (100) that is displayed by the execution of an application and a recognition image area (110) that displays an image of a camera (60) located on the rear side (meaning the opposite side of the display unit). For example, the recognition image area (100) may be placed on the top or the entire surface of the display unit (30).

[0051] The recognition menu area (100) may include a shooting button area (105) for camera shooting in the center part, a first recognition group area (or first recognition area) (101) on the left side of the center part, a second recognition group area (or second recognition area) (102) on the upper side of the center part, a third recognition group area (or third recognition area) (103) on the right side of the center part, and a fourth recognition group area (or fourth recognition area) (104) on the lower side of the center part.

[0052] The input unit (40) provides an interface for inputting command signals between the user and the terminal.

[0053] The input unit (40) allows a user to input a user gesture or user command signal (or command), such as an input signal, to the terminal, and when the user inputs a user gesture or user command signal (or command) through the input unit (40), the device can detect various types of user gestures or user command signals (or commands) and perform the user gesture or command under the control of the control unit (20).

[0054] The input section (40) may be a terminal button area including a terminal first button, a terminal second button, a terminal third button, a terminal fourth button, and a terminal camera button that are displayed by the execution of an application (or a visual aid application).

[0055] For example, if the input unit (40) is a touchscreen, the user can input a user command signal by touching a specific button (or specific area) of the terminal button area.

[0056] In some embodiments, such as a touchscreen, the display unit (30) and the input unit (40) may be integrated or separated.

[0057] The transmitting and receiving unit (50) is a part that transmits and receives data using signals such as electromagnetic waves. The transmitting and receiving unit (50) converts electrical signals into electromagnetic waves and communicates with a communication network and other communication devices through these electromagnetic waves.

[0058] The transmitting and receiving unit (50) can transmit the image captured by the camera, etc., to the outside based on the recognition analysis result according to the control of the control unit (20).

[0059] In addition, the transmitting and receiving unit (50) can receive various data, including pre-set commands, from an external server under the control of the control unit (20).

[0060] The camera (60) is a part that captures various information such as images or photos. The user can use the camera (60) of the terminal to capture colors, distances, facial expressions, atmosphere, objects, situations, banknotes, letters, receipts, text organization, etc.

[0061] Additionally, the camera (60) can be set to a pre-set camera setting that matches a pre-set specific recognition method according to the control of the control unit (20).

[0062] Hereinafter, it is noted that the series of processes performed by the terminal in the present invention can be viewed as processes performed by the control unit (20) of the present invention.

[0064] FIG. 3 is a flowchart illustrating a method for providing image information for physically disabled people according to an embodiment of the present invention.

[0065] Referring to FIG. 3, when a user inputs a user command signal to execute an application by pressing any button on the terminal, the terminal receives the user command signal to execute the application and executes the application (S301).

[0066] The terminal displays a recognition menu area on the display (S302). At this time, the recognition menu area displayed on the display includes a shooting button area for camera shooting in the center part, a first recognition group area (or first recognition area) on the left side of the center part, a second recognition group area (or second recognition area) on the upper side of the center part, a third recognition group area (or third recognition area) on the right side of the center part, and a fourth recognition group area (or fourth recognition area) on the lower side of the center part.

[0067] Here, the first recognition group area, the second recognition group area, the third recognition group area, and the fourth recognition group area included in the recognition menu area can be in a slide form.

[0068] Additionally, the terminal may display a recognition image area that displays an image from a camera located on the rear (meaning the side opposite the display). For example, the recognition image area may be placed at the top or across the entire display.

[0069] The terminal checks whether a user gesture is detected (S303). For example, if the display part of the terminal is a touchscreen, when the user touches the touchscreen with a finger to perform a gesture (e.g., a sliding gesture), the terminal checks whether the user gesture is detected.

[0070] When a user gesture is detected, the terminal checks whether the user gesture is valid according to a pre-set user gesture standard (S304).

[0071] For example, when a user gesture is detected, the terminal checks the touch start point coordinates (or start point coordinates) and touch end point coordinates (or end point coordinates) of the user gesture, calculates a vector between the two coordinates (touch start point coordinates and touch end point coordinates) to determine the sliding user gesture direction information (or user gesture direction) and user gesture distance information (or user gesture distance). At this time, the terminal can detect the coordinates in real time by detecting the user gesture.

[0072] The terminal checks user gesture direction information (or user gesture direction) and user gesture distance information (or user gesture distance), and if it satisfies a pre-set user gesture criterion (or pre-set gesture validity criterion), determines that the user gesture is valid, and if it does not satisfies the pre-set user gesture criterion, determines that the user gesture is invalid.

[0073] At this time, the pre-set user gesture criteria can be of various types.

[0074] For example, a pre-configured user gesture criterion may define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction)) and slides over a pre-configured distance or greater (e.g., a case where the movement distance is greater than or equal to a pre-configured distance).

[0075] In another embodiment, a pre-set user gesture criterion can be set to define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction), slides over a pre-set distance or more (e.g., a case where the movement distance is greater than or equal to a pre-set distance), and the user gesture is at a speed greater than or equal to a specific speed.

[0076] In the present invention, if a user gesture is not the correct direction among the specific directions—left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction)—the terminal determines the direction closest to the specific direction as the specific direction.

[0077] If the terminal does not detect a user gesture, it performs step S303.

[0078] If the user gesture is valid according to the pre-set user gesture criteria, the terminal analyzes the user gesture and generates a user gesture analysis result (S305).

[0079] That is, if the terminal determines that a user gesture has a specific direction and moves a distance greater than a preset distance, and the user gesture is valid according to preset user gesture criteria, it analyzes the user gesture direction information and user gesture distance information to generate a user gesture analysis result.

[0080] For example, if the starting point coordinates of a user gesture are (x₁, y₁) and the ending point coordinates of a user gesture are (x₂, y₂), if the difference between x₁ and x₂ is (+), the user gesture analysis result is a leftward sliding gesture; if the difference between x₁ and x₂ is (-), the user gesture analysis result is a rightward sliding gesture; if the difference between y₁ and y₂ is (+), the user gesture analysis result is a downward sliding gesture; and if the difference between x₁ and x₂ is (-), the user gesture analysis result is an upward sliding gesture.

[0081] In another embodiment, if the user gesture is not in the correct direction among the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), and downward direction (6 o'clock direction), the terminal can check the starting point coordinates and the ending point coordinates to determine the direction closest to any of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), and downward direction (6 o'clock direction) as a specific direction.

[0082] If the user gesture is invalid, the terminal performs step S304.

[0083] The terminal is set to a specific recognition method based on the user gesture analysis result (S306).

[0084] For example, if the user gesture analysis result is a leftward sliding gesture (first-direction sliding gesture), the terminal can set a pre-configured specific recognition method to a first specific recognition method (e.g., character recognition); if the user gesture analysis result is an upward sliding gesture (second-direction sliding gesture), the terminal can set a pre-configured specific recognition method to a second specific recognition method (e.g., color recognition); if the user gesture analysis result is a rightward sliding gesture (third-direction sliding gesture), the terminal can set a pre-configured specific recognition method to a third specific recognition method (e.g., facial expression recognition); and if the user gesture analysis result is a downward sliding gesture (fourth-direction sliding gesture), the terminal can set a pre-configured specific recognition method to a fourth specific recognition method (e.g., object recognition).

[0085] In another embodiment, if the terminal performs a leftward sliding gesture (first-direction sliding gesture), a pre-configured specific recognition method may be any one of character recognition, text organization, receipt recognition, civil petition document recognition, or translation functions included in the first recognition group; if it performs an upward sliding gesture (second-direction sliding gesture), a pre-configured specific recognition method may be any one of color recognition, distance recognition, or additional service functions included in the second recognition group; if it performs a rightward sliding gesture (third-direction sliding gesture), a pre-configured specific recognition method may be any one of facial expression recognition or mood recognition included in the third recognition group; and if it performs a downward sliding gesture (fourth-direction sliding gesture), a pre-configured specific recognition method may be any one of object recognition, banknote recognition, situation recognition, or spatial recognition included in the fourth recognition group. However, the present invention allows for various forms of the recognition group.

[0086] The terminal provides a specific recognition method that has been pre-configured via voice (S307). The terminal provides the specific recognition method to the user via voice.

[0087] For example, if the user gesture analysis result is a leftward sliding gesture (first-direction sliding gesture), the terminal may set a pre-configured specific recognition method as a first specific recognition method (e.g., character recognition) and provide the first specific recognition method (e.g., character recognition) to the user via voice; if the user gesture analysis result is an upward sliding gesture (second-direction sliding gesture), the terminal may set a pre-configured specific recognition method as a second specific recognition method (e.g., color recognition) and provide the second specific recognition method (e.g., color recognition) to the user via voice; if the user gesture analysis result is a rightward sliding gesture (third-direction sliding gesture), the terminal may set a pre-configured specific recognition method as a third specific recognition method (e.g., facial expression recognition) and provide the third specific recognition method (e.g., facial expression recognition) to the user via voice; and if the user gesture analysis result is a downward sliding gesture (fourth-direction sliding gesture), the terminal may set a pre-configured specific recognition method as a fourth specific recognition method (e.g., object recognition) and provide the fourth specific recognition method (e.g., object recognition) to the user via voice.

[0088] The terminal checks whether it receives a shooting command signal (S308).

[0089] For example, if the shooting command signal is a user's double touch, when the user double-touches the display, the terminal confirms the double touch, which is the shooting command signal.

[0090] When the terminal receives a shooting command signal, it activates the camera to capture an image and acquires a specific image (S309).

[0091] For example, when the terminal receives a double touch from the user, it launches the camera to capture text, colors, facial expressions, objects, etc., and acquires a specific image.

[0092] If the terminal does not receive a shooting command signal, it performs step S308.

[0093] The terminal analyzes a specific image using artificial intelligence and generates a recognition analysis result (S310).

[0094] For example, if a specific image is text, the terminal uses artificial intelligence to analyze the text and generates a recognition analysis result based on the text analysis.

[0095] The terminal provides a recognition analysis result (S311).

[0096] For example, the terminal can provide the user with the text, which is the result of recognition analysis, as voice.

[0097] By doing this, users can select the recognition method they want and efficiently and easily check the recognition analysis results in voice.

[0099] FIG. 4 is a flowchart illustrating a method for providing image information for physically disabled people according to another embodiment of the present invention.

[0100] Referring to FIG. 4, the terminal detects a user gesture (S401). For example, if the terminal's display is a touchscreen, when the user touches the touchscreen with a finger to make a gesture (e.g., a sliding gesture), the terminal detects the user contact and the user gesture.

[0101] The terminal obtains user gesture direction information and user gesture distance information (S402).

[0102] For example, when a user gesture is detected, the terminal checks the touch start point coordinates (or start point coordinates) and touch end point coordinates (or end point coordinates) from the user gesture, calculates a vector between the two coordinates (touch start point coordinates and touch end point coordinates) to obtain sliding user gesture direction information (or user gesture direction) and user gesture distance information (or user gesture distance). At this time, the terminal can detect the coordinates in real time by detecting the user gesture.

[0103] The terminal checks whether the user gesture meets the pre-set user gesture criteria (S403).

[0104] At this time, the pre-set user gesture criteria can be of various types.

[0105] For example, a pre-configured user gesture criterion may define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction)) and slides over a pre-configured distance or greater (e.g., a case where the movement distance is greater than or equal to a pre-configured distance).

[0106] In another embodiment, a pre-set user gesture criterion can be set to define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction), slides over a pre-set distance or more (e.g., a case where the movement distance is greater than or equal to a pre-set distance), and the user gesture is at a speed greater than or equal to a specific speed.

[0107] When a user gesture satisfies a preset user gesture standard, the terminal analyzes the direction of the user gesture and the number of user gestures to generate a user gesture analysis result (S404).

[0108] For example, when the starting point coordinates of a user gesture are (x₁, y₁) and the ending point coordinates of a user gesture are (x₂, y₂), the terminal can generate a user gesture analysis result by analyzing the direction of the user gesture such that if the difference between x₁ and x₂ is (+), the user gesture analysis result is a leftward sliding gesture; if the difference between x₁ and x₂ is (-), the user gesture analysis result is a rightward sliding gesture; if the difference between y₁ and y₂ is (+), the user gesture analysis result is a downward sliding gesture; and if the difference between x₁ and x₂ is (-), the user gesture analysis result is an upward sliding gesture. Additionally, the terminal can analyze the number of user gestures and include the number of user gestures in the user gesture analysis result.

[0109] If the terminal does not satisfy the pre-set user gesture criteria, it performs step S403.

[0110] The terminal is set to a specific recognition method based on the user gesture analysis result (S405).

[0111] In the first embodiment, when the user gesture direction is a left-direction sliding gesture (first-direction sliding gesture) in the user gesture analysis result, if the left-direction sliding gesture (first-direction sliding gesture) occurs once, it is set to character recognition of the first recognition group using a pre-set specific recognition method; if the left-direction sliding gesture (first-direction sliding gesture) occurs twice, it is set to text organization of the first recognition group using a pre-set specific recognition method; if the left-direction sliding gesture (first-direction sliding gesture) occurs three times, it is set to receipt recognition of the first recognition group using a pre-set specific recognition method; if the left-direction sliding gesture (first-direction sliding gesture) occurs four times, it is set to civil complaint document recognition of the first recognition group using a pre-set specific recognition method; and if the left-direction sliding gesture (first-direction sliding gesture) occurs five times, it is set to translation function of the first recognition group using a pre-set specific recognition method.

[0112] In a second embodiment, when the user gesture direction is an upward sliding gesture (second sliding gesture) in the user gesture analysis result, if the upward sliding gesture (second sliding gesture) occurs once, the terminal is set to color recognition of the second recognition group using a pre-set specific recognition method; if the upward sliding gesture (second sliding gesture) occurs twice, the terminal is set to distance recognition of the second recognition group using a pre-set specific recognition method; and if the upward sliding gesture (second sliding gesture) occurs three times, the terminal is set to additional service functions of the second recognition group using a pre-set specific recognition method.

[0113] In a third embodiment, the terminal can, in the result of analyzing user gestures, set the expression recognition of the third recognition group using a pre-set specific recognition method when the user gesture direction is a right-direction sliding gesture (third-direction sliding gesture) and when the right-direction sliding gesture (third-direction sliding gesture) occurs once, set the expression recognition of the third recognition group using a pre-set specific recognition method, and when the right-direction sliding gesture (third-direction sliding gesture) occurs twice, set the mood recognition of the third recognition group using a pre-set specific recognition method.

[0114] In the fourth embodiment, when the user gesture direction is a downward sliding gesture (fourth direction sliding gesture) in the user gesture analysis result, if the downward sliding gesture (fourth direction sliding gesture) occurs once, the terminal is set to object recognition of the fourth recognition group using a pre-set specific recognition method; if the downward sliding gesture (fourth direction sliding gesture) occurs twice, the terminal is set to banknote recognition of the fourth recognition group using a pre-set specific recognition method; if the downward sliding gesture (fourth direction sliding gesture) occurs three times, the terminal is set to situation recognition of the fourth recognition group using a pre-set specific recognition method; and if the downward sliding gesture (fourth direction sliding gesture) occurs four times, the terminal is set to spatial recognition of the fourth recognition group using a pre-set specific recognition method.

[0115] The terminal provides a specific recognition method that has been pre-configured via voice (S406). The terminal provides the specific recognition method to the user via voice.

[0116] The terminal checks whether it receives a shooting command signal (S407).

[0117] When the terminal receives a shooting command signal, it sets the camera to a pre-configured camera setting that matches a specific pre-configured recognition method (S408). The terminal can set the camera to a pre-configured camera setting that matches the most optimal camera setting for a specific pre-configured recognition method.

[0118] For example, if the terminal determines that a specific pre-configured recognition method is character recognition of the first recognition group, it sets the camera to a pre-configured camera setting that enables the camera to recognize characters as well as possible. This is intended to further improve the recognition effect of objects or things by configuring the camera to match the specific recognition method.

[0119] If the terminal does not receive a shooting command signal, it performs step S407.

[0120] The terminal captures video with a pre-configured camera setting and acquires a specific video (S409).

[0121] The terminal analyzes a specific image using artificial intelligence and generates a recognition analysis result (S410).

[0122] For example, if a specific image contains text, the terminal sets the camera to a pre-configured camera setting optimized for text recognition to capture an image containing the text, analyzes the text using artificial intelligence, and generates a recognition analysis result based on the text analysis.

[0123] The terminal provides a recognition analysis result (S411).

[0124] For example, the terminal can provide the user with the text, which is the result of recognition analysis, as voice.

[0126] FIG. 5 is a flowchart illustrating a method for providing image information for physically disabled people according to another embodiment of the present invention.

[0127] Referring to FIG. 5, the user detects a first user gesture of the terminal (S501). For example, if the terminal's display is a touchscreen, when the user touches the touchscreen with a finger to perform a gesture (e.g., a sliding gesture), the terminal checks whether the first user gesture is detected.

[0128] The terminal obtains first user gesture direction information and first user gesture distance information (S502).

[0129] For example, when a first user gesture is detected, the terminal checks the first touch start point coordinates (or first start point coordinates) and the first touch end point coordinates (or first end point coordinates) in the first user gesture, calculates a vector between the two coordinates, the first touch start point coordinates and the first touch end point coordinates, to obtain sliding first user gesture direction information (or first user gesture direction) and first user gesture distance information (or first user gesture distance). At this time, the terminal can detect the coordinates in real time by detecting the first user gesture.

[0130] The terminal checks whether the first user gesture meets the pre-set user gesture criteria (S503).

[0131] At this time, the pre-set user gesture criteria can be of various types.

[0132] For example, a pre-configured user gesture criterion may define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction)) and slides over a pre-configured distance or greater (e.g., a case where the movement distance is greater than or equal to a pre-configured distance).

[0133] In another embodiment, a pre-set user gesture criterion can be set to define a user gesture as valid if the user gesture has a specific direction (e.g., a case where the direction can be determined as one of the left direction (9 o'clock direction), upward direction (12 o'clock direction), right direction (3 o'clock direction), or downward direction (6 o'clock direction), slides over a pre-set distance or more (e.g., a case where the movement distance is greater than or equal to a pre-set distance), and the user gesture is at a speed greater than or equal to a specific speed.

[0134] When the terminal determines that the first user gesture satisfies the pre-set user gesture criteria, it analyzes the direction of the first user gesture and the number of first user gestures to generate a first user gesture analysis result (S504).

[0135] For example, when the starting point coordinates of the first user gesture are (x₁, y₁) and the ending point coordinates of the first user gesture are (x₂, y₂), the terminal can generate a first user gesture analysis result by analyzing the direction of the first user gesture such that if the difference between x₁ and x₂ is (+), the first user gesture analysis result is a leftward sliding gesture; if the difference between x₁ and x₂ is (-), the first user gesture analysis result is a rightward sliding gesture; if the difference between y₁ and y₂ is (+), the first user gesture analysis result is a downward sliding gesture; and if the difference between x₁ and x₂ is (-), the first user gesture analysis result is an upward sliding gesture. Additionally, the terminal can analyze the number of first user gestures and include the number of first user gestures in the first user gesture analysis result.

[0136] If the terminal does not satisfy the pre-set user gesture criteria for the first user gesture, it performs step S503.

[0137] The terminal is set to a first specific recognition method pre-configured in the first recognition group based on the first user gesture analysis result (S505).

[0138] For example, when the terminal analyzes the first user gesture and the first user gesture direction is a leftward sliding gesture (first direction sliding gesture), it can set the first specific recognition method to character recognition of the first recognition group.

[0139] The terminal provides a pre-configured first specific recognition method in voice (S506). The terminal provides the first specific recognition method to the user in voice.

[0140] The terminal checks whether a second user gesture is detected (S507). For example, if the display unit is a touchscreen, the terminal checks whether a second user gesture is detected after the user makes a first touch with a finger on the touchscreen to detect a first user gesture.

[0141] When a second user gesture is detected, the terminal obtains second user gesture direction information and second user gesture distance information (S502).

[0142] For example, when a second user gesture is detected, the terminal checks the second touch start point coordinates (or second start point coordinates) and the second touch end point coordinates (or second end point coordinates) in the second user gesture, calculates a vector between the two coordinates, which are the second touch start point coordinates and the second touch end point coordinates, to obtain sliding second user gesture direction information (or second user gesture direction) and second user gesture distance information (or second user gesture distance). At this time, the terminal can detect the coordinates in real time by detecting the second user gesture.

[0143] The terminal checks whether the second user gesture meets the pre-set user gesture validity criteria (S509).

[0144] If the terminal determines that the second user gesture satisfies the pre-set user gesture validity criteria, it checks whether the direction of the second user gesture is the same as the direction of the first user gesture (S510).

[0145] If the terminal does not satisfy the pre-set user gesture validity criteria for the second user gesture, it performs step S509.

[0146] If the terminal's second user gesture direction is the same as the first user gesture direction, it is set to the second specific recognition method pre-configured in the first recognition group (S511).

[0147] For example, when the terminal analyzes the second user gesture and the second user gesture direction is a leftward sliding gesture (first direction sliding gesture) that is the same as the first user gesture direction, it can set the text organization of the first recognition group using a pre-configured second specific recognition method.

[0148] If the terminal does not have the same second user gesture direction as the first user gesture direction, it is set to a third specific recognition method pre-configured in the second recognition group (S512).

[0149] For example, when the terminal analyzes the second user gesture and the second user gesture direction is an upward sliding gesture (second direction sliding gesture) that is not the same as the first user gesture direction, it can set the color recognition of the second recognition group using a pre-configured third specific recognition method.

[0150] In another embodiment, when the terminal analyzes the second user gesture and the second user gesture direction is a rightward sliding gesture (third direction sliding gesture) that is not the same as the first user gesture direction, it can set the facial expression recognition of the third recognition group using a pre-set third specific recognition method.

[0151] In another embodiment, when the second user gesture analysis result shows that the second user gesture direction is a downward sliding gesture (fourth direction sliding gesture) that is not the same as the first user gesture direction, the terminal can set the object recognition of the fourth recognition group using a pre-set third specific recognition method.

[0152] In this way, the present invention checks the direction, distance, speed, etc. of a user gesture to verify the validity of the gesture, sets a specific recognition method according to the gesture, and provides a recognition result using artificial intelligence, thereby enabling physically vulnerable people such as the visually impaired, the elderly, and children to efficiently recognize information about objects or things.

[0153] A method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium. A computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The programs (program instructions) recorded on the recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROMs; RAMs; flash memory; and hardware devices specifically configured to store and execute program instructions. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-mentioned hardware devices may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0154] Although the present invention has been described above with reference to the embodiments illustrated in the drawings, this is merely for the purpose of explaining the invention, and those skilled in the art will understand that various modifications or equivalent embodiments are possible from the detailed description of the invention. Accordingly, the true scope of the rights of the present invention should be determined by the technical concept of the claims. Explanation of the symbols

[0155] 10: Memory 20: Control unit 30: Display unit 40: Input section 50: Transmitter / Receiver 60: Camera

Claims

Claim 1 A method for a terminal to provide image information for a physically disabled person comprises the steps of: executing an application; displaying a recognition menu area; checking whether a user gesture is detected; if the user gesture is detected, checking whether the user gesture is valid according to a preset user gesture standard; if the user gesture is valid according to the preset user gesture standard, analyzing the user gesture to generate a user gesture analysis result; and setting a preset specific recognition method according to the user gesture analysis result, wherein the step of setting a preset specific recognition method according to the user gesture analysis result comprises, in the user gesture analysis result, when the user gesture direction is a first-direction sliding gesture, if the first-direction sliding gesture occurs once, setting the preset specific recognition method to character recognition of the first recognition group; if the first-direction sliding gesture occurs twice, setting the preset specific recognition method to text organization of the first recognition group; if the first-direction sliding gesture occurs three times, setting the preset specific recognition method to receipt recognition of the first recognition group; and if the first-direction sliding gesture occurs four times, setting the preset specific recognition method to the It is set to civil complaint document recognition of the first recognition group, and if the first direction sliding gesture is 5 times, it is set to the translation function of the first recognition group using the pre-set specific recognition method; in the user gesture analysis result, when the user gesture direction is a second direction sliding gesture, if the second direction sliding gesture is 1 time, it is set to color recognition of the second recognition group using the pre-set specific recognition method; if the second direction sliding gesture is 2 times, it is set to distance recognition of the second recognition group using the pre-set specific recognition method; and if the second direction sliding gesture is 3 times, it is set to an additional service function of the second recognition group using the pre-set specific recognition method.A method for providing image information for a physically disabled person, wherein in the above user gesture analysis result, when the user gesture direction is a third-direction sliding gesture, if the third-direction sliding gesture is performed once, the pre-set specific recognition method is set to facial expression recognition of the third recognition group, and if the third-direction sliding gesture is performed twice, the pre-set specific recognition method is set to atmosphere recognition of the third recognition group; wherein in the above user gesture analysis result, when the user gesture direction is a fourth-direction sliding gesture, if the fourth-direction sliding gesture is performed once, the pre-set specific recognition method is set to object recognition of the fourth recognition group, if the fourth-direction sliding gesture is performed twice, the pre-set specific recognition method is set to banknote recognition of the fourth recognition group, if the fourth-direction sliding gesture is performed three times, the pre-set specific recognition method is set to situation recognition of the fourth recognition group, and if the fourth-direction sliding gesture is performed four times, the pre-set specific recognition method is set to spatial recognition of the fourth recognition group. Claim 2 A method for providing image information for a physically disabled person according to claim 1, further comprising the steps of: providing a specific recognition method set in the above-mentioned manner as voice; checking whether a shooting command signal is received; when the shooting command signal is received, executing a camera to capture an image and acquiring a specific image; analyzing the specific image using artificial intelligence and generating a recognition analysis result; and providing the recognition analysis result. Claim 3 delete Claim 4 A method for a terminal to provide image information for a physically disabled person, comprising: a step of detecting a first user gesture; a step of obtaining first user gesture direction information and first user gesture distance information; a step of checking whether the first user gesture satisfies a pre-set user gesture standard; a step of generating a first user gesture analysis result by analyzing the first user gesture direction and the number of first user gestures if the first user gesture satisfies the pre-set user gesture standard; a step of setting a first specific recognition method pre-set in a first recognition group according to the first user gesture analysis result; a step of providing the pre-set first specific recognition method as voice; a step of checking whether a second user gesture is detected; a step of obtaining second user gesture direction information and second user gesture distance information if the second user gesture is detected; a step of checking whether the second user gesture satisfies a pre-set user gesture validity standard; a step of checking whether the second user gesture direction is the same as the first user gesture direction if the second user gesture satisfies the pre-set user gesture validity standard if the second user gesture satisfies the pre-set user gesture validity standard; and a step of, if the second user gesture direction is the same as the first user gesture direction, a pre-set in the first recognition group The method includes a step of setting a second specific recognition method, and if the second user gesture direction is not the same as the first user gesture direction, a step of setting a third specific recognition method pre-configured in the second recognition group, wherein the step of acquiring the first user gesture direction information and the first user gesture distance information comprises, when the first user gesture is detected, checking the first starting point coordinates and the first ending point coordinates in the first user gesture, calculating a vector between the two coordinates, the first starting point coordinates and the first ending point coordinates, to acquire the sliding first user gesture direction information and the first user gesture distance information, and the step of setting a first specific recognition method pre-configured in the first recognition group according to the first user gesture analysis result.In the above first user gesture analysis result, when the direction of the first user gesture is a first-direction sliding gesture, the first recognition group is set to character recognition using the pre-set first specific recognition method; and when the second user gesture is detected, the step of obtaining the second user gesture direction information and the second user gesture distance information involves, when the second user gesture is detected, checking the second starting point coordinates and the second ending point coordinates in the second user gesture, calculating the vector between the two coordinates, the second starting point coordinates and the second ending point coordinates, to obtain the sliding second user gesture direction information and the second user gesture distance information; and when the direction of the second user gesture is the same as the direction of the first user gesture, the step of setting the first recognition group to the pre-set second specific recognition method involves, in the above second user gesture analysis result, when the direction of the second user gesture is a first-direction sliding gesture identical to the direction of the first user gesture, the first recognition group is set to text organization using the pre-set second specific recognition method; and when the direction of the second user gesture is the first user gesture A method for providing image information for a physically disabled person, wherein if the direction is not the same, the step of setting the pre-set third specific recognition method in the second recognition group is, when the second user gesture direction is a second direction sliding gesture that is not the same as the first user gesture direction in the second user gesture analysis result, the pre-set third specific recognition method is set to color recognition of the second recognition group. Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete

Images