Wearable terminal, program, method, and system

Abstract

To facilitate intention transmission by gesture.SOLUTION: A wearable terminal according to an embodiment of the present disclosure is a wearable terminal having a photographing function and a sound output function, wherein a controller of the wearable terminal analyzes an image including at least a hand of a wearer of the wearable terminal, the image being photographed by the wearable terminal, detects a gesture represented by at least the hand of the wearer of the wearable terminal, and notifies the detected gesture.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a wearable terminal, a program, a method, and a system. [Background technology]

[0002] Conventionally, communication has been carried out using gestures such as sign language. In order to facilitate dialogue with sign language speakers (hereinafter referred to as sign speakers), a technology is known that converts sign language into text or audio based on images of the sign language (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-169814 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional methods, sign language is converted into text or audio using an image of the signer taken from the position of the person facing the signer (i.e., from the person facing the signer toward the signer). Therefore, the signer has to have someone else take a picture of their sign language, and cannot convert their sign language into text or audio at their own desired timing.

[0005] The present disclosure aims to facilitate communication through gestures. [Means for solving the problem]

[0006] A wearable terminal according to a first aspect of the present disclosure includes: A wearable device equipped with a photographing function and a sound output function, The control unit of the wearable terminal Analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture made by at least the hand of the wearer of the wearable device; The detected gesture is notified.

[0007] According to the first aspect of the present disclosure, a wearable device wearer can communicate their intentions using gestures at a timing of their choice.

[0008] A second aspect of the present disclosure is the wearable terminal according to the first aspect, The gesture notification is performed by converting a gesture made by at least the hand of the wearer of the wearable device into text and outputting the text as synthesized voice.

[0009] According to the second aspect of the present disclosure, the wearer of the wearable terminal can give a voice notification.

[0010] A third aspect of the present disclosure is the wearable terminal according to the second aspect, The device further includes a storage unit in which the gesture and the text are stored in association with each other.

[0011] According to the third aspect of the present disclosure, the wearer of the wearable terminal can communicate content that is pre-associated with a gesture.

[0012] A fourth aspect of the present disclosure is the wearable terminal according to the third aspect, General-purpose gestures and text are associated and stored.

[0013] According to the fourth aspect of the present disclosure, a wearable device wearer can use general-purpose gestures such as sign language.

[0014] A fifth aspect of the present disclosure is the wearable terminal according to the third aspect, The gestures and text registered by the wearer of the wearable device are associated with each other and stored.

[0015] According to the fifth aspect of the present disclosure, the wearer of the wearable terminal can use gestures that he or she has registered in advance.

[0016] A sixth aspect of the present disclosure is a wearable terminal according to any one of the second to fifth aspects, The synthetic speech is generated each time a gesture is made.

[0017] According to the sixth aspect of the present disclosure, there is no need to generate synthetic speech in advance.

[0018] A seventh aspect of the present disclosure is a wearable terminal according to any one of the second to fifth aspects, The synthetic speech is generated in advance.

[0019] According to the seventh aspect of the present disclosure, it is possible to save the effort of generating synthetic speech.

[0020] An eighth aspect of the present disclosure is a wearable terminal according to any one of the first to seventh aspects, The gestures are sign language.

[0021] According to the eighth aspect of the present disclosure, a wearable device wearer can communicate using sign language.

[0022] A ninth aspect of the present disclosure is a wearable terminal according to any one of the first to eighth aspects, The wearable device is equipped with a microphone function, The wearable device collects the voice of the person with whom the wearer of the wearable device is having a conversation, converts the voice into text using voice recognition, and displays the text on a device equipped with a display function.

[0023] According to the ninth aspect of the present disclosure, a wearable device wearer can communicate with gestures at a timing of their choice, and can also recognize what the other person in conversation is saying.

[0024] A tenth aspect of the present disclosure is a wearable terminal according to any one of the first to ninth aspects, The wearable device is equipped with a microphone function, When the wearable device picks up the voice of the person with whom the wearer of the wearable device is having a conversation, the wearable device vibrates.

[0025] According to the tenth aspect of the present disclosure, the wearer of the wearable device can communicate with gestures at a timing of their choice, and can also recognize when the person they are conversing with has started speaking.

[0026] A program according to an eleventh aspect of the present disclosure includes: A wearable device equipped with photography and sound output functions. a step of analyzing an image captured by the wearable device and including at least the hand of the wearer of the wearable device, and detecting a gesture made by at least the hand of the wearer of the wearable device; notifying the detected gesture Execute the following.

[0027] A method according to a twelfth aspect of the present disclosure, comprising: A method executed by a wearable device equipped with a photographing function and a sound output function, analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture performed by at least the hand of the wearer of the wearable device; notifying the detected gesture; Includes:

[0028] A system according to a thirteenth aspect of the present disclosure includes: A system including a wearable device equipped with a photographing function and a sound output function, and a server, Analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture made by at least the hand of the wearer of the wearable device; The detected gesture is notified.

[0029] A fourteenth aspect of the present disclosure is the system according to the thirteenth aspect, The server Associating the gestures and text registered by the wearer of the wearable device, The wearable terminal converts gestures made by at least the hands of a person wearing the wearable terminal into text and outputs the text as synthesized voice.

[0030] According to the fourteenth aspect of the present disclosure, the wearer of the wearable terminal can use gestures that he or she has registered in advance. [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 1 is a diagram for explaining an overview according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an overall configuration according to an embodiment of the present disclosure. [Figure 3] 1 is an example of a wearable terminal according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a diagram illustrating a hardware configuration of a wearable terminal according to an embodiment of the present disclosure. [Figure 5] FIG. 2 is a diagram illustrating a hardware configuration of a display terminal according to an embodiment of the present disclosure. [Figure 6] FIG. 2 is a functional block diagram of a control unit of a wearable terminal according to an embodiment of the present disclosure. [Figure 7] FIG. 2 is a functional block diagram of a control unit of a display terminal according to an embodiment of the present disclosure. [Figure 8]10 is a flowchart of a process for notifying a gesture according to an embodiment of the present disclosure. [Figure 9] 10 is a flowchart of a process for outputting a gesture as synthetic speech according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a sequence diagram of a process for displaying speech as text according to an embodiment of the present disclosure. [Figure 11] 10 is a flowchart of a process for displaying audio as text according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0032] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0033] <Terminology> In this specification, a "gesture" refers to a bodily movement of the hands or other parts of the body used to communicate an intention to another person. For example, a gesture is sign language. Note that while this specification mainly describes the case of sign language, the present disclosure can be applied to gestures including sign language.

[0034] 1 is a diagram illustrating an overview according to an embodiment of the present disclosure. A signer 10 wearing a wearable terminal 11 and a display terminal 12 converses with a speaker 20. The signer 10 communicates using sign language, and the speaker 20 communicates using voice.

[0035] A case will be described in which sign language user 10 performs a sign language action. Wearable device 11 captures a sign language action performed by sign language user 10 (for example, a sign language action that indicates "the work is completed"). Next, wearable device 11 converts the sign language into text through image analysis (i.e., converts the sign language "the work is completed" into text). Next, wearable device 11 outputs a synthesized voice of the text (i.e., the text "the work is completed") from the speaker of wearable device 11. Note that while the case of synthesized voice will be mainly described in this specification, wearable device 11 may notify speaker 20 by a buzzer (electronic sound) or the like, without being limited to synthesized voice.

[0036] In this way, the sign language actions being performed by sign speaker 10 (i.e., the hands of sign speaker 10, etc.) are photographed from the viewpoint of sign speaker 10 (i.e., in the direction from sign speaker 10 to speaker 20). Since sign speaker 10 can photograph the sign language actions he or she is performing using wearable terminal 11 that he or she is wearing, he or she does not need to have someone else photograph him or her, and can communicate his or her intentions at a desired timing.

[0037] A case where the speaker 20 makes a speech will be described. The wearable terminal 11 collects the voice uttered by the speaker 20 (for example, "I understand"). Next, the wearable terminal 11 converts the voice into text (that is, converts the voice "I understand" into text). Next, the wearable terminal 11 displays the text (that is, the text "I understand") on the display terminal 12.

[0038] In this way, the speech uttered by speaker 20 is collected by wearable terminal 11 worn by sign speaker 10 and converted into text. Using wearable terminal 11, sign speaker 10 can not only convey the content of the sign language to speaker 20, but also recognize the content of the speech uttered by speaker 20.

[0039] <Overall composition> 2 is a diagram showing an overall configuration according to an embodiment of the present disclosure. The sign language system 1 (note that the sign language system 1 is an example of a system) includes a wearable terminal 11 and a display terminal 12 worn by a sign speaker 10 (note that the sign speaker 10 is an example of a person who makes gestures), and a speaker 20 (note that the speaker 20 is an example of a person who is conversing with the person who makes gestures). Each of these will be described below.

[0040] <<Wearable devices>> The wearable terminal 11 is a terminal equipped with a camera function, a speaker function, and a microphone function, and is used by the sign language user 10. For example, the wearable terminal 11 is a neck-worn terminal. The wearable terminal 11 can transmit and receive data to and from the display terminal 12 via any network.

[0041] [Sign language audio output] Specifically, the wearable terminal 11 captures an image of the sign language action performed by the sign speaker 10 (that is, at least the hands of the sign speaker 10), converts the sign language into text, and outputs the text as synthesized voice.

[0042] [Textual representation of the utterance] Specifically, the wearable terminal 11 collects the voice uttered by the speaker 20, converts the voice into text, and causes the display terminal 12 to display the text.

[0043] The server 13 may execute part of the processing executed by the wearable terminal 11 described in this specification. The server 13 is composed of one or more computers, and can transmit and receive data to and from the wearable terminal 11 via any network.

[0044] <<Display terminal>> The display terminal 12 is a terminal equipped with a display function and used by the sign language user 10. For example, the display terminal 12 is a smart watch. The display terminal 12 can transmit and receive data to and from the wearable terminal 11 via any network.

[0045] Specifically, the display terminal 12 receives text (specifically, the text of the voice uttered by the speaker 20) from the wearable terminal 11 and displays the text.

[0046] Although the case where the wearable terminal 11 and the display terminal 12 are separate terminals has been described in FIG. 1, the wearable terminal 11 and the display terminal 12 may be implemented as a single terminal.

[0047] <Wearable device configuration> FIG. 3 shows an example of a wearable terminal 11 according to an embodiment of the present disclosure. For example, the wearable terminal 11 is a terminal having a neck strap shape as shown in FIG. 3. For example, the wearable terminal 11 includes an imaging unit (camera) 107 and a sound output unit (speaker) 106 as shown in FIG. 3. Furthermore, the wearable terminal 11 may include a sound input unit (microphone) 105 and an operation unit 104 as shown in FIG. Each unit will be described in detail with reference to FIG. 4.

[0048] <Hardware configuration> An example of the hardware configuration of the wearable terminal 11 will be described below with reference to FIG. 4, and an example of the hardware configuration of the display terminal 12 will be described with reference to FIG.

[0049] 4 is a diagram illustrating a hardware configuration of a wearable terminal 11 according to an embodiment of the present disclosure. The wearable terminal 11 can include a control unit (processor) 101, a storage unit (memory) 102, a communication unit 103, an operation unit 104, a sound input unit (microphone) 105, a sound output unit (speaker) 106, an imaging unit (camera) 107, and various sensors 108. Each of these will be described below.

[0050] The control unit (processor) 101 is a processor that controls the wearable terminal 11. For example, the control unit (processor) 101 is a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0051] The storage unit (memory) 102 is a memory that stores any data.

[0052] The communication unit 103 connects to any network and communicates with other computers (such as the display terminal 12).

[0053] The operation unit 104 is a button or the like that is used by the sign language user 10 to input instructions to the wearable terminal 11.

[0054] The sound input unit (microphone) 105 collects sound (specifically, the voice uttered by the speaker 20).

[0055] The sound output unit (speaker) 106 outputs sound (specifically, synthesized speech of the text expressed in the sign language of the sign language user 10).

[0056] An imaging unit (camera) 107 captures still images and moving images.

[0057] The various sensors 108 are one or more arbitrary sensors such as an acceleration sensor, a GPS (Global Positioning System), and the like.

[0058] 5 is a diagram showing the hardware configuration of the display terminal 12 according to an embodiment of the present disclosure. The display terminal 12 may include a control unit (processor) 201, a storage unit (memory) 202, a communication unit 203, and a display unit 204. Each of these will be described below.

[0059] The control unit (processor) 201 is a processor that controls the display terminal 12. For example, the control unit (processor) 201 is a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0060] The storage unit (memory) 202 is a memory that stores any data.

[0061] The communication unit 203 connects to any network and communicates with other computers (such as the wearable terminal 11).

[0062] The display unit 204 is a display that displays any data (specifically, text received from the wearable terminal 11).

[0063] 6 is a functional block diagram of the control unit 101 of the wearable terminal 11 according to an embodiment of the present disclosure. The control unit 101 of the wearable terminal 11 includes a gesture detection unit 111 and a notification unit 100 (the notification unit 100 may include a text conversion unit 112 and a synthetic speech generation unit 113). The control unit 101 of the wearable terminal 11 executes a program to function as the gesture detection unit 111 and the notification unit 100 (the notification unit 100 may include the text conversion unit 112 and the synthetic speech generation unit 113).

[0064] The gesture detection unit 111 analyzes an image captured by the imaging unit (camera) 107 of the wearable terminal 11, and detects a gesture (for example, sign language) made by at least the hands of the sign user 10. Specifically, the gesture detection unit 111 detects, from the image captured by the imaging unit (camera) 107 of the wearable terminal 11, an image in which the same gesture (for example, sign language) as an image stored in an arbitrary storage unit such as the text storage unit 114 is being made.

[0065] 1, the gesture (for example, sign language) movement (i.e., the hand of the sign speaker 10) made by the sign speaker 10 is photographed from the viewpoint of the sign speaker 10 (i.e., in the direction from the sign speaker 10 to the speaker 20). Therefore, the gesture detection unit 111 detects the gesture by analyzing the image of the gesture movement seen from the side of the person making the gesture, rather than the image of the gesture movement seen from the side of the person opposite the person making the gesture.

[0066] <Gesture notification> The notification unit 100 notifies the user of the gesture detected by the gesture detection unit 111. For example, the notification unit 100 can convert the gesture into text and output the text as synthetic voice (the text conversion unit 112 and the synthetic voice generation unit 113 will be described later).

[0067] The notification unit 100 can notify the speaker 20 of the gesture detected by the gesture detection unit 111 by any means such as a buzzer (electronic sound). For example, the correspondence between the gesture and the buzzer (electronic sound) is stored in advance in any storage unit, and the notification unit 100 can output the buzzer (electronic sound) corresponding to the gesture detected by the gesture detection unit 111 from the sound output unit (speaker) 106.

[0068] As an example of notification, a case will be described below in which a gesture is converted into text and the text is output as synthesized voice.

[0069] [Sign language text conversion] The text generation unit 112 generates text from the gesture (for example, sign language) detected by the gesture detection unit 111. Specifically, the text generation unit 112 refers to the text storage unit 114 and extracts text corresponding to the gesture (for example, sign language) detected by the gesture detection unit 111. The text includes one or more sentences, words, Japanese alphabet, etc.

[0070] Gestures (specifically, images of at least the hand of a person making a gesture, taken in the direction from signer 10 to speaker 20 in FIG. 1) are associated with text and stored in text storage unit 114. Text storage unit 114 may be implemented in storage unit (memory) 102 of wearable device 11 (for example, wearable device 11 downloads gesture and text data from server 13), or may be implemented in storage unit (memory) within server 13 (for example, wearable device 11 refers to gesture and text data in server 13).

[0071] [General Gestures and Text] For example, data on general-purpose gestures and text (for example, sign language actions and the text represented by the actions) may be stored in the text storage unit 114. For example, the wearable device 11 downloads the data on general-purpose gestures and text from the server 13 and stores it in the wearable device 11.

[0072] [Gestures and text registered by each individual] For example, text storage unit 114 may store gestures and text (for example, actions decided by the wearer and the text represented by the actions) registered by the wearer of wearable device 11 (for example, sign language user 10). For example, the wearer of wearable device 11 (for example, sign language user 10) transmits an image of his / her hands making a gesture captured by wearable device 11 and text entered into an arbitrary terminal such as a personal computer (for example, a long sentence to convey to the other party) to server 13 for registration, and server 13 associates the gestures with the text. Wearable device 11 downloads the data of the gestures and text registered by the wearer from server 13 and stores them in wearable device 11.

[0073] The gesture detection unit 111 and the text generation unit 112 may use a machine-learned model to generate text from gestures (for example, sign language) made by at least the hands of the sign language user 10.

[0074] [Speech to text] The text conversion unit 112 converts into text the voice (specifically, the voice uttered by the speaker 20) collected by the sound input unit (microphone) 105 of the wearable device 11. The text conversion unit 112 can use any voice recognition method that analyzes voice and converts it into text.

[0075] The synthetic speech generation unit 113 generates synthetic speech from the text that the text generation unit 112 has converted into text the gestures (for example, sign language). The synthetic speech generation unit 113 can use any speech synthesis method for converting text into speech. The synthetic speech generation unit 113 outputs the generated synthetic speech from the sound output unit (speaker) 106 of the wearable device 11. The synthetic speech generation unit 113 may generate synthetic speech each time (that is, each time a gesture is made), or may generate synthetic speech for each text in advance.

[0076] [Generating synthetic speech on the fly] For example, when the text conversion unit 112 converts a gesture into text, the synthetic speech generation unit 113 generates synthetic speech from the text.

[0077] [Pre-generating synthetic speech] For example, the synthetic speech generation unit 113 generates synthetic speech for each text in advance, and when the text conversion unit 112 converts a gesture into text, it extracts the synthetic speech for that text from the synthetic speech that was generated in advance.

[0078] 7 is a functional block diagram of the control unit 201 of the display terminal 12 according to an embodiment of the present disclosure. The control unit 201 of the display terminal 12 includes a text receiving unit 211 and a text display unit 212. The control unit 201 of the display terminal 12 functions as the text receiving unit 211 and the text display unit 212 by executing a program.

[0079] The text receiving unit 211 receives text (specifically, the text of the voice uttered by the speaker 20) from the wearable terminal 11.

[0080] The text display unit 212 displays the text received by the text receiving unit 211 on the display unit 204 .

[0081] <Processing method> Below, a processing method for notifying a gesture will be described with reference to Figure 8, a processing method for outputting a gesture as voice will be described with reference to Figure 9, and a processing method for displaying speech as text will be described with reference to Figures 10 and 11.

[0082] FIG. 8 is a flowchart of a process for notifying a gesture according to an embodiment of the present disclosure.

[0083] In step 101 (S101), the wearable device 11 (specifically, the imaging unit (camera) 107) captures an image of a gesture (e.g., sign language) being made by the sign speaker 10 (i.e., at least the hand of the sign speaker 10). As described in FIG. 1 , the gesture (e.g., sign language) being made by the sign speaker 10 (i.e., the hand of the sign speaker 10, etc.) is captured from the viewpoint of the sign speaker 10 (i.e., in the direction from the sign speaker 10 to the speaker 20).

[0084] In step 102 (S102), the wearable terminal 11 analyzes the image captured in S101 to detect a gesture (for example, sign language) made by at least the hands of the sign user 10. Specifically, the wearable terminal 11 detects, from the image captured in S101, an image in which the same gesture (for example, sign language) as that stored in an arbitrary storage unit such as the text storage unit 114 is being made.

[0085] In step 103 (S103), the wearable terminal 11 notifies the user of the gesture detected in S102. For example, the wearable terminal 11 may convert the gesture into text and output the text as synthesized voice (described in detail in FIG. 9), or may notify the user of the gesture by a buzzer (electronic sound).

[0086] For example, when the sign user 10 inputs a command to the wearable terminal 11 (for example, pressing a button on the wearable terminal 11, starting an application in the wearable terminal 11, etc.), the wearable terminal 11 may start capturing images in S101. Also, when the sign user 10 inputs a command to the wearable terminal 11 (for example, pressing a button on the wearable terminal 11, ending an application in the wearable terminal 11, etc.), the wearable terminal 11 may end capturing images in step 101.

[0087] FIG. 9 is a flowchart of a process for outputting a gesture as synthetic voice according to an embodiment of the present disclosure.

[0088] In step 11 (S11), the wearable terminal 11 (specifically, the imaging unit (camera) 107) captures an image of a gesture (e.g., sign language) (i.e., at least the hand of the sign speaker 10) being performed by the sign speaker 10. As described in FIG. 1 , the gesture (e.g., sign language) (i.e., the hand of the sign speaker 10, etc.) being performed by the sign speaker 10 is captured from the viewpoint of the sign speaker 10 (i.e., in the direction from the sign speaker 10 to the speaker 20).

[0089] In step 12 (S12), the wearable terminal 11 analyzes the image captured in S11 to detect a gesture (for example, sign language) made by at least the hands of the sign user 10. Specifically, the wearable terminal 11 detects, from the image captured in S11, an image in which the same gesture (for example, sign language) as that stored in an arbitrary storage unit such as the text storage unit 114 is being made.

[0090] In step 13 (S13), the wearable terminal 11 converts the gesture (for example, sign language) detected in S12 into text. Specifically, the wearable terminal 11 refers to the text storage unit 114 and extracts text corresponding to the gesture (for example, sign language) detected in S12.

[0091] In step 14 (S14), the wearable device 11 generates synthetic speech of the text of S13. The wearable device 11 can use any speech synthesis method for converting text into speech. The wearable device 11 can also extract the synthetic speech of the text of S13 from synthetic speech that has been generated in advance.

[0092] In step 15 (S15), the wearable terminal 11 (specifically, the sound output unit (speaker) 106) outputs the synthesized voice generated in S14.

[0093] For example, when the sign language user 10 inputs a command to the wearable terminal 11 (for example, pressing a button on the wearable terminal 11, starting an application in the wearable terminal 11, etc.), the wearable terminal 11 may start capturing images in S11. Also, when the sign language user 10 inputs a command to the wearable terminal 11 (for example, pressing a button on the wearable terminal 11, closing an application in the wearable terminal 11, etc.), the wearable terminal 11 may end capturing images in S11.

[0094] FIG. 10 is a sequence diagram of a process for displaying speech as text according to an embodiment of the present disclosure.

[0095] In step 1 (S1), the wearable terminal 11 (specifically, the sound input unit (microphone) 105) collects the sound uttered by the speaker 20. For example, when the wearable terminal 11 collects the sound uttered by the speaker 20, the wearable terminal 11 may vibrate to allow the signer 10 to recognize the speech of the speaker 20.

[0096] In step 2 (S2), the wearable terminal 11 converts the voice collected in S1 into text. Note that the wearable terminal 11 can use any voice recognition method that analyzes voice and converts it into text.

[0097] In step 3 (S3), the wearable terminal 11 transmits the text of S2 to the display terminal 12. The display terminal 12 receives the text from the wearable terminal 11.

[0098] In step 4 (S4), the display terminal 12 displays on the display unit 204 the text received in S3.

[0099] FIG. 11 is a flowchart of a process for displaying speech as text according to an embodiment of the present disclosure.

[0100] In step 21 (S21), the display terminal 12 receives text (specifically, the text of the voice uttered by the speaker 20) from the wearable terminal 11.

[0101] In step 22 (S22), the display terminal 12 displays on the display unit 204 the text received in S21.

[0102] Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. [Explanation of symbols]

[0103] 1 Sign Language System 10 Sign Language Users 11 Wearable devices 12 Display terminal 13 Server 20 Speaker 101 Control unit (processor) 102 Memory 103 Communications Department 104 Operation section 105 Sound input unit (microphone) 106 Sound output unit (speaker) 107 Imaging unit (camera) 108 Various sensors 111 Gesture detection unit 100 Notification Department 112 Text Conversion Department 113 Synthetic speech generation unit 114 Text Storage 201 Control unit (processor) 202 Memory 203 Communications Department 204 Display section 211 Text Receiving Unit 212 Text display section

Claims

1. A wearable device equipped with a photographing function and a sound output function, The control unit of the wearable terminal Analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture made by at least the hand of the wearer of the wearable device; Notifying the detected gesture Wearable device.

2. The wearable terminal according to claim 1 , wherein the notification of the gesture is performed by converting a gesture made by at least a hand of a wearer of the wearable terminal into text and outputting the text as synthesized voice.

3. The wearable terminal according to claim 2 , further comprising a storage unit in which the gesture and the text are stored in association with each other.

4. The wearable terminal according to claim 3 , wherein general-purpose gestures and texts are stored in association with each other.

5. The wearable terminal according to claim 3 , wherein gestures registered by a wearer of the wearable terminal are stored in association with text.

6. The wearable terminal according to claim 2 , wherein the synthetic voice is generated each time a gesture is performed.

7. The wearable terminal according to claim 2 , wherein the synthetic voice is generated in advance.

8. The wearable terminal according to claim 1 , wherein the gesture is sign language.

9. The wearable device is equipped with a microphone function, The wearable terminal according to claim 1, wherein the wearable terminal collects the voice of a person with whom the wearer of the wearable terminal is having a conversation, converts the voice into text using voice recognition, and displays the text on a terminal equipped with a display function.

10. The wearable device is equipped with a microphone function, The wearable device according to claim 1 , wherein the wearable device vibrates when it picks up a voice of a person with whom the wearer of the wearable device is having a conversation.

11. A wearable device equipped with photography and sound output functions. a step of analyzing an image captured by the wearable device and including at least the hand of the wearer of the wearable device, and detecting a gesture made by at least the hand of the wearer of the wearable device; notifying the detected gesture A program to execute.

12. A method executed by a wearable device equipped with a photographing function and a sound output function, analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture performed by at least the hand of the wearer of the wearable device; notifying the detected gesture; A method comprising:

13. A system including a wearable device equipped with a photographing function and a sound output function, and a server, Analyzing an image captured by the wearable device, the image including at least the hand of the wearer of the wearable device, to detect a gesture made by at least the hand of the wearer of the wearable device; Notifying the detected gesture system.

14. The server Associating the gestures and text registered by the wearer of the wearable device, The system of claim 13 , wherein the wearable device converts gestures made by at least a hand of a wearer of the wearable device into text and outputs the text as synthesized speech.

Images