Communication support system, communication support method, and program

Abstract

To provide a communication support system, a communication support method, and a program for performing flexible word registration according to the conditions of a patient.SOLUTION: A communication support system 1 includes: a registration unit 13 that searches for words or images associated with one or more related words when a user designates the one or more related words, and registers at least one of the words or images included in the search results in a registration DB 17 in association with a paired image or paired word; and an output control unit 18 that outputs, when the user selects at least one image from among the images registered in the registration DB 17, the word associated with the selected image, in the registration DB 17.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a communication support system, a communication support method, and a program. In particular, the present disclosure relates to a communication support system, a communication support method, and a program using an electronic communication board.

Background Art

[0002] Communication support methods using images have been proposed to assist patients with aphasia, dementia, autism, etc. in expressing their intentions. For example, Patent Document 1 discloses a communication device that communicates via icons. The communication device described in Patent Document 1 analyzes an audio signal received from a counterpart device to extract word information, displays an icon image corresponding to the word information, and transmits response information to the counterpart device upon receiving a selection of the icon image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, communication support methods using a communication board that outputs words from images have been proposed for the above-mentioned patients. Currently, word registration and image registration on the communication board are performed manually, but a flexible registration method according to the patient's condition is required.

[0005] An object of the present disclosure is to provide a communication support system, a communication support method, and a program that perform flexible word registration according to the patient's condition in view of the above-mentioned problems.

Means for Solving the Problems

[0006] A communication support system relating to one aspect of this disclosure is When a user specifies one or more related terms, the registration unit searches for words or images associated with the one or more related terms, and registers at least one of the words or images included in the search results in the registration database, associating it with the target image or paired word. When the user selects at least one image from the images registered in the registration database, the registration database has an output control unit that outputs a word associated with the selected image. It is equipped with.

[0007] A communication support method relating to one aspect of this disclosure is: If the user specifies one or more related words, the system searches for words or images associated with those one or more related words, and registers at least one of the words or images included in the search results in the registration database, associating it with the target image or paired word. If the user selects at least one image from the images registered in the registration database, the registration database outputs the word associated with the selected image.

[0008] A program relating to one aspect of this disclosure is The procedure involves, when a user specifies one or more related terms, searching for words or images associated with those one or more related terms, and registering at least one of the words or images included in the search results in a registration database, associating it with the target image or paired word. If the user selects at least one image from the images registered in the registration database, the registration database shall output the word associated with the selected image. Let the computer execute it. [Effects of the Invention]

[0009] This disclosure enables the provision of a communication support system, communication support method, and program that perform flexible word registration tailored to the patient's situation. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the communication support system according to Embodiment 1. [Figure 2] This flowchart shows the flow of the communication support method according to Embodiment 1. [Figure 3] This is a block diagram showing the overall configuration of the communication support system according to Embodiment 2. [Figure 4] This is a block diagram showing the configuration of the user terminal according to Embodiment 2. [Figure 5] This flowchart shows the flow of the word registration method according to Embodiment 2. [Figure 6] This flowchart shows the flow of the word registration method according to Embodiment 2. [Figure 7] This flowchart shows the flow of the word registration method according to Embodiment 2. [Figure 8] This is a flowchart showing the flow of the word output method according to Embodiment 2. [Figure 9] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 10] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 11] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 12] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 13] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 14] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 15] This figure shows the display screen of the user terminal according to Embodiment 3. [Figure 16] It is a diagram showing a display screen of a user terminal according to Embodiment 3. [Figure 17] It is a flowchart showing the overall flow of the communication support method according to Embodiment 3. [Figure 18] It is a flowchart showing the flow of the registration method from a photograph according to Embodiment 3. [Figure 19] It is a flowchart showing the flow of the registration method from an Internet search according to Embodiment 3. [Figure 20] It is a flowchart showing the flow of the output method from fixed phrases and registered sentences according to Embodiment 3. [Figure 21] It is a flowchart showing the flow of the output method from a history according to Embodiment 3. [Figure 22] It is a block diagram showing the overall configuration of the communication support system according to Embodiment 4. [Figure 23] It is a diagram showing a display screen of a user terminal according to Embodiment 4. [Figure 24] It is a diagram showing a display unit of an external terminal according to Embodiment 4.

Mode for Carrying Out the Invention

[0011] <Problems of the Embodiment> Here, the problems of the embodiment will be explained again. In recent years, in order to support the expression of intentions by patients with aphasia, dementia, autism, etc., a communication support method using a communication board has been proposed. As the communication board, there is a laminated paper medium, but the number of contents is limited in the paper medium.

[0012] Therefore, applications (apps) have been developed to support the expression of wishes of those requiring care by displaying icon images on smartphones or tablets and outputting words when the user touches an icon image. Such apps require that icon images and words be pre-registered and linked together. In the following, the process of registering images and words together will be referred to as "word registration." Some apps allow users to register photos they have taken and link them to new words, but this lacks convenience because the pronunciation and category of the word must be entered manually.

[0013] As such, current communication boards have limitations in the amount of words they can store, and are therefore often specialized for caregiving settings where advanced communication is not particularly required. Consequently, even with communication boards, there is a challenge in that it is difficult to improve the quality of life (QoL) of those receiving care through effective communication.

[0014] Furthermore, communication boards are often used to alleviate symptoms in young children with autism. When using communication boards to alleviate symptoms in patients with autism, it is necessary to register a variety of words. Therefore, it is desirable that word registration be flexible according to the patient's situation.

[0015] This embodiment was made to solve these problems.

[0016] <Explanation of Terms> In the following, the terms listed below are defined as follows: A "word" is a string of characters and refers to a single word (a so-called word), phrase, vocabulary, or clause. "Output" refers to display, audio output, or transmission to an external source, unless otherwise specified. An "icon" is an example of an image.

[0017] <Embodiment 1> Next, Embodiment 1 of this disclosure will be described. Figure 1 is a block diagram showing the configuration of the communication support system 1 according to Embodiment 1. The communication support system 1 is a computer system including one or more computer devices that support communication for patients with aphasia, dementia, autism, etc. (hereinafter referred to as users). The communication support system 1 comprises a registration unit 13, a registration database (DB) 17, and an output control unit 18.

[0018] The registration unit 13 searches for words associated with one or more related words when the user specifies one or more related words. Related words are words specified by the user. The registration unit 13 registers at least one of the words included in the search results in the registration DB 17, associating it with the target image. The words included in the search results may be called search result words. Search result words are words recalled from the related words specified by the user. For example, search result words may be recalled from at least one of the related words specified by the user, or from all of the related words specified by the user. For example, if the search result word and the related word or combination of related words are similar in concept or pronunciation, the search result word may be considered to have been recalled from the related word. The target image may be the image included in the search results as associated with the search result word, or it may be an image specified by the user.

[0019] Furthermore, if the user specifies one or more related words, the registration unit 13 searches for images associated with one or more related words. The registration unit 13 registers at least one of the images included in the search results in the registration DB 17, associating it with the paired word. The images included in the search results may be called search result images. Search result images are images recalled from the related words specified by the user, or images corresponding to words recalled from the related words specified by the user. For example, a search result image may be recalled from at least one of the related words specified by the user, or from all of the related words specified by the user. For example, if the word corresponding to the search result image and the related word or combination of related words are similar in concept or pronunciation, the search result image may be recalled from the related word. As an example, if the related words are "red" and "fruit," the search result image may be an image of an apple or an image of a strawberry. The paired word may be the word included in the search results when associated with the search result image, or it may be a word specified by the user or a person related to the user (e.g., a caregiver). Alternatively, the paired word may be a word estimated by image recognition from at least one of the search result images.

[0020] The registration DB17 is a memory device that stores words and images in association.

[0021] When the user selects at least one image from the images registered in the registration DB 17, the output control unit 18 reads the word associated with the selected image from the registration DB 17 and outputs it.

[0022] Figure 2 is a flowchart showing the flow of the communication support method according to Embodiment 1. First, the communication support system 1 determines whether or not the user has specified a related word (S100).

[0023] If the user specifies a related term (Yes in S100), the registration unit 13 searches for words or images associated with the related term (S101). Next, if the search target is a word (Yes in S102), the registration unit 13 registers at least one of the search result words included in the search results in the registration DB 17, associating it with the target image (S103). Then the communication support system 1 proceeds to process S105.

[0024] For example, if the related words are "red" and "fruit," the search results may include the search term "apple." The registration unit 13 then registers the search term "apple" in association with the target image, which is an image of an apple. If the search results include multiple search terms, such as "apple" and "strawberry," the registration unit 13 may select at least one search term from the search results based on its relevance to the related words, or it may select it by accepting a selection operation from the user.

[0025] Meanwhile, if the search target is an image (No in S102), the registration unit 13 registers at least one of the search result images included in the search results in the registration DB 17, associating it with the paired word (S104). Then the communication support system 1 proceeds to processing in S105.

[0026] For example, if the related words are "red" and "fruit," the search results may include search result images of apples. The registration unit 13 then registers the search result images of apples in association with the pair word "apple" estimated by image recognition. If the search results include multiple search result images, such as images of apples and images of strawberries, the registration unit 13 may select at least one search result image from the search results based on its relevance to the related words, or it may select it by accepting a selection operation from the user.

[0027] On the other hand, if the user does not specify a related word (No in S100), the communication support system 1 proceeds to S105.

[0028] In S105, the communication support system 1 determines whether the user has selected at least one image registered in the registration DB 17 (S105). If the user has selected an image (Yes in S105), the output control unit 18 reads the word associated with the selected image from the registration DB 17 and outputs it (S106), and terminates the process. On the other hand, if the user has not selected an image (No in S105), the process terminates.

[0029] As described above, according to Embodiment 1, the communication support system 1 simplifies word registration by obtaining the word or image that the user wants to register through search, thereby simplifying manual operation by the user. Furthermore, since the communication support system 1 can find the word that the user wants to register by searching using related words, it enables flexible word registration tailored to the user's situation (e.g., symptoms). This makes it easy to create a communication board that can output words tailored to the user's situation.

[0030] <Embodiment 2> Next, Embodiment 2 of this disclosure will be described. Embodiment 2 provides a function to search for and register new words from a group of related words via a network. Embodiment 2 also provides a highly flexible function that allows for easy registration of new words by analyzing images captured by a camera using AI (Artificial Intelligence) and estimating names from the images. These functions make it possible to create a communication board with a high degree of flexibility and easy word expansion. This makes it possible to provide a communication board that is tailored to the context of the patient's living environment or behavior. It also makes it possible to provide a highly versatile communication board that can be used commonly in public facilities, etc.

[0031] Figure 3 is a block diagram showing the overall configuration of the communication support system 1a according to Embodiment 2. The communication support system 1a is an example of the communication support system 1 described above. The communication support system 1a comprises a user terminal 10a, a search server 50, and a search database (DB) 60. The user terminal 10a and the search server 50 are able to communicate via a network N. The network N is a wired or wireless communication line. Examples of wireless communication lines include Wi-Fi® and mobile communication networks. For example, the network N may be at least one of a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet, or a combination thereof.

[0032] User terminal 10a is a terminal device such as a smartphone or tablet used by the user, and is an example of a communication board. User terminal 10a has a word registration function and a word output function. In Embodiment 2, the user terminal 10a has a word registration function corresponding to each registration mode. For example, in the first registration mode, when the user terminal 10a receives a specification of related words from the user, it sends a search request including the related words to the search server 50. The user terminal 10a then receives the search results from the search server 50 and registers the words based on the search results. For example, in the second registration mode, the user terminal 10a estimates words from images captured by the camera using image recognition and registers the words based on the estimation results. Furthermore, in output mode, when the user terminal 10a is executing the word output function, it displays the registered images. When the user terminal 10a receives an image selection from the user, it outputs the word associated with the image.

[0033] When the search server 50 receives a search request, it refers to the search database 60 and retrieves words, images, or sets thereof associated with the relevant words included in the search request from the search database 60. The search server 50 then returns the retrieved words, images, or sets thereof to the user terminal 10a as search results.

[0034] The search DB60 is a storage device that stores one or more related words, words, and images in relation to each other.

[0035] Figure 4 is a block diagram showing the configuration of a user terminal 10a according to Embodiment 2. The user terminal 10a includes an input unit 11, a selection unit 12, a registration unit 13a, a camera 16, a registration DB 17a, an output control unit 18a, and an output unit 19.

[0036] The input unit 11 accepts selection or input operations from the user.

[0037] When the selection unit 12 receives a selection of a registration mode from the user, it selects one registration mode from among several registration modes. The selection unit 12 then notifies the registration unit 13a of the selected registration mode. The selection unit 12 also notifies the output control unit 18a of the selection of an output mode from the user.

[0038] The registration unit 13a is an example of the registration unit 13 described above. The registration unit 13a includes a first registration unit 14 that operates when the selection unit 12 selects the first registration mode or the first' registration mode, and a second registration unit 15 that operates when the selection unit 12 selects the second registration mode.

[0039] In the first registration mode, the first registration unit 14 accepts one or more related words from the user. The first registration unit 14 then requests the search server 50 to search for words associated with the one or more related words. The first registration unit 14 automatically or by user input selects at least one of the search result words included in the search results received from the search server 50. Here, if the search results include an image corresponding to the search result word, the first registration unit 14 designates that image as the paired image. The first registration unit 14 may also designate an image taken by the camera 16 or an image registered in the registration DB 17a as the paired image if selected by the user. The first registration unit 14 then registers the selected search result word in the registration DB 17a, associating it with the paired image. This allows for easy word registration without manual naming. Furthermore, since users can find the words they want to use through searching, flexible word registration tailored to the user's situation can be achieved.

[0040] The first registration unit 14 accepts one or more related words from the user even in the first' registration mode. The first registration unit 14 then requests the search server 50 to search for images associated with one or more related words. The first registration unit 14 automatically or by user operation selects at least one of the search result images included in the search results received from the search server 50. Here, if the search results include a word corresponding to the search result image, the first registration unit 14 designates that word as a pair word. The first registration unit 14 may also designate words entered by the user's assistant or words estimated by image recognition from the selected search result image as pair words. The first registration unit 14 then registers the selected search result image in the registration DB 17a, associating it with the pair word. This allows for easy word registration without manual naming. Furthermore, since the user can find the image they want to use through searching, flexible word registration tailored to the user's situation can be achieved.

[0041] In this way, the first registration unit 14 estimates words related to the related words selected by the user or a person related to the user. Therefore, the complexity of input can be reduced.

[0042] In the second registration mode, when the user specifies an image captured by the camera 16, the second registration unit 15 registers words estimated from the captured image by image recognition in the registration DB 17a, associating them with the captured image. The user specifying an image captured by the camera 16 may mean that the camera 16 generates an image through user operation. If there are multiple estimated words, the second registration unit 15 may automatically or through user operation select at least one word. This allows for easy word registration without manual naming. Furthermore, increasing the variations in registration methods enhances the flexibility of word registration.

[0043] Camera 16 is a camera that is activated when the second registration mode is selected and generates captured images.

[0044] Registration DB17a is an example of the above-described registration DB17. Registration DB17a stores words and images in association. Alternatively, registration DB17a may also store categories in association with words.

[0045] The output control unit 18a is an example of the output control unit 18 described above. In output mode, the output control unit 18a displays the image registered in the registration DB 17a on the display unit 20. Then, in output mode, when the input unit 11 accepts the selection of an image, the output control unit 18a refers to the registration DB 17a and reads out the word registered in the registration DB 17a that corresponds to the selected image. The output control unit 18a then displays the read word on the display unit 20 or outputs it as sound to the audio output unit 21.

[0046] The output unit 19 includes a display unit 20 and an audio output unit 21. The display unit 20 is a display device. The display unit 20 may be integrated with the input unit 11, such as a touch panel. The audio output unit 21 is a speaker.

[0047] Figures 5-7 are flowcharts illustrating the flow of the word registration method according to Embodiment 2. First, the selection unit 12 of the user terminal 10a determines whether or not it has received a selection operation for the first registration mode by the user via the input unit 11 (S200). If the input unit 11 has received a selection operation for the first registration mode by the user (Yes in S200), the selection unit 12 selects the first registration mode (S201) and notifies the registration unit 13a of this. The first registration unit 14 of the registration unit 13a, upon receiving the notification, accepts the specification of a related word (S202). For example, the first registration unit 14 accepts text input of a related word by the user via the input unit 11. Also, for example, if the user selects a captured image taken by the camera 16 or an image registered in the registration DB 17a, the first registration unit 14 determines that the user has specified a related word. More specifically, when the first registration unit 14 receives a selection operation for an image captured by the camera 16 or an image registered in the registration DB 17a, it estimates a word from the image using a machine learning model for image recognition. The first registration unit 14 then accepts the designation of related words by designating the estimated word as a related word. In this example, it is assumed that the first registration unit 14 has received the designation of multiple related words.

[0048] Next, the first registration unit 14 sends a request to the search server 50 to search for a word associated with any of the related words (S203). The first registration unit 14 then receives the search results from the search server 50 (S204).

[0049] The first registration unit 14, having obtained the search results, outputs the search result words included in the search results to the output unit 19 via the output control unit 18 (S205). The first registration unit 14 then accepts a selection operation for at least one search result word from the outputted search result words via the input unit 11 (S206). The first registration unit 14 then accepts the specification of a pair image via the input unit 11 (S207). Finally, the first registration unit 14 registers the at least one search result word selected in S206 in the registration DB 17a, associating it with the pair image (S208). This allows users, especially patients with aphasia, to appropriately register words they wish to use in conversation.

[0050] On the other hand, if the input unit 11 has not accepted the user's operation to select the first registration mode (No in S200), the user terminal 10a proceeds to S210. In S210, the selection unit 12 determines whether the input unit 11 has accepted the user's operation to select the first' registration mode. If the input unit 11 has accepted the user's operation to select the first' registration mode (Yes in S210), the selection unit 12 selects the first' registration mode (S211) and notifies the registration unit 13a of this. The first registration unit 14 of the registration unit 13a, upon receiving the notification, accepts the specification of related words in the same manner as in S202 (S212). In this example as well, it is assumed that the first registration unit 14 has accepted the specification of multiple related words.

[0051] Next, the first registration unit 14 sends a request to the search server 50 to search for images associated with any of the related words (S213). The first registration unit 14 then receives the search results from the search server 50 (S214).

[0052] The first registration unit 14, having acquired the search results, displays the search result images included in the search results on the display unit 20 via the output control unit 18 (S215). The first registration unit 14 then accepts a selection operation for at least one of the displayed search result images via the input unit 11 (S216).

[0053] The first registration unit 14 then accepts the designation of a pair of words (S217). The pair of words to be designated may be text input via the input unit 11. Finally, the first registration unit 14 registers at least one search result image selected in S216 in the registration DB 17a, associating it with the pair of images (S219).

[0054] Alternatively, instead of S217-218, the first registration unit 14 may perform the following processing. For example, the first registration unit 14 outputs the words estimated by image recognition from the search result images selected in S217 to the output unit 19 via the output control unit 18a. If the user selects at least one of the output words, the first registration unit 14 registers the selected at least one word as a pair word in the registration DB 17a, associating it with the search image selected in S216. If the user does not select any of the output words, the first registration unit 14 accepts word input from the user and registers the input word in the registration DB 17a, associating it with the search result image selected in S217. If the desired word is not available, the flexibility of word registration can be further increased by allowing editing or manual input.

[0055] If the input unit 11 does not accept the selection operation for the first registration mode (No in S210), the user terminal 10a proceeds to S220. In S220, the selection unit 12 determines whether the input unit 11 has accepted the selection operation for the second registration mode. If the input unit 11 has accepted the user's selection operation for the second registration mode (Yes in S220), the selection unit 12 selects the second registration mode (S221) and notifies the registration unit 13a of this. Upon receiving the notification, the second registration unit 15 of the registration unit 13a activates the camera 16 (S222). Then, at the user's operation, the camera 16 takes a picture of the object, person, animal, or landscape (hereinafter referred to as "object") that the user wants to register (S223). Next, the second registration unit 15 estimates a word from the captured image by image recognition (S224). The second registration unit 15 then outputs the word estimated from the captured image to the output unit 19 (S225). If multiple words are estimated, the second registration unit 15 may output multiple words to the output unit 19. The second registration unit 15 may display the word on the display unit 20 or output it as sound to the sound output unit 21. The user determines whether the outputted word is the word they want to register by viewing the display or listening to the sound. If the user selects at least one word from the outputted words, the second registration unit 15 accepts the selection operation via the input unit 11 (S226). The second registration unit 15 registers the selected at least one word in the registration DB 17a, associating it with the captured image (S227). This allows users, especially patients with aphasia, to appropriately register words they want to use in conversation.

[0056] Furthermore, if the user does not select any of the outputted words in S226, the second registration unit 15 may, instead of proceeding to S227, accept word input from the user and register the input word in the registration DB 17a, associating it with the captured image. By allowing editing or manual input if the desired word is not available, the degree of flexibility in word registration can be further increased.

[0057] In S220, the selection unit 12 terminates processing if the input unit 11 has not accepted the user's selection operation for the second registration mode (No in S220).

[0058] Figure 8 is a flowchart showing the flow of the word output method according to Embodiment 2. First, the selection unit 12 of the user terminal 10a determines whether or not it has received a selection operation of the output mode from the user via the input unit 11 (S230). If the input unit 11 has received a selection operation of the output mode from the user (Yes in S230), the selection unit 12 selects an output mode (S231) and notifies the output control unit 18a of this.

[0059] The output control unit 18a refers to the registration DB 17a and displays the images registered in the registration DB 17a on the display unit 20 (S232). Then, if the user selects at least one of the displayed images, the output control unit 18a accepts the image selection operation via the input unit 11 (S233). The output control unit 18a outputs the word associated with the selected image in the registration DB 17a to the display unit 20 or the audio output unit 21 (S234).

[0060] Thus, according to Embodiment 2, the user terminal 10a can function as a communication board capable of outputting words tailored to the user's situation through highly flexible word registration. Although the words used in speech differ from facility to facility, the flexible registration method according to Embodiment 2 makes it possible to create a communication board that is independent of the place of use or the type or severity of the disease.

[0061] In the above-described embodiment 2, the communication support method was performed by the user terminal 10a, but it may also be performed by a server (not shown) that is connected to the user terminal 10a via the network N in a way that allows communication. Specifically, the server may have the functions of a selection unit 12, a registration unit 13a, and an output control unit 18a.

[0062] <Embodiment 3> Next, Embodiment 3 of this disclosure will be described. Embodiment 3 is a detailed specific example of Embodiment 2. In Embodiment 3, the user terminal 10a is a smartphone or a tablet terminal. Also in Embodiment 3, the network N includes the internet. Figures 9 to 16 show the display screen 100 of the user terminal 10a according to Embodiment 3. The display screen 100 shown in Figure 9 represents the initial screen. The display screen 100 includes icons 101 to 104. Icon 101 is an icon that allows users to initiate a dialogue about their own body. Icon 102 is an icon used by the user to initiate a conversation about something located indoors, a short distance away from them. Icon 103 is an icon used by the user to initiate a conversation about things that exist outdoors. Icon 104 is an icon that allows users to omit the subject and noun and convey a state through descriptive expressions.

[0063] Display screen 100 also includes icons 105-108. Icon 105 is an icon for registering words from the camera. Specifically, icon 105 is an icon for registering words by taking a picture of an object using the camera 16 provided on the user terminal 10a, for the purpose of using it in new conversations. Icon 106 is an icon for registering words from internet searches. Specifically, icon 106 is an icon for searching the internet for a target word (target word) from multiple related words. When icon 106 is selected on user terminal 10a, if the user selects, for example, "red" and "fruit," the icon will suggest strawberries and apples to the user. Icon 107 is an icon used to speak pre-set phrases and registered sentences. Icon 108 is an icon for uttering words from the history of words used in the dialogue.

[0064] If icon 101 is selected, the user terminal 10a transitions the display screen to the display screen 100 shown in Figure 10. Similarly, if icon 102 is selected in Figure 9, the user terminal 10a transitions the display screen to a display screen related to objects that are likely to be found indoors. On the subsequent display screen, the user can decide which object to interact with. If icon 103 is selected, the user terminal 10a transitions the display screen to a display screen related to objects found outdoors.

[0065] Figure 10 allows you to select which part of the body to interact with. The display screen 100 includes icons 201 to 206. Icon 201 is an icon used when a user wants to talk about a part of their back, allowing them to select a word corresponding to one of the parts of their back. Icon 202 is an icon used by the user to select a word corresponding to one of the front parts of their body when they wish to interact with a part of their body. When icons 201 and 202 are selected, the voice output unit 21 of the display unit 20 of the user terminal 10a reads out the name. The user terminal 10a allows the user to select multiple icons as the subject of the interaction. Icon 203 is used to select a word that corresponds to the whole body or the first person. Icon 205 is used to return to the previous screen. Conversely, icon 206 is used to advance from display screen 100 to the next screen.

[0066] In Figure 10, if the right foot icon 204 is selected on the display screen 100, the user terminal 10a transitions the display screen to the display screen 100 shown in Figure 11. In Figure 11, the user terminal 10a first displays only the right foot icon 302 in the selected icon display area 301. At this time, icons corresponding to nouns related to "(right) foot" are preferentially displayed in the noun display area 306. For inferring related nouns, the Markov chain Monte Carlo method or the N-fold Markov chain method can be used from pre-trained data. If the shoe icon 307 is selected, the user terminal 10a displays an icon 303 similar to the selected shoe icon 307 in the selected icon display area 301. Then, the user terminal 10a displays icons in the verb display area 308 that correspond to verbs inferred from "(right) foot" and "shoe". For example, icon 309 indicates "to put on (shoes)", and icon 310 indicates "to take off (shoes)". Let's assume that the "Put on" icon 309 is selected. In this case, the user terminal 10a adds an icon 304 similar to the "Put on" icon 309 to the selected icon display area 301.

[0067] Next, the user selects an adjective describing the feelings or state associated with "(right) foot," "shoes," and "wearing." In the adjective display area 311, icons 312-314 indicating emotions and icons 315 indicating other adjectives that are highly relevant to the selected word are preferentially displayed. For example, the user selects the "itchy" icon 315. The user terminal 10a then adds an icon 316 similar to the "itchy" icon 315 to the selected icon display area 301. In this state, if the speech icon 317 is selected, the voice output unit 21 of the user terminal 10a speaks in a machine voice: "right foot," "shoes," "wear," and "itchy." From this voice output, the person interacting with the user can understand that wearing shoes makes them itchy, thus establishing a dialogue.

[0068] In Figure 9, if either the indoor icon 102 or the outdoor icon 103 is selected, the user terminal 10a similarly estimates and suggests appropriate nouns, verbs, and adjectives for the selected noun based on the combination of information already selected. This makes dialogue using the communication board more efficient.

[0069] In the display screen 100 shown in Figure 11, when the user selects the icon 318 for registering from a photograph, the user terminal 10a recognizes that the second registration mode has been selected and selects the second registration mode. The photograph is an example of a photograph taken as described above. In the second registration mode, the user can register a new word by taking a photograph using the camera 16. The second registration unit 15 of the user terminal 10a uses a machine learning model to automatically extract and classify attributes from the photographed image and identify the name (word). The attributes, classification (category), and word are examples of paired words as described above. The second registration unit 15 of the user terminal 10a then associates the image with the attributes, classification, and word and registers them in the registration DB 17a. If automatic classification occurs that the user did not intend, the user can manually select or input a name from the extracted attributes and classification. A typical service that extracts attributes from an image and identifies a name in this way is Google Cloud VISION. Deep learning using a neural network may also be used as a means of identifying the name.

[0070] Figures 12 and 13 show the display screen 100 that has been transitioned from the display screen 100 in Figure 11. The display screen 100 in Figure 12 is displayed after the icon 318 for registering from a photograph is selected in Figure 11 and the camera 16 takes a photograph. The display unit 20 of the user terminal 10a displays the name (word) corresponding to the photograph 401 in the display area 402. The display unit 20 of the user terminal 10a may also display attributes and classifications related to the multiple icons selected in Figure 11.

[0071] Specifically, in the display screen 100 of Figure 12, the user can activate the camera 16 and take a photograph by selecting the shutter icon 403. The display unit 20 of the user terminal 10a then displays the word estimated from the photograph in the display area 402. For example, icon 404 was generated because the photograph was estimated to mean a couple. Icon 405 was generated because the photograph was estimated to mean a salaryman. Icon 406 was generated because the photograph was estimated to mean going out. When the user selects one of the estimated icons, the voice output unit 21 of the user terminal 10a may read aloud the name (word) corresponding to the selected icon. This allows the user to confirm whether the word read aloud is the word they wanted. Suppose the user selects the "going out" icon 406 from the icons corresponding to the word estimated from photograph 401, and then selects the registration icon 407. In this case, the second registration unit 15 of the user terminal 10a associates photograph 401 with "going out" and registers it in the registration DB 17a. The user terminal 10a then transitions its display screen to the display screen 100 shown in Figure 13.

[0072] As shown in Figure 13, the user terminal 10a adds an icon 501 indicating going out to the verb display area 308 of the display screen 100. Here, in addition to the foot icon, the shoe icon, and the "put on" icon, the newly registered going out icon 501 and the icon 502 indicating a positive emotion are selected, and the speech icon 503 is selected. In this case, the voice output unit 21 of the user terminal 10a may output the voice "I want to put on my shoes and go out."

[0073] In Figure 11, if icon 319 is selected, the selection unit 12 of the user terminal 10a switches to the first registration mode. The user terminal 10a then transitions its display screen to the display screen shown in Figure 14. In Figure 14, the user terminal 10a searches for a word (target word) that the user wishes to register from among several related words related to the target word. For example, suppose the user wants to register the new target word "going out". In this case, the user can select related words related to that target word from the list of registered words displayed by selecting icon 601.

[0074] The user can select related words from new word estimations using the same method as in the second registration mode, utilizing the camera shooting function activated by selecting icon 602. The camera shooting function is the same as the function described in Figure 12. Here, suppose the user selects the "walk" icon 603 and the "city" icon 604 as icons corresponding to related words in order to add "going out". In this case, the first registration unit 14 of the user terminal 10a performs a search using the related words. Specifically, when the search icon 605 is selected while icons corresponding to related words are selected, the first registration unit 14 of the user terminal 10a sends a search request including related words to the search server 50 via the internet. As a result, the first registration unit 14 of the user terminal 10a obtains words estimated from the related words "city" and "walk" and their corresponding images as search results, and displays the set of words and images included in the search results as target word candidates in the search results display area 606. The above words correspond to search result words, and the above images correspond to search result images. If the user selects one of the images corresponding to a candidate target word, the voice output unit 21 of the user terminal 10a may read aloud the word corresponding to the selected image. This allows the user to confirm the word. The user then selects the image corresponding to the intended target word from the images displayed in the search results display area 606 and selects the registration icon 607. As a result, the first registration unit 14 of the user terminal 10a associates the target word with the selected image and registers it as a new word in the registration DB 17a.

[0075] Next, we will explain the case where the icon 105 for registering a word from the camera is selected in Figure 9. In this case, the user terminal 10a transitions the display screen to a display screen 100 similar to that in Figure 12. The functions executed on the display screen 100 after the transition are the same as the functions explained when transitioning from Figure 11 to Figure 12.

[0076] Next, we will explain the case in Figure 9 where the icon 106 for registering a word from an internet search is selected. In this case, the user terminal 10a transitions the display screen to the same display screen 100 as in Figure 13. The functions executed on the display screen 100 after the transition are the same as the functions explained in Figure 13.

[0077] Next, we will explain the case in Figure 9 where an icon 107 for speaking from a pre-defined phrase or registered sentence is selected. In this case, the user terminal 10a transitions its display screen to the display screen 100 shown in Figure 15. The display screen 100 in Figure 15 includes a list 701, which is a list of pre-defined phrases and registered sentences, and a speech icon 703. On this screen, the user selects a set of icons they want to speak from the list 701, for example, an icon group 702. If the icon group 702 is selected, the voice output unit 21 of the user terminal 10a speaks the sentence corresponding to the icon group 702 for confirmation. If the selected sentence is the sentence the user wants to speak, the user can speak by selecting the speech icon 703.

[0078] Next, we will explain the case in Figure 9 where the icon 108 for speaking is selected from the history. In this case, the user terminal 10a transitions the display screen to the display screen 100 shown in Figure 16. The display screen 100 in Figure 16 includes a list 801, which is a list of words used in the past n utterances (where n is a predetermined natural number), and an utterance icon 803. On this screen, the user selects a set of icons to speak from the list 801, for example, an icon group 802. If the icon group 802 is selected, the voice output unit 21 of the user terminal 10a speaks the sentence corresponding to the icon group 802 for confirmation. If the selected sentence is the sentence the user wants to speak, the user can speak by selecting the utterance icon 803.

[0079] By implementing the various functions described above, the user terminal 10a functions as a communication board that allows for easy registration of new words.

[0080] Next, the operation of the user terminal 10a according to Embodiment 3 will be explained with reference to Figures 9 to 16 described above. Figure 17 is a flowchart showing the overall flow of the communication support method according to Embodiment 3. In S1 after startup (START in Figure 17), the user terminal 10a displays the display screen 100 (initial screen) shown in Figure 9. In Figure 9, the user can select an icon from eight icons. Next, in S2, if the icon 105 for registering a word from the camera in Figure 9 is selected (Yes in S2), the user terminal 10a executes the function of registering a word from a photograph (S3).

[0081] If icon 105 is not selected in S2 (No in S2), user terminal 10a proceeds to S4. In S4, if icon 106 for registering a word from an internet search as shown in Figure 9 is selected (Yes in S4), user terminal 10a executes the function to register a word from an internet search (S5).

[0082] If icon 106 is not selected in S3 (No in S4), the user terminal 10a proceeds to S6. In S6, if icon 107 for speaking from the predefined phrases and registered sentences shown in Figure 9 is selected (Yes in S6), the user terminal 10a executes the function to speak from the predefined phrases and registered sentences (S7).

[0083] If icon 107 is not selected in S6 (No in S6), the user terminal 10a proceeds to S8. In S8, if icon 108 for speaking from the history in Figure 9 is selected (Yes in S7), the user terminal 10a executes the function to speak from the history (S9).

[0084] If icon 108 is not selected in S8 (No in S8), the user terminal 10a proceeds to S10. In S10, if icon 104, which conveys the state using the descriptive expression in Figure 9, is selected (Yes in S10), the user terminal 10a proceeds to S15. In S15, the user terminal 10a displays icons representing the state and descriptive expression icons (S15), and executes a function to speak in short sentences with omitted subjects and verbs. Then the user terminal 10a proceeds to S18.

[0085] If icon 104 is not selected in S10 (No in S10), the user terminal 10a determines whether another icon was selected (S11).

[0086] If the icon selected in S11 is the body icon 101 in Figure 9, the user terminal 10a displays an icon corresponding to a noun related to the body that can be selected as a subject, as shown in Figure 11 (S12). Then, in S16, the user terminal 10a accepts the selection of an icon from the user. At this time, the user terminal 10a may accept the selection of multiple icons. When an icon selection is accepted, the user terminal 10a recognizes that the word corresponding to the icon has been selected.

[0087] If the icon selected in S11 is the indoor icon 102 in Figure 9, the user terminal 10a displays an icon corresponding to a noun that can be selected as a subject and is related to indoors (S13). In this case as well, in S16, the user terminal 10a may accept the selection of one or more icons from the user.

[0088] If the icon selected in S11 is the outdoor icon 103 in Figure 9, the user terminal 10a displays an icon corresponding to a noun that can be selected as a subject and is related to the outdoors (S14). For example, if the user terminal 10a is used in a nursing home, icons representing nouns such as "home," "supermarket," or "park" will be displayed. In this case as well, in S16, the user terminal 10a may accept the selection of one or more icons from the user.

[0089] In S12, S13, and S14, once word selection is complete, the user terminal 10a transitions the display screen to the display screen 100 in Figure 11. Then, in S17, the user terminal 10a accepts the selection of a verb or adjective. In the noun display area 306, verb display area 308, and adjective display area 311 of Figure 11, highly relevant words estimated from the selected word are displayed. If the user selects the icon 318 for registering a word from a photograph, the user terminal 10a proceeds to S3. Similarly, if the user selects the icon 319 for searching from the internet, the user terminal 10a proceeds to S5. Furthermore, if the icon included in the display screen 100 of Figure 11 is not the intended icon, the user can select the icon 320 for searching for a registered word. In this case, the user can search for and select a word registered in the registration DB 17a of the user terminal 10a. Word selection can be repeated until it matches what the user wants to say. The user terminal 10a then proceeds to process S18.

[0090] If the speech icon 317 in Figure 11 is selected in S18, the user terminal 10a will speak the content the user wants to say in machine voice (S19). This establishes a dialogue through word selection.

[0091] Next, we will explain the function for registering words from a photograph (S3 in Figure 17). Figure 18 is a flowchart showing the flow of the registration method from a photograph according to Embodiment 3. If icon 105 in Figure 9 or icon 318 in Figure 11 is selected, S3-1 is executed.

[0092] In S3-1, the user terminal 10a displays a photo registration function screen as shown in Figure 12. Next, in S3-2, the user terminal 10a takes a photograph corresponding to the word to be registered, based on user operation. When the icon 403 in Figure 12 is selected, the user terminal 10a activates the camera 16. This allows the user to take a photograph. The photograph 401 taken is displayed on the display screen 100. Next, in S3-3, the user terminal 10a analyzes the captured photograph using a machine learning model to estimate the attributes, classifications, and names of the photograph. Next, in S3-4, the user terminal 10a displays the analysis results, that is, the words estimated from the photograph, in the display area 402 shown in Figure 12. Next, in S3-5, if the user selects one of the displayed analysis results, the user terminal 10a will utter the selected word (target word candidate). This allows the user to confirm the word. Next, in S3-6, the user determines whether the word they intended exists. If it does (Yes in S3-6), the user selects the word they want to register and the registration icon 407. Then, in S3-7, the user terminal 10a registers the selected word as a newly available word in the registration DB 17a, associating it with the photograph.

[0093] Furthermore, if the word intended by the user is not found in S3-6 (No in S3-6), the user selects the "Try Again" icon 409 in Figure 12. This causes the user terminal 10a to return to processing S3-2.

[0094] Next, we will explain the function for registering words from an internet search (S5 in Figure 17). Figure 19 is a flowchart showing the flow of the registration method from an internet search according to Embodiment 3. If icon 106 in Figure 9 or icon 319 in Figure 11 is selected, S5-1 is executed.

[0095] In S5-1, the user terminal 10a displays a registration function screen from an internet search, as shown in Figure 14. Next, in S5-2, the user terminal 10a accepts the specification of multiple related words related to the word (target word) that the user wants to search for. Methods for specifying related words include selecting from words registered in the registration DB 17a using icon 601 in Figure 14, and creating a new word by taking a picture using icon 602 and then selecting it. If an incorrect word is specified as a related word, the user deletes it using icon 608. Next, in S5-3, if the search icon 605 is selected, the user terminal 10a searches the internet for words that can be recalled from the selected related words. Next, in S5-4, the user terminal 10a retrieves the search results and displays the images corresponding to the words included in the search results in the search results display area 606. Next, in S5-5, if the user selects an image, the user terminal 10a speaks a word corresponding to the selected image as a target word candidate. This allows the user to confirm whether it is the intended word. Next, in S5-6, the user determines whether the word they intended exists. If it does (Yes in S5-6), the user selects the word they want to register and the registration icon 607. Then, in S5-7, the user terminal 10a registers the selected word as a newly available word in the registration DB 17a, associating it with an image.

[0096] Furthermore, if the word intended by the user does not exist in S5-6 (No in S5-6), the user selects the "Try Again" icon 609 in Figure 14. This causes the user terminal 10a to return to S5-1 and start over from the beginning.

[0097] Next, we will explain the function of speaking from pre-defined phrases and registered sentences (S7 in Figure 17). Figure 20 is a flowchart showing the flow of the output method from pre-defined phrases and registered sentences according to Embodiment 3.

[0098] First, in S7-1, the user terminal 10a displays a list of predefined and registered phrases, as shown in Figure 15. Next, in S7-2, the user terminal 10a receives a selection from the user for a group of icons included in list 701. Next, in S7-3, the user terminal 10a speaks the text corresponding to the selected group of icons. The user then confirms the content. Next, in S7-4, if the speech icon 703 is selected, the user terminal 10a communicates using the text corresponding to the group of icons that was selected immediately before.

[0099] Next, we will explain the function of speaking from the history (S9 in Figure 17). Figure 21 is a flowchart showing the flow of the output method from the history according to Embodiment 3.

[0100] First, in S9-1, the user terminal 10a displays a speech history list screen, as shown in Figure 16. This list screen contains the most recent n history sentences. Next, in S9-2, the user terminal 10a receives a selection from the user of a group of icons recently used in speech, which are included in list 801. Next, in S9-3, the user terminal 10a speaks the text corresponding to the selected group of icons. The user then confirms the content. Next, in S9-4, if the speech icon 803 is selected, the user terminal 10a communicates using the text corresponding to the group of icons that was selected immediately before.

[0101] Thus, the user terminal 10a of Embodiment 3 provides the same effects as Embodiment 2.

[0102] <Embodiment 4> Next, Embodiment 4 of the present disclosure will be described. Figure 22 is a block diagram showing the overall configuration of the communication support system 1b according to Embodiment 4. The communication support system 1b includes a user terminal 10b instead of a user terminal 10a, and an external terminal 70. In Embodiment 4, the user terminal 10b has a second speech means that transmits speech content selected by the user to the external terminal 70 via the network N, in addition to or instead of a first speech means that outputs to the output unit 19 of the user terminal 10b.

[0103] For example, the external terminal 70 can use an external communication application (hereinafter referred to as the external application). In this case, the user terminal 10b may send a sentence consisting of selected words to the external terminal 70 using an API provided by the external application.

[0104] Figure 23 shows the display screen 1601 of the user terminal 10b according to Embodiment 4. The display screen 1601 includes a speech icon 1602 and a display area 1603 for selecting a speech means. When the speech icon 1602 is selected, the user selects from the display area 1603 whether to speak through the communication board (i.e., the first speech means) or through an external application (i.e., the second speech means).

[0105] If the icon 1604, which indicates the second means of speech, is selected, the user terminal 10b first outputs the text to be sent to the output unit 19. This allows the user to confirm the content. After confirmation, the user terminal 10b selects a destination registered in the external application via user operation. The destination is selected by selecting the icon indicating the destination. By sending the text on the external application, interaction with the external terminal 70 is enabled.

[0106] Figure 24 shows the display unit 1702 of the external terminal 70 according to Embodiment 4. Text sent on an external application is sent to the destination external terminal 70. The external terminal 70 displays the received text on the display unit 1702. If the external terminal 70 receives a group of icons instead of text, it converts the group of icons into natural language text and displays it on the display unit 1702. For example, the external terminal 70 converts the received group of icons 1704 into text 1703 and displays it on the display unit 1702. The external terminal 70 may have a function to read the text aloud. This allows the user (receiver) of the external terminal 70 to understand the received content by voice. In this way, the receiver and speaker can communicate seamlessly using natural language, pictures, and icons through the network N.

[0107] Although the embodiments described above were described in terms of hardware configuration, the invention is not limited thereto. The disclosure also allows any processing to be implemented by having a processor execute a computer program.

[0108] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals.

[0109] This disclosure is not limited to the embodiments described above, and may be modified as appropriate without departing from its intent.

[0110] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) When a user specifies one or more related terms, the registration unit searches for words or images associated with the one or more related terms, and registers at least one of the words or images included in the search results in a registration database, associating it with a paired image or paired word. When the user selects at least one image from the images registered in the registration database, the registration database has an output control unit that outputs a word associated with the selected image. A communication support system equipped with the following features. (Note 2) The aforementioned registration unit is When the results of the search are obtained, the words or images included in the search results are output to the output unit. If the user selects at least one word or image from the outputted words or images, the selected at least one word or image is registered in the registration database in association with the paired image or the paired word. The communication support system described in Appendix 1. (Note 3) When the registration unit registers an image included in the search results into the registration database, it registers a word estimated from the image by image recognition as a pair word in the registration database, associating it with the image. A communication support system as described in Appendix 1 or 2. (Note 4) The aforementioned registration unit is The word estimated from the aforementioned image by image recognition is output to the output unit. If the user selects at least one of the outputted words, the selected at least one word is registered in the registration database as a pair word, associated with the image. The communication support system described in Appendix 3. (Note 5) If the user does not select any of the outputted words, the registration unit accepts word input from the user and registers the input word in the registration database, associating it with the image. The communication support system described in Appendix 4. (Note 6) The aforementioned pair of images or the aforementioned pair of words are The image or word included in the search results, corresponding to the aforementioned word or image, or, The image or word specified by the user. A communication support system as described in Appendix 1 or 2. (Note 7) When the user selects a registration mode, the system includes a selection unit that selects one of several registration modes. The aforementioned registration unit is In the first registration mode, if the user specifies one or more related words, the system searches for words or images associated with the one or more related words, and registers at least one of the words or images included in the search results in the registration database, associating it with the paired image or paired word. In the second registration mode, when the user specifies a captured image taken by the camera, the words estimated from the captured image by image recognition are registered in the registration database in association with the captured image. A communication support system as described in any one of the items 1 through 6 of the appendix. (Note 8) The registration unit determines that, in the first registration mode, if the user selects an image captured by the camera or an image registered in the registration database, the user has specified the related term. The communication support system described in Appendix 7. (Note 9) In the second registration mode, the registration unit causes the output unit to output the word estimated from the captured image. If the user selects at least one of the outputted words, the selected at least one word is registered in the registration database in association with the captured image. A communication support system as described in Appendix 7 or 8. (Note 10) In the second registration mode, if the user does not select any of the outputted words, the registration unit accepts word input from the user and registers the input word in the registration database in association with the captured image. The communication support system described in Appendix 9. (Note 11) If the user specifies one or more related terms, the system searches for words or images associated with those one or more related terms, and registers at least one of the words or images included in the search results in the registration database, associating it with a pair of images or a pair of words. If the user selects at least one image from the images registered in the registration database, the registration database outputs the word associated with the selected image. Communication support methods. (Note 12) The procedure involves, when a user specifies one or more related terms, searching for words or images associated with those one or more related terms, and registering at least one of the words or images included in the search results in a database, associating it with a paired image or paired word. If the user selects at least one image from the images registered in the registration database, the registration database shall output the word associated with the selected image. A program to be executed by a computer. [Explanation of symbols]

[0111] 1,1a,1b Communication support system 10a, 10b User terminals 11 Input section 12 Selection Section 13,13a Registration Section 14. Registration Section 1 15. Second Registration Section 16 cameras 17,17a Registered DB 18,18a Output control unit 19 Output section 20 Display section 21 Audio output section 50 Search Servers 60 Search Database 70 External terminals N Network 100,1601 display screen 101,102,103,104,105,106,107,108,201,202,203,205,206,302,303,304,307,309,310,312,313,314,315,316,317,318,319,320,403,404,405,406,407,409,501,502,503,601,602,603,604,605,607,608,609,703,803,1602,1604 Icons 301 Selection icon display area 306 Noun display area 308 Verb display area 311 Adjective display area 401 Photo 402 Display area 606 Search Results Display Area List of 701,801 702,802 icon groups 1603 Display area 1702 Display section 1703 Sentences 1704 Selection Icons

Claims

1. A reception unit that accepts one registration mode selected by the user from among multiple registration modes, A registration unit that registers words and images in a registration database, Equipped with, When the reception unit receives the first registration mode, the registration unit requests a search for words related to one or more related words specified by the user that the user wishes to register, and registers the words selected by the user from the search results, along with images associated with the selected words, in the registration database. When the reception unit receives the second registration mode, the registration unit registers the words estimated by image recognition from the captured image specified by the user into the registration database, associating them with the captured image. The image associated with the selected word is an image included in the search results that corresponds to the selected word, or an image selected by the user. Communication support system.

2. In the second registration mode, the registration unit outputs the word estimated from the captured image. If the user selects at least one of the outputted words, the registration unit registers the selected at least one word in the registration database in association with the captured image. The communication support system according to claim 1.

3. In the second registration mode, if the user does not select any of the outputted words, the registration unit accepts word input from the user and registers the input word in the registration database in association with the captured image. The communication support system according to claim 2.

4. The communication support system according to any one of claims 1 to 3, wherein the word estimated by the image recognition is a word that indicates the attribute or behavior of a person in the captured image.

5. Computers The system accepts one registration mode selected by the user from among several registration modes. When the first registration mode is accepted, the system requests a search for words associated with one or more related words specified by the user that the user wishes to register, and registers the words selected by the user from the search results, along with images associated with the selected words, into the registration database. When the second registration mode is accepted, the words estimated by image recognition from the captured image specified by the user are registered in the registration database in association with the captured image. The image associated with the selected word is an image included in the search results that corresponds to the selected word, or an image selected by the user. Communication support methods.

6. A process to accept one registration mode selected by the user from among multiple registration modes, When the first registration mode is accepted, the system requests a search for words associated with one or more related words specified by the user that the user wishes to register, and registers the words selected by the user from the search results, along with images associated with the selected words, into the registration database. When the second registration mode is accepted, the process involves registering the word estimated by image recognition from the captured image specified by the user into the registration database, in association with the captured image. Let the computer run it, The image associated with the selected word is an image included in the search results that corresponds to the selected word, or an image selected by the user. program.

Images