Voice response system and terminal device

Abstract

To provide a voice response system which outputs different solutions in different tones so that a user can easily understand even when the solutions to one character information cannot be specified as one.SOLUTION: In a voice response device (terminal device) which responds to inputted character information via voice, processing includes response acquisition means S22 for acquiring a plurality of different responses to character information and voice output means S28 for outputting the plurality of different responses in different tones.SELECTED DRAWING: Figure 3

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This international application is a continuation of Japanese Patent Application No. 2 filed with the Japan Patent Office on June 18, 2012. This international application claims priority based on Japanese Patent Application No. 012-137065, Japanese Patent Application No. 2012-137066, and Japanese Patent Application No. 2012-137067, the entire contents of which are incorporated herein by reference. [Technical field]

[0002] The present invention relates to a voice response system that allows responses to be made by voice. [Background technology]

[0003] As the above voice response device, there is known a device that searches a dictionary for an answer to an input question and outputs the searched answer by voice (see, for example, Patent Document 1). In addition, a technology is also known that generates an answer to a question based on the content of a dialogue with a user (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4832097 [Patent Document 2] Patent No. 4924950 Summary of the Invention [Problem to be solved by the invention]

[0005] The above technique is simply set to provide one answer to one question, as specified by a dictionary. It is one aspect of the present invention to provide a voice response system that is more user-friendly for users. [Means for solving the problem]

[0006] In the first aspect of the invention, A voice response system including a request source device that is a request source of a request, a providing source device that is an information providing source, and a server capable of communicating with the request source device and the providing source device, the providing device is configured to be able to set whether or not to permit information provision; an information acquisition unit configured to acquire a request from the request source device, and if the request includes a request that specifies the providing source device, transmit the request to the specified providing source device, and if the information provision is permitted, acquire the provided information provided by the providing source device; a providing unit configured to generate a voice response based on the provided information as a response to a request from the requesting device, and provide the response to the requesting device; Equipped with. In another aspect of the invention, A voice response device that responds to input text information by voice, A response acquisition means for acquiring a plurality of different responses to the character information; a voice output means for outputting the plurality of different responses in different voice tones; The present invention is characterized by comprising:

[0007] According to such a voice response device, since it is possible to output a plurality of responses in different tones of voice, even when a single answer cannot be specified for one piece of text information, different answers can be output in different tones of voice in a manner that is easy for the user to understand, thereby making it easier for the user to use.

[0008] The voice response device of the present invention may be configured as, for example, a terminal device carried by a user, or may be configured as a server that communicates with the terminal device. Furthermore, character information may be input using an input means such as a keyboard, or may be input by converting voice into character information.

[0009] By the way, in the above voice response device, as in the second aspect of the invention, A voice input means for a user to input voice, and converting the input voice into text information; a voice transmission means for transmitting to an external device a plurality of different responses to the text information and transmitting the responses to the voice response device; Equipped with The response acquisition means acquires the response from the external device. This may be done.

[0010] According to such a voice response device, since voice can be inputted into the voice response device, text information can be inputted by voice, and since a response can be generated in an external device, the processing load on the voice response device can be reduced.

[0011] In the voice transmitting means, the operation of "converting the input voice into text information" may be performed by the voice response unit or an external device. Further, in the above voice response device, as in the third aspect of the invention, The voice response device or the external device includes a response recording means for recording a plurality of different responses, including positive and negative responses, for each of a plurality of pieces of text information; the response acquisition means acquires the positive response and the negative response as the plurality of different responses; The audio output means may reproduce the positive response and the negative response in different tones of voice.

[0012] According to such a voice response device, responses from different positions, such as positive responses and negative responses, can be reproduced in different tones of voice, so that the voice can be reproduced as if it were being spoken by a different person. This makes it possible to prevent the user who hears the voice from feeling strange.

[0013] The tone of voice may be changed depending on the type of response or the wording used when responding. For example, when responding in a gentle tone, a calm female voice may be used, and when responding in a fierce tone, a brave male voice may be used. In other words, the content of the response may be associated with a personality, and the tone of voice may be set according to the personality.

[0014] Furthermore, the voice response device can be configured, as in the fourth aspect of the invention, to be used at the reception desk of a workplace or company, or to convey something that a user finds difficult to say directly to someone else.

[0015] When using a voice response device at the reception desk, the name of the salesperson and the company name are recorded in advance on the voice response device or an external device, and the person who comes to the reception desk gives this name and company name. If so, a response may be generated to play back a voice refusing the call.

[0016] Furthermore, if the device is configured to communicate something that is difficult to say, for example, before a date, you can tell the device what you would like to say today, and the voice response device will speak for you (play back audio) at an appropriate time (for example, at a preset time or when a certain amount of time has passed since the conversation stopped).

[0017] Alternatively, the device may be configured to speak words that trigger something difficult to say, such as, "By the way, didn't you say that you were going to tell her something?" In other words, instead of outputting the response immediately, the device may be configured to output the response when a playback condition is met, such as after a certain period of time has passed.

[0018] Furthermore, in the above voice response device, as in a fifth aspect of the invention, the external device or the voice response device may acquire information for generating a response to text information from another voice response device. Also, in the above voice response device, as in a sixth aspect of the invention, when information for generating a response to text information is requested by another voice response device, the voice response device may return information in response to this request.

[0019] In this case, the voice response unit may be provided with sensors for detecting location information, temperature, humidity, illuminance, noise level, etc., and a database for dictionary information, etc., and may extract necessary information in response to a request.

[0020] Such a voice response unit (external device) can obtain information for generating a response from another voice response unit. In this case, information specific to the other voice response unit, such as the location of the other voice response unit, can be obtained.

[0021] Also, the voice response unit can transmit its own unique information to other voice response units. Furthermore, in the above voice response device, as in the seventh aspect of the invention, a response (e.g., an affirmative or negative response) output by the voice response device itself or another voice response device may be input as text information, and a response for refuting this response may be generated. In other words, from the user's point of view, it is possible to hear arguments from both pro and con positions. Then, after listening to this argument, the user can make a final decision.

[0022] This configuration can be realized by using one or more voice response units. In this case, the voice response units may directly input and output the voice to each other, or may use wireless communication or the like to exchange voice.

[0023] In the eighth aspect of the invention, A voice response device that responds to input text information by voice, a personality information acquiring means for acquiring personality information in which the personality of a user or a related person representing a person related to the user is associated with the user in accordance with a preset classification; A response acquisition means for acquiring response candidates representing a plurality of different responses to the text information; a voice output means for selecting a response to be output from among response candidates in accordance with the personality information and outputting the selected response; The present invention is characterized by comprising:

[0024] Such a voice response device can respond differently depending on the personality of the user and the person (stakeholder) related to the user, thereby improving usability for the user.

[0025] In the voice response device, as in a ninth aspect of the invention, a first personality information generating means for generating personality information of the user or the related person based on answers to a plurality of questions set in advance, The personality information acquisition means acquires the personality information generated by the personality information generation means. This may be done.

[0026] According to such a voice response device, personality information can be generated in the voice response device. When generating the personality information, a well-known personality analysis technique (Rorschach test, Szondi test, etc.) may be used. When generating the personality information, aptitude test techniques used by companies for employment examinations may be used.

[0027] Further, in the above voice response device, as in a tenth aspect of the invention, a second personality information generating means for generating personality information of the user or the related person based on a character string included in the input character information; The personality information acquisition means acquires the personality information generated by the personality information generation means. This may be done.

[0028] According to such a voice response device, personality information can be generated while the user is using the voice response device. In the voice response device, as in an eleventh aspect of the invention, a preference information generating means for generating preference information indicating a tendency of a preference of the user or the related person based on a character string included in the character information, The voice output means selects a response to be output from the response candidates based on the preference information, and outputs the selected response. This may be done.

[0029] Such a voice response unit can respond according to the preferences of the user or the person concerned. Furthermore, in the above voice response device, as in the 12th aspect of the invention, the user's actions (conversations, places traveled, and what is captured by the camera) may be learned (recorded and analyzed) to compensate for any shortcomings in the user's conversation.

[0030] For example, in a conversation in which the user responds with "Curry would be good" to the question "Is hamburger steak okay today?", the device can add, "Because we had hamburger steak yesterday," thereby conveying the reason why the user said that curry was good.

[0031] Such configuration may also be implemented during a phone call and may be configured to participate spontaneously in the user's conversation. Further, in the above voice response device, as in a thirteenth aspect of the invention, a response candidate acquisition means for acquiring response candidates from a predetermined server or the Internet; The device may include:

[0032] According to such a voice response device, it is possible to obtain candidate responses not only from the device itself or an external device, but also from any device connected via the Internet or a dedicated line. In the voice response device, as in a fourteenth aspect of the invention, A character information generating means for converting a user's action into character information; The device may include:

[0033] Here, the movement referred to in the present invention corresponds to conversation, handwriting, or gestures (for example, sign language) that are caused by muscle movements. Such a voice response unit can convert the user's actions into text information.

[0034] Further, in the above voice response device, as in a fifteenth aspect of the invention, The text information generating means converts the user's speech into text information and accumulates speech habits (pronunciation habits, etc.) as learning information (capturing and recording the characteristics). This may be done.

[0035] According to such a voice response device, since it is possible to generate text information based on learning information, it is possible to improve the accuracy of generating text information. In the voice response device, as in a sixteenth aspect of the invention, a transfer means for transferring the learning information to another voice response unit; The device may include:

[0036] According to such a voice response device, even when a user uses another voice response device, the learning information recorded in this voice response device can be used, and therefore, the accuracy of generating text information can be improved even when using another voice response device.

[0037] Furthermore, in the above voice response unit, as in a seventeenth aspect of the invention, any one of the actions and operations of the user may be detected, and learning information or personality information may be generated based on the detected actions and operations.

[0038] With such a voice response device, for example, if it detects that the user has jumped on a train for several days in a row, it can urge the user to leave the house a few minutes earlier from the next day, or if it detects from conversation that the user has a tendency to get angry easily, it can output voice or music to calm the user down.

[0039] In the voice response device, as in an eighteenth aspect of the invention, The voice response unit may further include an other device information acquisition unit for acquiring information recorded in another voice response unit from the other voice response unit.

[0040] Such a voice response unit can generate a response based on information recorded in another voice response unit. Furthermore, in the above voice response device, as in a nineteenth aspect of the invention, a reproduction condition determination means for determining whether or not a state of the voice response unit matches a reproduction condition that is set in advance as a condition for outputting a voice when the character information is not input; a message reproducing means for outputting a preset message when the reproducing condition is met; The device may include:

[0041] Such a voice response device can output voice even when no text information is input (i.e., when the user does not speak). For example, by forcing the user to speak, it can be used as a measure to suppress drowsiness while driving a car. In addition, by determining whether a person living alone responds, it is possible to confirm the safety of the person.

[0042] In the above voice response device, as in the invention of a twentieth aspect, The message reproducing means acquires news information and outputs a message relating to the news in the form of a question requesting a response from the user. This may be done.

[0043] Such a voice response device allows conversations about the news, This can prevent conversations from becoming the same over and over again. For example, when acquiring information about a company's stock price, the content of the conversation can be something like, "The stock price of company X went up by X yen today. Did you know?"

[0044] Further, in the above voice response device, as in the invention of a twenty-first aspect, The voice output means or message playback means outputs a preset message by adding separately acquired external information (such as news or environmental information (temperature, weather, location information, etc.)). This may be done.

[0045] Such a voice response unit can output a response that combines a predetermined message with the acquired information. In the above voice response device, as in the invention of a twenty-second aspect, Get multiple messages and select the message to be played depending on the message playback frequency. This may be done.

[0046] With such a voice response device, it is possible to make it difficult to play frequently played messages, thereby creating a sense of randomness when playing back messages, or to repeatedly play frequently played messages to draw attention or help solidify the memory.

[0047] Further, in the above voice response device, as in the invention of a twenty-third aspect, a no-answer transmission means for transmitting information identifying the user and a message indicating that no answer has been received to a pre-set contact point when no response or reply to the message is received; The device may include:

[0048] According to such a voice response device, if no answer is received, it is possible to report to a contact person. Therefore, for example, an abnormality in an elderly person living alone can be reported early. In the above voice response device, as in a twenty-fourth aspect of the invention, The message replay means memorizes the contents of the conversation and asks questions to obtain the same contents as those heard (memory confirmation process). This may be done.

[0049] Such a voice response device can check the memory ability of the user and also help the user to solidify the memory. Further, in the above voice response device, as in a twenty-fifth aspect of the invention, a speech accuracy detection means for detecting the accuracy of pronunciation and accent of a voice input by a user; an accuracy output means for outputting the detected accuracy; The device may include:

[0050] Such a voice response device makes it possible to check the accuracy of pronunciation and accent, which is useful, for example, when practicing a foreign language. In the above voice response device, as in a twenty-sixth aspect of the invention, The accuracy output means outputs a voice including the closest word when the accuracy is equal to or less than a certain value. This may be done.

[0051] Such a voice response device allows the user to check the accuracy of pronunciation and accent. Further, in the above voice response device, as in a twenty-seventh aspect of the invention, The message reproducing means may be configured to output the same question again when the degree of accuracy is equal to or lower than a certain value.

[0052] With such a voice response unit, accurate answers can be obtained by outputting the same questions. In the above voice response device, as in a twenty-eighth aspect of the invention, a connection control means for identifying a communication partner based on input character information and connecting the communication partner to a communication destination preset for each communication partner; The device may include:

[0053] Such a voice response device can assist with reception duties and telephone responses. In particular, in the above voice response device, as in the twenty-ninth aspect of the invention, The connection control means distinguishes between sales activities and visitors, and plays a message to decline if the visit is for sales activities. This may be done.

[0054] Such a voice response device makes it possible to exclude people who may be disruptive to the user's business without the user having to respond. Furthermore, in the voice response device, as in the invention of the thirtieth aspect, a keyword included in the input character information (particularly voice) may be extracted, and a connection may be made to a connection destination corresponding to the keyword. Note that, for example, a keyword such as the name of the other party and the connection destination may be associated in advance.

[0055] Such a voice response device can assist with tasks such as transferring calls and calling the reception desk. In the above voice response unit, as in the thirty-first aspect of the invention, the requirements of what the other party is saying may be recognized based on keywords, and an outline of what the other party said may be conveyed to the user.

[0056] Such a voice response unit can assist in the intermediate business with customers. Further, in the above voice response device, as in the invention of a thirty-second aspect, An emotion determination means for reading emotions from the tone of voice input by the user and outputting which emotion corresponds to the voice, usually including at least one of anger, joy, confusion, sadness, and elation. The device may include:

[0057] Such a voice response unit can output a response according to the user's emotions. Next, the invention of the 33rd aspect is as follows: a response generating means for generating a response according to an image captured by capturing an image of the surroundings of the voice response unit when the character information is input; a voice output means for outputting the response by voice; The present invention is characterized by comprising:

[0058] According to such a voice response device, a response can be output by voice in response to a captured image, which improves usability compared to a configuration in which a response is generated only from text information.

[0059] A specific configuration of the present invention is, for example, a configuration in which text information is input to respond as to what has been recognized, and what (who) has been recognized from the captured image is output by voice.

[0060] In the above voice response device, as in the thirty-fourth aspect of the invention, a position identification means for searching for an object included in the character information from within the captured image by image processing and identifying the position of the object; A guide means for guiding the object to a position thereof; The device may include:

[0061] Such a voice response device can guide the user to an object in a captured image. Further, in the above voice response device, as in the invention of a 35th aspect, A voice recognition system that acquires moving images of the shape of a user's mouth when inputting text information by voice A voice input video acquisition means; a character information conversion means for converting the voice into character information and correcting the character information by estimating unclear parts of the voice based on the moving image; The device may include:

[0062] According to such a voice response device, the contents of the speech can be estimated from the shape of the mouth, so that unclear parts of the speech can be estimated well. In the voice response device, as in a thirty-sixth aspect of the invention, The message reproducing means detects the user's irritation or agitation by detecting an unexpected voice and generates a message to suppress the user's irritation or agitation. This may be done.

[0063] According to such a voice response device, when the user becomes irritated or upset, it is possible to suppress these, and therefore it is possible to suppress the occurrence of trouble between the user and those around him or her. Further, in the above voice response device, as in the invention of a 37th aspect, a route information acquisition means for acquiring route information such as weather, temperature, humidity, traffic information, and road surface conditions to the destination when providing guidance to the destination; The message reproducing means outputs the route information by voice. This may be done.

[0064] Such a voice response device can notify the user of the status (route information) to the destination by voice. In the above voice response device, as in the invention of a thirty-eighth aspect, A gaze detection means for detecting a gaze of a user; a gaze movement request transmitting means for outputting a voice requesting the user to move their gaze to a predetermined position when the user does not move their gaze to a predetermined position in response to a call from the message reproducing means; The device may include:

[0065] Such a voice response device allows the user to view a specific location, thereby enabling the user to reliably check for safety when driving a vehicle. In the above voice response device, as in the invention of a thirty-ninth aspect, a change request transmission means for observing the position of a body part or a facial expression, and outputting a voice requesting a change in the position of a body part or a facial expression if there is little change in response to the call; The device may include:

[0066] According to such a voice response device, the position of a part of the user's body can be moved to a specific position, or the user can be induced to make a specific facial expression. The present invention can be used when driving a vehicle or during a physical examination.

[0067] Further, in the above voice response device, as in a fortieth aspect of the invention, A broadcast program acquisition means for acquiring a broadcast program similar to a broadcast program viewed by a user; a broadcast program complementing means for complementing the interrupted broadcast program by outputting the broadcast program acquired by the broadcast program complementing means when the broadcast program is interrupted; The device may include:

[0068] Such a voice response device can compensate for the interruption of a broadcast program being viewed by a user. In the voice response device, as in the invention of the forty-first aspect, The device is provided with a lyrics adding means for comparing a song with lyrics with the lyrics added by the user and outputting the lyrics by voice in a portion where only the lyrics of the user are not present, when the user adds lyrics to a song without lyrics and sings the song.

[0069] Such a voice response device can compensate for a portion that the user cannot sing in so-called karaoke (a portion where the lyrics are cut off). Further, in the above voice response device, as in the invention of a forty-second aspect, a pronunciation output means for, when a captured image contains characters and a user asks how to read the characters, acquiring information about the characters from an external source and outputting the pronunciation of the characters contained in the information by voice; The device may include:

[0070] Such a voice response device can teach a user how to read characters. In the voice response device, as in the invention of a 43rd aspect, The device includes an action environment detection means for detecting an action of a user and an environment surrounding the user, The message generating means generates a message according to the detected behavior and the surrounding environment. This may be done.

[0071] Such a voice response device can notify a user of dangerous places, restricted areas, etc. It can also detect abnormal behavior of the user. Further, in the above voice response device, as in the invention of a 44th aspect, a health condition determining means for determining a health condition of a user based on a captured image of the user; a health message generating means for generating a message according to a health condition; The device may include:

[0072] Such a voice response device makes it possible to manage the health condition of the user. In the voice response device, as in the invention of a 45th aspect, A notification means for notifying a predetermined contact when the health condition falls below a standard value; The device may include:

[0073] Such a voice response device can report when the health condition of the user is below a reference value, thereby making it possible to notify others of an abnormality at an earlier stage. Furthermore, in the above voice response unit, as in a forty-sixth aspect of the invention, information about the user may be output in response to an inquiry from a person other than the user.

[0074] With such a voice response device, if the user's dietary habits, walking distances, etc. are detected, the device can answer questions at hospitals, etc. on behalf of the user. The device may also learn the user's health condition, self-introduction, etc.

[0075] Each aspect of the invention does not need to be based on other inventions, and should be treated as an independent invention as far as possible. It is possible. [Brief description of the drawings]

[0076] [Figure 1] 1 is a block diagram showing a schematic configuration of a voice response system to which the present invention is applied; [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of a terminal device. [Diagram 3] 13 is a flowchart showing a voice response terminal process executed by an MPU of the terminal device. [Figure 4] 13 is a flowchart showing a voice response server process executed by a calculation unit of the server. [Diagram 5] FIG. 11 is an explanatory diagram illustrating an example of a response candidate DB. [Figure 6] 13 is a flowchart showing an automatic conversation terminal process executed by an MPU of the terminal device. [Figure 7] 13 is a flowchart showing an automatic conversation server process executed by a calculation unit of the server. [Figure 8] 13 is a flowchart showing a message terminal process executed by an MPU of the terminal device. [Figure 9] 13 is a flowchart showing a message server process executed by a calculation unit of the server. [Figure 10] 13 is a flowchart showing a guiding terminal process executed by an MPU of the terminal device. [Figure 11] 13 is a flowchart showing a guidance server process executed by a calculation unit of the server. [Figure 12] 13 is a flowchart showing a reception process executed by a calculation unit of the server. [Figure 13] 13 is a flowchart showing an information providing terminal process executed by an MPU of the terminal device. [Figure 14] FIG. 11 is an explanatory diagram showing an example of a personality DB. [Figure 15] 13 is a flowchart showing a personality information generation process executed by an MPU of the terminal device. [Figure 16] FIG. 2 is an explanatory diagram illustrating an example of a preference DB. [Figure 17] 13 is a flowchart showing a preference information generating process executed by a calculation unit of the server. [Figure 18] FIG. 11 is an explanatory diagram showing an example of a combination of personality categories and preferences. [Figure 19] 13 is a flowchart showing a motion character input process executed by a calculation unit of the server. [Figure 20] 13 is a flowchart showing another terminal use process executed by a calculation unit of the server. [Figure 21] 13 is a flowchart showing a storage confirmation process executed by a calculation unit of the server. [Figure 22] 13 is a flowchart showing a pronunciation determination process 1 executed by a calculation unit of the server. [Diagram 23] 13 is a flowchart showing pronunciation determination process 2 executed by a calculation unit of the server. [Figure 24] 13 is a flowchart showing pronunciation determination process 3 executed by a calculation unit of the server. [Diagram 25] 13 is a flowchart showing an emotion determination process executed by a calculation unit of the server. [Figure 26] 13 is a flowchart showing an emotion response generation process executed by a calculation unit of the server. [Figure 27] 13 is a flowchart showing a guidance process executed by a calculation unit of the server. [Figure 28] 13 is a flowchart showing a movement request process 1 executed by a calculation unit of the server. [Figure 29] 13 is a flowchart showing a movement request process 2 executed by a calculation unit of the server. [Diagram 30] 13 is a flowchart showing a broadcast music complementation process executed by a calculation unit of the server. [Diagram 31] 13 is a flowchart showing a character explanation process executed by a calculation unit of the server. [Diagram 32] 13 is a flowchart showing an action response terminal process executed by a calculation unit of the server. [Diagram 33] 13 is a flowchart showing an action response server process executed by a calculation unit of the server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0077] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [First embodiment] [Configuration of this embodiment] A voice response system 100 to which the present invention is applied is a system configured to generate an appropriate response in a server 90 in response to a voice input in a terminal device 1, and to output the response by voice in the terminal device 1. In detail, as shown in Fig. 1, the voice response system 100 is configured such that a plurality of terminal devices 1 and the server 90 can communicate with each other via a communication base station 80 and an Internet network 85.

[0078] The server 90 has the functions of a normal server device. In particular, the server 90 has a calculation unit 101 and various databases (DB). The calculation unit 101 is configured as a well-known calculation device having a CPU and memories such as a ROM and a RAM, and performs various processes based on a program in the memory, such as communication with the terminal device 1 via the Internet network 85, reading and writing data in various DBs, and voice recognition and response generation for conversation with a user who uses the terminal device 1.

[0079] 1, the various DBs include a voice recognition DB 102, a predictive conversion DB 103, a voice DB 104, a response candidate DB 105, a personality DB 106, a learning DB 107, a preference DB 108, a news DB 109, a weather DB 110, a playback condition DB 111, a handwritten character / sign language DB 112, a terminal information DB 113, an emotion determination DB 114, a health determination DB 115, a karaoke DB 116, a report destination DB 117, a sales DB 118, and a client DB 119. Details of these DBs will be described in each processing explanation.

[0080] Next, as shown in FIG. 2, the terminal device 1 is configured by providing the behavior sensor unit 10, a communication section 50, a notification section 60, and an operation section 70 in a predetermined housing. The behavior sensor unit 10 is equipped with a well-known MPU 31 (microprocessor unit), memory 39 such as ROM and RAM, and various sensors, and the MPU 31 performs processing such as activating a heater to optimize the temperature of the sensor element so that the sensor elements constituting the various sensors can properly detect the test subject (humidity, wind speed, etc.).

[0081] The behavior sensor unit 10 is equipped with various sensors, including a three-dimensional acceleration sensor 11 (3DG sensor), a three-axis gyro sensor 13, a temperature sensor 15 arranged on the rear surface of the housing, a humidity sensor 17 arranged on the rear surface of the housing, a temperature sensor 19 arranged on the front surface of the housing, a humidity sensor 21 arranged on the front surface of the housing, an illuminance sensor 23 arranged on the front surface of the housing, a wetness sensor 25 arranged on the rear surface of the housing, a GPS receiver 27 for detecting the current location of the terminal device 1, and a wind speed sensor 29.

[0082] The behavior sensor unit 10 also includes various sensors, such as an electrocardiogram sensor 33, a heart sound sensor 35, a microphone 37, and a camera 41. The temperature sensors 15, 19 and the humidity sensors 17, 21 measure the temperature or humidity of the air outside the housing as an inspection target.

[0083] The three-dimensional acceleration sensor 11 detects acceleration in three mutually perpendicular directions (the vertical direction (Z direction), the width direction of the housing (Y direction), and the thickness direction of the housing (X direction)) applied to the terminal device 1, and outputs the detection results.

[0084] The three-axis gyro sensor 13 detects angular acceleration (counterclockwise speed in each direction is considered positive) in the vertical direction (Z direction) and any two directions perpendicular to the vertical direction (the width direction of the housing (Y direction) and the thickness direction of the housing (X direction)) as angular velocity applied to the terminal device 1, and outputs the detection result.

[0085] The temperature sensors 15 and 19 are configured to include, for example, a thermistor element whose electrical resistance changes according to temperature. In this embodiment, the temperature sensors 15 and 19 detect Celsius temperature, and all temperature indications in the following description are in Celsius temperature.

[0086] The humidity sensors 17 and 21 are configured as, for example, well-known polymer film humidity sensors. The polymer film humidity sensors are configured as capacitors whose dielectric constant changes as the amount of moisture contained in the polymer film changes in response to changes in relative humidity.

[0087] The illuminance sensor 23 is configured as a well-known illuminance sensor including, for example, a phototransistor. The wind speed sensor 29 is, for example, a known wind speed sensor, and calculates the wind speed from the power (amount of heat dissipation) required to maintain the heater temperature at a predetermined temperature.

[0088] The heart sound sensor 35 is configured as a vibration sensor that captures vibrations caused by the pulsation of the user's heart, and the MPU 31 takes into consideration the detection results by the heart sound sensor 35 and the heart sounds input from the microphone 37 to distinguish between vibrations and noise caused by pulsation and other vibrations and noise.

[0089] The wetness sensor 25 detects water droplets on the surface of the housing, and the electrocardiogram sensor 33 detects the heartbeat of the user. The camera 41 is disposed within the housing of the terminal device 1 so as to capture the outside of the terminal device 1 as its imaging range.

[0090] The communication unit 50 includes a well-known MPU 51, a wireless telephone unit 53, and a contacts memory 55, and is configured to be able to acquire detection signals from various sensors constituting the behavior sensor unit 10 via an input / output interface (not shown). The MPU 51 of the communication unit 50 executes processing according to the detection results by the behavior sensor unit 10, input signals input via the operation unit 70, and programs stored in a ROM (not shown).

[0091] Specifically, the MPU 51 of the communication unit 50 performs the functions of a motion detection device that detects specific motions performed by the user, a positional relationship detection device that detects the positional relationship with the user, an exercise load detection device that detects the load of the exercise performed by the user, and a function of transmitting the results of processing by the MPU 51.

[0092] The wireless telephone unit 53 is configured to be able to communicate with, for example, a mobile phone base station, and the MPU 51 of the communication section 50 outputs the processing results by the MPU 51 to the notification section 60, or transmits them to a pre-set destination via the wireless telephone unit 53.

[0093] The contact memory 55 functions as a storage area for storing location information of places visited by the user. In the contact memory 55, information on contacts (such as telephone numbers) to be contacted in the event of an abnormality occurring to the user is recorded.

[0094] The notification unit 60 includes a display 61 configured as, for example, an LCD or an organic EL display, illumination 63 configured of, for example, an LED capable of emitting light in seven colors, and a speaker 65. Each unit constituting the notification unit 60 is driven and controlled by the MPU 51 of the communication unit 50.

[0095] Next, the operation unit 70 includes a touch pad 71 , a confirmation button 73 , a fingerprint sensor 75 , and a rescue request lever 77 . The touch pad 71 outputs a signal according to the position and pressure of the touch by a user (the user, the user's guardian, etc.).

[0096] The confirmation button 73 is configured so that when pressed by the user, the contacts of a built-in switch are closed, and the communication unit 50 can detect that the confirmation button 73 has been pressed.

[0097] The fingerprint sensor 75 is a well-known fingerprint sensor, and is configured to be able to read a fingerprint using, for example, an optical sensor. Note that, instead of the fingerprint sensor 75, any means capable of recognizing a person's physical characteristics (means capable of biometric authentication: means capable of identifying an individual), such as a sensor that recognizes the shape of veins in the palm of the hand, can be used.

[0098] It also includes a rescue request lever 77 that, when operated, connects to a specified contact point. [Processing of this embodiment] The process performed in such a voice response system 100 will be described below.

[0099] The voice response terminal process executed by the terminal device 1 is a process of receiving voice input by the user, sending this voice to the server 90, and reproducing the response as voice when a response to be output is received from the server 90. This process is started when the user inputs via the operation unit 70 that he or she will make a voice input.

[0100] 3, first, the microphone 37 is put into a state in which it can receive input (ON state) (S2), and image capture (recording) by the camera 41 is started (S4). Then, it is determined whether or not there is audio input (S6).

[0101] If there is no voice input (S6: NO), it is determined whether a timeout has occurred (S8). Here, a timeout refers to exceeding the allowable time for waiting for processing, and the allowable time is set to, for example, about 5 seconds here.

[0102] If the timeout has occurred (S8: YES), the process proceeds to S30, which will be described later. If the timeout has not occurred (S8: NO), the process returns to S6. If there is voice input (S6: YES), the voice is recorded in memory (S10), and it is determined whether the voice input has ended (S12). Here, if the voice is interrupted for a certain period of time or more, or if an instruction to end the voice input is input via the operation unit 70, it is determined that the voice input has ended.

[0103] If the voice input has not ended (S12: NO), the process returns to S10. If the voice input has ended (S12: YES), the ID for identifying the device, the voice, the captured image, and other data are transmitted as packets to the server 90 (S14). The data transmission process may be performed between S10 and S12.

[0104] Next, it is determined whether the data transmission is complete (S16). If the data transmission is not complete (S16: NO), the process returns to S14. If the transmission is complete (S16: YES), the process determines whether data (packets) transmitted in the voice response server process described below have been received (S18). If data has not been received (S18: NO), the process determines whether a timeout has occurred (S20).

[0105] If the timeout has occurred (S20: YES), the process proceeds to S30, which will be described later. If the timeout has not occurred (S20: NO), the process returns to S18. If data is received (S18: YES), a packet is received (S22). In this process, one or more different responses to the text information, each of which is associated with a different tone of voice, are obtained.

[0106] Then, it is determined whether or not the reception is complete (S24). If the reception is not complete (S24: NO), it is determined whether or not a timeout has occurred (S26). If a timeout has occurred (S26: YES), a message indicating that an error has occurred is output via the notification unit 60, and the voice response terminal process ends. If a timeout has not occurred (S26: NO), the process returns to S22.

[0107] If reception is complete (S24: YES), a response based on the received packet is output as voice from speaker 65 (S28). In this process, if multiple responses are to be played back, each of the multiple responses is played back in a different voice tone. When this process is completed, the voice response terminal process is terminated.

[0108] Next, the voice response server process executed by the server 90 (external device) will be described with reference to Fig. 4. The voice response server process is a process in which voice is received from the terminal device 1, voice recognition is performed to convert the voice into text information, and a response to the voice is generated and returned to the terminal device 1. Particularly, in this embodiment, a plurality of responses may be transmitted in association with voices of different tones.

[0109] 4, the voice response server process first determines whether or not a packet has been received from any of the terminal devices 1 (S42). If no packet has been received (S42: NO), the process of S42 is repeated.

[0110] If a packet has been received (S42: YES), the communication partner terminal device 1 is identified (S44). In this process, the terminal device 1 is identified by the ID of the terminal device 1 contained in the packet.

[0111] Next, the speech included in the packet is recognized (S46). In the speech recognition DB 102, many speech waveforms are associated with many characters. In addition, in the predictive conversion DB 103, words that tend to be used following a certain word are associated with each other.

[0112] In this process, the voice recognition DB 102 and the predictive conversion DB 103 are referenced to perform a well-known voice recognition process and convert the voice into text information. Next, the captured image is processed to identify objects in the captured image (S48), and the user's emotions are determined based on the waveform of the voice, the endings of words, etc. (S50).

[0113] In this process, by referring to the emotion determination DB 114 in which the waveform of the voice (tone of voice) and the endings of words are associated with emotion categories such as normal, anger, joy, confusion, sadness, and elation, it is determined which category the user's emotion falls into, and the determination result is recorded in memory. Next, by referring to the learning DB 107, words that the user frequently speaks are searched for, and parts of the character information generated by the voice recognition that were ambiguous are corrected.

[0114] In addition, the characteristics of each user, such as the words the user frequently uses and pronunciation habits, are recorded in the learning DB 107. In addition, data is added and corrected in the learning DB 107 during conversation with the user.

[0115] Next, the corrected character information is specified as the input character information (S54), and a sentence similar to the character information is input and searched from the answer candidate DB 105 to obtain a response from the answer candidate DB 105 (S56). Here, the answer candidate DB 105 contains the input character information, the first output, the tone of the first output, the second output, and the tone of the second output, as shown in FIG. It is associated with righteousness.

[0116] For example, as shown in the first row of Fig. 5, when text information "Today's weather at ※" is input, a first output "Today's weather at ※ is ※" is output in association with the voice tone of Woman 1. However, the "※" part is acquired by accessing the weather DB 110 in which area names are associated with weather forecasts for several days in that area.

[0117] Furthermore, when text information such as "Today's weather as of *" is input, the weather at the timing when today's weather changes is also obtained from the weather DB 110, and the second output "However, * is *" is output in association with the voice of Man 1. When the weather in Tokyo today is sunny and tomorrow's weather is rainy, if "Today's weather in Tokyo" is input, "Today's weather in Tokyo is sunny" is output in Woman 1's voice, and "However, it will rain tomorrow" is output in Man 1's voice.

[0118] In this embodiment, a case where multiple responses are output has been described, but if there is only one answer to the input, there will be only one response. Therefore, it is determined whether there is only one response (S58). If there is only one response (S58: YES), the process proceeds to S62, which will be described later.

[0119] If there are multiple responses (S58: NO), the response contents are associated with the tone of voice (S60). Here, the voice DB 104 stores a database of artificial voices for each tone of voice, and in this process, the tone of voice set for each response is associated with the tone of voice in the database.

[0120] Next, the response content is converted into voice (S62). In this process, the response content (text information) is output as voice based on the database stored in the voice DB 104.

[0121] Then, the generated response (voice) is transmitted as a packet to the communication partner terminal device 1 (S64). Note that the voice of the response may be transmitted as a packet while being generated. Next, the conversation content is recorded (S68). In this process, the input character information and the output response content are recorded as the conversation content in the learning DB 107. At this time, keywords (words recorded in the voice recognition DB 102) included in the conversation content, pronunciation characteristics, and the like are recorded in the learning DB 107.

[0122] When this processing is completed, the voice response server processing is terminated. [Effects of this embodiment] The voice response system 100 described above in detail is a system that responds to input text information by voice, and the terminal device 1 (MPU 31) acquires a number of different responses to the text information and outputs the multiple different responses in different voice tones.

[0123] According to such a voice response system 100, since it is possible to output a plurality of responses in different tones of voice, even when a single answer cannot be specified for one piece of text information, different answers can be output in different tones of voice in an easy-to-understand manner for the user, thereby making it easier for the user to use.

[0124] In the voice response system 100, the terminal device 1 inputs a voice from a user via the microphone 37, and the server 90 (the calculation unit 101) converts the input voice into text information, generates a plurality of different responses to the text information, and transmits them to the terminal device 1. Then, the terminal device 1 obtains the responses from the server 90.

[0125] According to such a voice response system 100, since voice can be inputted into the terminal device 1, text information can be inputted by voice. Also, since a response can be generated in the server 90, the processing load in the voice response system 100 can be reduced.

[0126] Furthermore, in the voice response system 100, the server 90 converts the user's speech into text information and accumulates speech habits (pronunciation habits, etc.) as learning information (capturing and recording the characteristics).

[0127] According to such a voice response system 100, since text information can be generated based on learning information, the accuracy of generating text information can be improved. Furthermore, in the above voice response system 100, the server 90 reads the emotion from the tone of voice input by the user, and outputs which emotion corresponds to the voice, usually including at least one of anger, joy, confusion, sadness, and elation.

[0128] According to such a voice response system 100, a response can be output according to the user's emotions. [Modification of the first embodiment] In this embodiment, voice recognition is used as a configuration for inputting character information, but it is not limited to voice recognition, and input may be made using input means (operation unit 70) such as a keyboard or a touch panel. Also, the operation of "converting input voice into character information" is performed by the server 90, but it may be performed by the terminal device 1.

[0129] Furthermore, in the above voice response system 100, the server 90 may be provided with a candidate response DB 105 in which a plurality of different responses, including positive and negative responses to each piece of text information, are recorded, and the terminal device 1 may acquire the positive and negative responses as the plurality of different responses and play back the positive and negative responses in different tones of voice.

[0130] For example, as shown in the second row of Fig. 5, when a voice is input saying "May I buy this item?", positive information about the item, such as a good reputation, is output in a female voice. On the other hand, negative information, such as a bad reputation, is output in a voice tone different from the female voice associated with the positive information (here, a male voice).

[0131] According to the voice response system 100, responses from different positions, such as positive responses and negative responses, can be reproduced in different tones of voice, so that the voice can be reproduced as if it were being spoken by a different person. This makes it possible to prevent the user who hears the voice from feeling strange.

[0132] The tone of voice may be changed depending on the type of response or the wording used when responding. For example, when responding in a gentle tone, a calm female voice may be used, and when responding in a fierce tone, a brave male voice may be used. In other words, the content of the response may be associated with a personality, and the tone of voice may be set according to the personality.

[0133] Furthermore, in the voice response system 100, a response (e.g., an affirmative or negative response) output by one's own terminal device 1 or another terminal device 1 may be input as text information, and a response for refuting this response may be generated. In other words, from the user's point of view, it is possible to hear arguments based on both pro and con positions. Then, after listening to this argument, the user can make a final decision.

[0134] This configuration can be realized using one or more terminal devices 1. For the multiple terminal devices 1 to exchange voice with each other, the voice may be input and output directly, or wireless communication may be used. When multiple terminal devices 1 communicate with the server 90, data may be transmitted to the other terminal devices 1 in the process of S66.

[0135] Furthermore, in the above voice response system 100, the calculation unit 101 may learn (record and analyze) the user's behavior (conversation, places traveled, and what is captured by the camera) and compensate for any shortcomings in the user's conversation.

[0136] For example, in a conversation in which the user responds with "Curry would be good" to the question "Is hamburger steak okay today?", the device can add, "Because we had hamburger steak yesterday," thereby conveying the reason why the user said that curry was good.

[0137] Such configuration may also be implemented during a phone call and may be configured to participate spontaneously in the user's conversation. Furthermore, in the voice response system 100, the server 90 may obtain the response candidates from a predetermined server or from the Internet.

[0138] According to such a voice response system 100, response candidates can be obtained not only from the server 90 but also from any device connected via the Internet, a dedicated line, or the like. [Second embodiment] [Processing of the second embodiment] Next, another embodiment of the voice response system will be described. In the following embodiments, only the parts different from the voice response system 100 of the first embodiment will be described in detail, and the parts similar to those of the voice response system 100 of the first embodiment will be denoted by the same reference numerals and will not be described.

[0139] In the voice response system of the second embodiment, even if the user does not input text information, voice is output. In detail, the terminal device 1 executes an automatic conversation terminal process shown in Fig. 6. The automatic conversation terminal process is a process that is started, for example, when the terminal device 1 is powered on, and is executed repeatedly thereafter.

[0140] In the automatic conversation terminal process, first, it is determined whether or not the setting for automatic conversation is ON (S82). Note that the user can set whether or not to perform automatic conversation via the operation unit 70 or by inputting voice.

[0141] If the automatic conversation is OFF (S82: NO), the automatic conversation terminal process is terminated. If the automatic conversation is ON (S82: YES), the automatic conversation mode is set and the ID for identifying the terminal is sent to the server 90 (S84).

[0142] Next, it is determined whether or not a packet has been received from the server 90 (S86). If a packet has not been received (S86: NO), the process of S86 is repeated. If a packet has been received (S86: YES), the same processes as those of S22 to S30 described above are carried out, and when these processes are completed, the automatic conversation terminal process is terminated.

[0143] Furthermore, the server 90 executes an automatic conversation server process shown in Fig. 7. The automatic conversation server process is started, for example, when the server 90 is powered on, and is thereafter repeatedly executed.

[0144] In the automatic conversation server process, first, it is determined whether or not a notification that the automatic conversation mode has been set has been received from the terminal device 1 (S92). If the notification that the automatic conversation mode has been set has not been received (S92: NO), the process proceeds to S98.

[0145] If the automatic conversation mode has been set (S92: YES), the terminal device 1 to be the communication partner is identified based on the ID included in the received packet (S94), and an automatic conversation is set for this communication partner (S96). Next, for each of the terminal devices 1 set to the automatic conversation mode, it is determined whether the playback condition is satisfied (S98).

[0146] Here, the playback conditions indicate, for example, that a certain amount of time has passed since the last conversation (voice input), that it is a certain time of day, that the weather is specific, or that any sensor value indicates an abnormality.

[0147] If the reproduction conditions are not met (S98: NO), the automatic conversation server process is terminated. If the reproduction conditions are met (S98: YES), a message according to the reproduction conditions is generated (S100).

[0148] Here, the message according to the playback conditions may be, for example, a fixed phrase such as "Good morning" or "Hello," or may be related to the latest news obtained from the news DB 109, which automatically updates the latest news. In the case of a message related to the latest news, for example, when information on the stock price of a certain company is obtained, the message may be, "The stock price of XX company went up by XX yen today. Did you know?"

[0149] When this process is completed, the processes of S42 to S54 described above are carried out. Then, when the process of S54 is completed, it is determined whether or not a predetermined answer has been obtained from the terminal device 1 that is the communication partner (S112). Here, the predetermined answer may be, for example, some kind of voice or a specific answer. For example, a specific answer would be an answer of "I know" or "I don't know" in response to the question "Do you know?", and an answer containing a word indicating the weather, such as "It's raining" or "It's sunny" in response to the question "What's the weather like now?".

[0150] If there is a predetermined response (S112: YES), the automatic conversation server process ends. If there is no predetermined response (S112: NO), the message sent in S100 is resent (S114). When resending the message in this way, the tone of voice is changed to produce a stronger, harsher tone of voice.

[0151] Next, the report destination DB 117 in which the terminal device 1 and the report destination are previously associated is referenced, and a message is sent to the predetermined report destination that there was no reply (S116). When this process is completed, the automatic conversation server process is terminated.

[0152] [Effects of the second embodiment] In the above voice response system 100, when no text information is input, the server 90 judges whether or not the situation of the voice response system 100 matches a reproduction condition that is set in advance as a condition for outputting a voice. If the reproduction condition is matched, the server 90 outputs a preset message.

[0153] According to such a voice response system 100, even if no text information is input (that is, if the user does not speak), it is possible to output voice. For example, by forcing the user to speak, it can be used as a measure to suppress drowsiness while driving a car. In addition, by determining whether or not a person living alone responds, the safety of that person can be confirmed.

[0154] In the voice response system 100, the server 90 acquires news information and outputs a message relating to the news in the form of a question requesting a response from the user. According to such a voice response system 100, it is possible to have a conversation about the news, and therefore it is possible to prevent the conversation from becoming the same over and over again.

[0155] Furthermore, in the voice response system 100, the server 90 adds separately acquired external information (such as news and environmental information (temperature, weather, location information, etc.)) to a preset message and outputs the message.

[0156] According to such a voice response system 100, it is possible to output a response that combines a predetermined message with the acquired information. Furthermore, in the voice response system 100, when no response to a response or message is received, the server 90 transmits to a preset contact point information identifying the user and a message indicating that no response was received.

[0157] According to such a voice response system 100, when no answer is received, a notification can be sent to a contact person. Therefore, for example, an abnormality in an elderly person living alone can be reported at an early stage.

[0158] [Modification of the second embodiment] Furthermore, in the voice response system 100, the server 90 may acquire a plurality of messages and select and output a message to be reproduced depending on the reproduction frequency of the message.

[0159] According to such a voice response system 100, it is possible to make it difficult to play back frequently played messages, thereby creating a sense of randomness in the message playback, or to repeatedly play frequently played messages to draw attention or help solidify the memory.

[0160] [Third embodiment] [Processing of the third embodiment] Next, in the voice response system of the third embodiment, the terminal device 1 is configured to convey things that the user finds difficult to say directly to someone. For example, if a user speaks to the device before a date about what he or she would like to say today, the voice response system 100 will speak on the user's behalf (play back audio) at an appropriate time (for example, at a preset time or when a certain amount of time has passed since the conversation stopped).

[0161] In detail, the terminal device 1 performs a message terminal process shown in Fig. 8, and the server 90 performs a message server process shown in Fig. 9. The message terminal process is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed.

[0162] In the message terminal process, first, it is determined whether or not the message mode has been set by the user (S132), as shown in Fig. 8. If the message mode has not been set (S132: NO), the process of S132 is repeated.

[0163] If the message mode is set (S132: YES), the processes of S2 to S8 are carried out, and if the determination in S6 is affirmative, a message mode flag is set to ON in the memory of the terminal device 1 (S134). Then, the processes of S10 to S16 are carried out.

[0164] If the determination in S16 is affirmative, it is determined whether or not a packet has been received from the server 90 (S136). If a packet has not been received (S136: NO), the process of S136 is repeated. If a packet has been received (S136: YES), the processes of S24 to S30 are carried out, and the message terminal process is terminated.

[0165] Next, the message server process is a process that starts when the server 90 is powered on, for example, and is executed repeatedly thereafter. In detail, first, it is determined whether or not a packet has been received from any of the terminal devices 1 (S142). If no packet has been received (S142: NO), the process proceeds to S156, which will be described later.

[0166] If a packet has been received (S142: YES), the terminal device 1 of the communication partner is identified (S44), and it is determined whether or not the packet contains a mode flag such as a message mode flag (S144). If no mode flag is present (S144: NO), the process proceeds to S148.

[0167] Furthermore, if there is a mode flag (S144: YES), the server 90 also sets the mode by setting the flag corresponding to the terminal device 1 of the communication partner to ON (S146). For example, if the message mode flag is the corresponding message mode, the processes of S46 to S152 described later are executed, and if the guidance mode flag described later is the corresponding guidance mode, the processes of S46 to S176 (see FIG. 11) are executed.

[0168] Next, it is determined whether the message flag is ON (S148). If the message flag is ON (S148: YES), the processes of S46 to S54 are carried out, and then the message replay conditions are extracted (S150).

[0169] Here, the message playback conditions can be set in advance by the user via the operation unit 70 of the terminal device 1, and include, for example, the time and the location. The message playback conditions are transmitted to the server 90 when packets are transmitted for the message terminal processing.

[0170] Next, the message and the voice (tone of voice) are associated with each other and recorded in memory (S152), and the process proceeds to S156. If the message flag is OFF (S148: NO), the process proceeds to S154 for other modes, and it is determined whether playback timing has arrived (S156). Here, the playback timing refers to the content set in the message playback conditions.

[0171] If it is not the playback timing (S156: NO), the message server process immediately ends, whereas if it is the playback timing (S156: YES), the process of S62 to S64 is carried out, and the message server process ends.

[0172] [Effects of the third embodiment] According to the voice response system of the third embodiment, the voice input by the user is not played back immediately, but is played back when the message playback condition is met after a certain period of time.

[0173] For example, as shown in the third row of Figure 5, if you input "Please tell Mr. / Ms. XX that XX," the sentence you want to convey will be played after Mr. / Ms. XX's voice is recognized (heard).

[0174] [Modification of the third embodiment] In the third embodiment, the contents of what the user said are reproduced. The terminal device 1 may be configured to speak words that will trigger the conversation, such as, for example, "By the way, didn't you say that you were going to tell her something?" In detail, the terminal device 1 performs the guiding terminal process shown in Fig. 10, and the server 90 performs the guiding server process shown in Fig. 11.

[0175] The guiding terminal process is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed. For example, the guiding terminal process is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed.

[0176] In the guidance terminal process, as shown in Fig. 10, first, it is determined whether or not the guidance mode has been set by the user (S162). If the guidance mode has not been set (S162: NO), the process of S162 is repeated.

[0177] If the guidance mode is set (S162: YES), the processes of S2 to S8 are carried out, and if a positive determination is made in S6, a guidance mode flag is set to an ON state in the memory of the terminal device 1 (S164). Then, the processes of S10 to S16 are carried out.

[0178] If the determination in S16 is affirmative, it is determined whether or not a packet has been received from server 90 (S166). If a packet has not been received (S166: NO), the process of S166 is repeated. If a packet has been received (S166: YES), the processes of S24 to S30 are performed, and the guiding terminal process is terminated.

[0179] Next, the guiding server process is started, for example, when the server 90 is powered on, and is thereafter repeatedly executed. In detail, the above-mentioned processes of S142 to S146 are executed. Then, it is determined whether the guiding flag is ON (S172).

[0180] If the guidance flag is ON (S172: YES), the processes of S46 to S54 are carried out, and then the guidance regeneration conditions are extracted (S174). Here, like the message playback conditions, the user can set the guided playback conditions in advance via the operation unit 70 of the terminal device 1, and the guided playback conditions include, for example, the time and the location. The guided playback conditions are transmitted to the server 90 when a packet is transmitted for the message terminal processing.

[0181] Next, the system generates guidance contents, associates the guidance contents with the voice (tone of voice), and records the association in memory (S176). For example, the system searches for words expressing desires such as "want to do" and "hope" contained in the input text information, extracts keywords before these words, and outputs words registered as words inducing these keywords as guidance contents. The keywords and words indicating the guidance contents are associated with each other in advance and recorded in the response candidate DB 105.

[0182] Next, the above-mentioned processes from S156 onward are performed, and the server process is terminated. If the guidance flag is OFF (S172: NO), processes related to other modes are performed (S154), and the above-mentioned processes from S156 onward are performed, and the server process is terminated.

[0183] According to the configuration of the modified example of the third embodiment, the user can be guided to speak the words he or she wants to say, rather than directly outputting the words he or she wants to say. [Fourth embodiment] [Processing of the fourth embodiment] Next, an example of using the terminal device 1 for reception work will be described. In this embodiment, the terminal device 1 is installed at the reception desk of a company. It can also be used for telephone reception for a company's main telephone number or telephone banking. Here, in this embodiment, the process of S56 in the first embodiment is realized by replacing it with the reception process shown in FIG.

[0184] In the reception process, as shown in Fig. 12, first, it is determined whether or not the character information includes a company name (S192). In this process, it is determined whether or not a general name or a company name (recorded in the voice recognition DB 102) is included.

[0185] If the text information does not include a company name or a personal name (S192: YES), a response to ask for the company name and the personal name is generated (S194), and the reception process ends. In this process, a response such as "Please tell us your name and business" is generated.

[0186] If the character information includes a company name or an individual name (S192: NO), the company name or individual name is extracted from the sales DB 118 and the client DB 119 (S196). Here, the sales DB 118 records the names of companies and their personnel who have made sales in the past, or customers who only complain. The client DB 119 records company names, personnel of the companies, personnel on the user's side of the terminal device 1 (their own company's side), schedules such as scheduled meeting times, and contact information for each personnel in association with each other.

[0187] Next, it is determined whether or not the company name or individual name can be extracted from the sales DB 118, that is, whether or not the company name or individual name included in the character information is included in the sales DB 118 (S198). If the company name or individual name can be extracted from the sales DB 118 (S198: YES), a sales refusal response (a response refusing to be referred) is generated to decline the sale (S200), and the reception process ends.

[0188] If the company name or individual name cannot be extracted from the sales DB 118 (S198: NO), it is determined whether the person who has come to the reception desk is a person who will visit at a nearby time (for example, within one hour before or after the current time) according to the schedule in the client DB 119 (S202). If the person is a person who will visit at a nearby time (S202: YES), the contact information of the person in charge of this person is extracted from the client DB 119, and the person is connected to this person so that the person in charge can talk with the person at the reception desk (S204). In this process, a connection can be made to the person in charge's extension, mobile phone, etc.

[0189] Next, a reception response for the client is generated (S206). For example, the reception response for the client may be, "Dear Mr. XX, thank you for your continued support. We are connecting you to the person in charge, so please wait a moment." When this process is completed, the reception process is terminated.

[0190] If the visitor is not coming soon (S202: NO), the system connects the caller to a preset contact for reception and connects the caller to a receptionist (S208) so that the caller can talk with the receptionist. Then, a normal reception response is generated (S210).

[0191] Here, as a normal reception response, for example, a response such as "We are connecting to the reception, so please wait for a while" is generated. When this process is completed, the reception process ends.

[0192] [Effects of the fourth embodiment] The voice response system 100 is configured to be used at the reception desk of a workplace or company. In this configuration, the name of a salesperson and the name of the company are recorded in advance in the sales DB 118 of the server 90, and when a person at the reception gives this name and company name, a response is generated to play back a voice of refusal.

[0193] In the voice response system 100, the server 90 responds to input text information. The communication partner is identified by the above and a communication destination preset for each communication partner is connected to the communication partner. Such a voice response system 100 can assist with reception work and telephone correspondence. Also, such a voice response system 100 can remove a person who may be disruptive to the user's work without the user having to respond to the person.

[0194] Furthermore, in the voice response system 100, the server 90 extracts a keyword contained in the input character information (particularly voice) and connects to a connection destination corresponding to the keyword. Note that a keyword such as a name of a destination and the connection destination are associated in advance.

[0195] Such a voice response system 100 can assist with tasks such as transferring calls and calling the reception desk. [Modification of the fourth embodiment] In the above embodiment, the connection destination is set according to the other party, but this technology may also be applied to recognize requirements (keywords contained in the text information) when accepting calls for telephone banking, telephone shopping, etc., and change the connection destination according to the requirements.

[0196] In addition, in the voice response system 100, the server 90 may recognize the requirements of the other party based on keywords and inform the user of the outline of what the other party said. Such a voice response system 100 can assist in the intermediate business with customers.

[0197] [Fifth embodiment] [Processing of the fifth embodiment] Next, the terminal device 1 may receive a request from another terminal device 1 and provide the information requested by the other terminal device 1.

[0198] When configured in this manner, the server 90 requests the necessary information from the other terminal device 1 in the process of S56, and generates a response after acquiring the necessary information from the other terminal device 1. Then, the terminal device 1 that provides the necessary information performs an information providing terminal process shown in Fig. 13. The information providing terminal process is a process that is started, for example, when a request is received from the server 90.

[0199] 13, the information providing terminal process first extracts an information providing destination (S222). This information providing destination indicates another terminal device 1 requesting information, and an ID for identifying this other terminal device 1 is included in the request from the server 90.

[0200] Next, it is determined whether the person is a person to whom the provision of information is permitted (S224). Here, the IDs of the people to whom the provision of information is permitted, such as family members and friends, are recorded in advance in the terminal information DB 113. In this process, the determination is made by referring to the terminal information DB 113.

[0201] If the information provision is permitted to the party (S224: YES), the requested information is obtained from its own memory 39, various sensors, etc. (S226), and this data is transmitted to the server 90 (S228). If the information provision is not permitted to the party (S224: NO), a message to the effect that the information provision is rejected is transmitted to the server 90 (S230).

[0202] When this processing is completed, the information provision terminal processing is terminated. In this configuration, for example, as shown in the fourth row of FIG. 5, in response to the question "What is Mr. XX doing?", the server 90 requests location information from Mr. XX's terminal device 1, and the terminal device 1 returns the location information.

[0203] The server 90 then recognizes Mr. XX's actions based on the location information. For example, if he is moving on the tracks at a speed faster than a human can run, it will determine that he is traveling on a train and generate a response such as "Mr. XX is on a train. It appears that he is on his way home."

[0204] [Effects of the fifth embodiment] In the voice response system 100, the server 90 acquires information recorded in another terminal device 1 different from the requesting terminal device 1, and provides the information to the other terminal device 1. That is, in the voice response system 100, the server 90 acquires information for generating a response to text information from the other terminal device 1.

[0205] According to such a voice response system 100, a response can be generated based on information recorded in another terminal device 1. Furthermore, in the above voice response system 100, when the terminal device 1 is requested by another terminal device 1 for information for generating a response to text information, the terminal device 1 returns information in response to this request.

[0206] In this configuration, the terminal device 1 includes sensors for detecting location information, temperature, humidity, illuminance, noise level, and the like, as well as a database of dictionary information, and the like, and extracts necessary information in response to a request.

[0207] According to such a voice response system 100, it is possible to obtain information specific to the other terminal device 1, such as the location of the other terminal device 1. In addition, it is possible to transmit information specific to the terminal device 1 to the other terminal device 1.

[0208] [Sixth embodiment] [Processing of the sixth embodiment] Next, in the voice response system of the sixth embodiment, a personality DB 106 is prepared, which stores personality information in which the personality of the user or related persons representing those related to the user is associated with a preset category. For example, as shown in Fig. 14, the personality DB 106 stores the names of the user and related persons in association with their personality categories.

[0209] In addition, in the personality DB 106 shown in Fig. 14, personality tests are administered to users and related parties, and the test results are also recorded. Here, when generating personality information, well-known personality analysis techniques (Rorschach test, Szondi test, etc.) may be used. When generating personality information, aptitude test techniques used by companies and the like for employment tests may also be used.

[0210] When generating personality information, for example, a personality information generation process shown in Fig. 15 is performed. The personality information generation process is started when, for example, an instruction to generate personality information is input using the operation unit 70 or the like in the terminal device 1.

[0211] In the personality information generation process, as shown in Fig. 15, first, the microphone 37 is turned on (S242), and one of the predetermined multiple-choice questions is output by voice (S244). At this time, the multiple-choice question may be obtained from the server 90, or a question previously recorded in the memory 39 may be posed.

[0212] Next, it is determined whether or not there is a voice response from the subject (the user or a related person) (S246). If there is no response (S246: NO), the process of S246 is repeated. If there is an answer (S246: YES), conversation parameters such as the ending of words and the conversation speed are extracted (S248), and it is determined whether the current question is the final question (S250). If it is not the final question (S250: NO), the next question is selected (S252), and the process returns to S242.

[0213] If it is the final question (S250: YES), a personality analysis is performed by answering a multiple-choice question (S254), and a personality analysis is performed using conversation parameters (S256). Here, the personality analysis using conversation parameters can capture the tendency that confident people tend to speak with stronger words and unconfident people tend to speak with weaker words, that impatient people tend to speak quickly and easygoing people tend to speak slowly, etc.

[0214] Next, these personality analysis results are analyzed comprehensively, for example by taking a weighted average (S258), and are then assigned to personality categories (S260). In detail, the personality of the subject obtained by the test is converted into a score, and each score is assigned to a personality category.

[0215] Next, the subject and the personality category are associated (S262) and recorded in the personality DB 106 (S264). That is, the relationship between the subject and the personality category is transmitted to the server 90. At this time, the test results are also transmitted to the server 90, and the server 90 constructs the personality DB 106 as shown in Fig. 14. When this process is completed, the personality information generation process is terminated.

[0216] When using the personality DB 106 thus generated, the personality categories are associated with different responses in the response candidate DB 105. Then, in the process of S56, the server 90 obtains response candidates representing a plurality of different responses to the text information, selects a response to be output from the response candidates according to the personality information, and in the processes of S60 and S64, outputs the selected response.

[0217] [Effects of the sixth embodiment] In the voice response system 100, the terminal device 1 generates personality information of a user or related person based on answers to a plurality of questions set in advance, and acquires the generated personality information.

[0218] According to such a voice response system 100, personality information can be generated in the server 90 or the terminal device 1. Furthermore, in the voice response system 100, the calculation unit 101 generates personality information of the user or related person based on the character string included in the input text information.

[0219] According to the voice response system 100, the user can generate personality information while using the voice response system 100. Moreover, such a voice response system 100 can provide different responses depending on the personality of the user and the person (stakeholder) related to the user, thereby improving usability for the user.

[0220] [Modification of the sixth embodiment] In the sixth embodiment, the response may be narrowed down to one according to the personality and then output, or a plurality of responses may be output in correspondence with voices of different tones.

[0221] Furthermore, the processes of S248 and S254 to S264 in the personality information generation process may be performed by the server 90. In this case, as in the first embodiment, the server 90 may identify the terminal device 1, and voices and questions may be exchanged between the terminal device 1 and the server 90.

[0222] Furthermore, in the voice response system 100, the server 90 may detect any of the actions and operations of the user and generate learning information or personality information based on the actions and operations.

[0223] According to such a voice response system 100, for example, if it is detected that the user has jumped on a train for several consecutive days, it can urge the user to leave the house a few minutes earlier from the next day, or if it is detected from a conversation that the user has a tendency to get angry easily, it can output voice or music to calm the user down.

[0224] [Seventh embodiment] [Processing of the Seventh Embodiment] Next, in the voice response system of the seventh embodiment, a preference DB 108 is prepared in which preference information in which the preferences of the user and related persons are associated with each other according to a preset classification is recorded. For example, as shown in Fig. 16, the preference DB 108 records the names of the user and related persons and their preferences in association with each of the preference types such as food preferences (food), color preferences (color), hobbies, etc.

[0225] In particular, food preferences are classified into sweet tooth (sweet), spicy tooth (spicy), and average, which is somewhere in between; color preferences are classified into warm colors (warm), cool colors (cold), and average, which is somewhere in between; and hobbies are classified into indoor hobbies (indoor), outdoor hobbies (outdoor), and both indoor and outdoor hobbies (indoor and outdoor).

[0226] When constructing such a preference DB 108, for example, a preference information generating process shown in Fig. 17 is executed. The preference information generating process is carried out, for example, between S48 and S54. 17, keywords related to preferences are extracted from the character information (S282), and objects identified by image processing are extracted that relate to preferences (S284). Note that keywords related to preferences are associated with types of preferences and classifications within those types (sweet, normal, spicy, etc. for food preferences) in preference DB 108, and in these processes, if the extracted keywords or objects are included in preference DB 108, they are extracted as being related to preferences.

[0227] Next, a counter is incremented for each group of keywords related to preferences (S288). For example, when a preference type of "food preference" and a type of "spicy" such as kimchi are extracted, the counters corresponding to "food preference" and "spicy" are incremented.

[0228] Then, the preference information (preference DB 108) is updated based on the counter value (S290). That is, for each "preference type", the "type" with the largest counter value is the one that best matches the preference, and is recorded in the preference DB 108 as a characteristic of the user's or related person's preferences. When this process is completed, the preference information generation process is terminated.

[0229] When the preference DB 108 thus generated is used, different responses corresponding to each preference are prepared in the response candidate DB 105, and the server 90 acquires response candidates representing a plurality of different responses to the text information in the processing of S56, selects a response to be output from the response candidates according to the preference information, and outputs the selected response in the processing of S60 and S64.

[0230] [Effects of the seventh embodiment] In the voice response system 100, the server 90 generates preference information indicating the tendency of preferences of the user or related person based on the character string included in the text information, selects a response to be output from the response candidates based on the preference information, and outputs the selected response.

[0231] According to the voice response system 100, responses can be made according to the preferences of the user or the person concerned. For example, when the user is buying a present for a related person, if the user asks the terminal device 1, "What would Mr. XX want?", a response according to the preference information can be obtained.

[0232] [Modification of the Seventh Embodiment] The reply candidate DB 105 may have a table in which personality categories are associated with preference information, as shown in FIG.

[0233] For example, in the example shown in FIG. 18, personality categories are associated with color preferences, and products that are estimated to please women as presents are arranged in a matrix. In the process of S56, a response can be generated taking into account both personality and preferences in this way.

[0234] [Eighth embodiment] [Processing of the Eighth Embodiment] In the above embodiment, the voice is converted into text information, but the action of the user may be converted into text information.

[0235] In detail, the terminal device 1 captures the user's motion as a captured image and transmits it to the server 90, and the server 90 executes, for example, the motion character input process shown in Fig. 19. The motion character input process is a process that is started when a body part of the user is captured in the captured image in the process of S48.

[0236] In the motion character input process, as shown in Fig. 19, first, a captured image is acquired (S302). Then, it is determined whether the user is trying to input characters by handwriting or by sign language (S304, S308).

[0237] In these processes, for example, if the captured image shows the user's upper body along with their face, it is determined that the user is attempting to input characters using sign language, and if the captured image shows the user's hands but not their face, it is determined that the user is attempting to input characters by hand.

[0238] If a character is to be input by handwriting (S304: YES), the behavior of the fingertip or pen tip is recorded (S306), and the behavior is converted into character information based on the recorded behavior (S312). Here, in the handwritten character / sign language DB 112, the behavior when writing a character is associated with the character, and the hand movement is associated with the character expressed by the sign language. In the process of S312, the character information is generated by referring to the handwritten character / sign language DB 112.

[0239] If the user is attempting to input characters using sign language (S304: NO, S308: YES), the system refers to the handwritten character / sign language DB 112 to recognize the sign language content and executes the process of S312 described above. If the user is not attempting to input characters using handwriting or sign language (S308: NO), the system performs input processing using another method (S314).

[0240] Next, the character input by the action and the character input by the voice are associated with each other, and it is determined whether there is a similar voice (whether the degree of agreement between the reference waveform based on the character and the pronunciation waveform is equal to or greater than a reference value) (S316). If there is such a voice input (S316: YES), the accent and pronunciation features when the user inputs this character are associated with the character and recorded in the learning DB 107 (S318), and the action character input process is terminated.

[0241] If there is no such voice input (S316: NO), the action character input process is terminated. do. [Effects of the eighth embodiment] In the voice response system 100, since the actions of the user are converted into text information, the user can input text information without speaking.

[0242] [Modification of the eighth embodiment] The movements in this embodiment may be not only handwritten characters or gestures (for example, sign language) but also movements resulting from muscle movements.

[0243] [Ninth embodiment] [Processing of the ninth embodiment] When the user uses a terminal device 1 other than the terminal device 1 that he or she normally uses, the contents of the learning DB 107 may be made available to the other terminal device 1. In this case, the other terminal device 1 transmits the ID and password of the terminal device 1 that he or she normally uses to the server 90 together with a request for use.

[0244] Then, the server 90 executes another terminal use process shown in Fig. 20. The other terminal use process is started when a use request is received. In the other terminal use process, first, it is determined whether or not an ID and password have been input (S332), as shown in Fig. 20. If an ID and password have not been input (S332: NO), the process of S332 is repeated.

[0245] If an ID and password have been input (S332: YES), it is determined whether authentication using the ID and password has been completed (S334). If authentication has been completed (S334: YES), a message indicating that authentication has been completed is transmitted to the other terminal device 1 (S336), and the other terminal device 1 is set to use the learning DB 107 of the terminal device 1 whose ID and password correspond (S338).

[0246] If the authentication is not completed (S334: NO), a message indicating an error is transmitted to the other terminal device 1 (S340), and the other terminal use process is terminated. [Effects of the ninth embodiment] In the voice response system 100, the server 90 transfers learning information of a certain terminal device 1 to other terminal devices 1.

[0247] According to such a voice response system 100, even when a user of a certain terminal device 1 uses another terminal device 1, the user can use the learning information recorded in the certain terminal device 1 (the learning information recorded in the server 90). Therefore, the accuracy of generating character information can be improved even when the other terminal device 1 is used. This is particularly effective when a user has multiple terminal devices 1.

[0248] Furthermore, in the voice response system 100, the server 90 outputs information about the user in response to an inquiry from a person other than the user. According to such a voice response system 100, if the user's meal contents, walking distance, etc. are detected, questions at a hospital, etc. can be answered on behalf of the user. In addition, the system may be configured to learn the user's health condition, self-introduction, etc.

[0249] [Modification of the ninth embodiment] As in the configuration of the ninth embodiment, when a request to end use and an ID and password are received, use of the learning DB 107 for the terminal device 1 corresponding to the ID and password may be ended (prohibited).

[0250] [Tenth embodiment] [Processing of the 10th embodiment] In the voice response system of the tenth embodiment, the server 90 stores the conversation contents and asks questions to obtain the same contents about the contents heard. In detail, in S100 of the automatic conversation server process shown in FIG. 7, the memory confirmation process shown in FIG. 21 is executed.

[0251] In the memory confirmation process, as shown in Fig. 21, past conversation contents are extracted from the learning DB 107 (S352), and a question is generated with a keyword included in any of the conversation contents as an answer (S353). When this process is completed, the memory confirmation process is terminated.

[0252] A memory check process might involve questions such as, "What was on the menu for dinner last night?" or "Where did you go three days ago?" [Effects of the Tenth Embodiment] According to the voice response system 100, the memory of the user can be confirmed and the memory can be consolidated. It is also considered to be effective in preventing the progression of dementia in the elderly.

[0253] [Eleventh embodiment] [Processing of the 11th embodiment] Next, the voice response system of the eleventh embodiment is configured so that a user can practice a foreign language by utilizing the terminal device 1 and the server 90.

[0254] In detail, pronunciation determination process 1 shown in Fig. 22, pronunciation determination process 2 shown in Fig. 23, and pronunciation determination process 3 shown in Fig. 24 are executed in this order. However, the server 90 executes one of the pronunciation determination processes 1 to 3 every time the voice response server process (Fig. 2) is executed. Also, each of the pronunciation determination processes 1 to 3 is executed as the process of S56 described above.

[0255] First, in the pronunciation determination process 1, as shown in Fig. 22, a response is generated to instruct the user to input a predetermined sentence by voice (S362). In this process, for example, a model sentence in a foreign language is generated, and the user is prompted to speak the model sentence by imitating it. When this process ends, the pronunciation determination process 1 ends.

[0256] Next, when a voice is input in accordance with the pronunciation judgment process 1, the pronunciation judgment process 2 is carried out. In the pronunciation judgment process 2, the accuracy of the pronunciation and accent is scored (S372) as shown in Fig. 23. In this process, the voice is regarded as a waveform, and the degree of match of the waveform with a model sentence is scored.

[0257] Then, this score is recorded in memory (S374), and pronunciation determination process 2 is terminated. Next, pronunciation determination process 3 is performed. In pronunciation determination process 3, as shown in Fig. 24, first, it is determined whether or not the score is less than a threshold value (S382).

[0258] If the score is less than the threshold (S382: YES), a response is generated to instruct the user to input a similar sentence again (S384). In this process, for example, a response is generated to prompt the user to speak by imitating the example again.

[0259] If the score is equal to or greater than the threshold (S382: NO), a response is generated to inform the user that the pronunciation was good and to prompt the user to input the next sentence (S386). For example, a response such as "Good pronunciation. Let's move on to the next sentence" is generated.

[0260] When this process is completed, the pronunciation determination process 3 is terminated. [Effects of the eleventh embodiment] In the voice response system 100, the server 90 detects the accuracy of the pronunciation and accent of the voice input by the user, and outputs the detected accuracy.

[0261] According to the voice response system 100, the accuracy of pronunciation and accent can be confirmed. This is effective, for example, when practicing a foreign language. Furthermore, in the voice response system 100, the server 90 outputs the same question again if the accuracy is equal to or lower than a certain value.

[0262] According to such a voice response system 100, it is possible to obtain an accurate answer by outputting the same question. [Modification of the eleventh embodiment] In the voice response system 100, when the accuracy is equal to or lower than a certain value, the server 90 may output, for confirmation, a voice including a word that is closest to the pronunciation made by the user.

[0263] Such a voice response system 100 allows the user to check the accuracy of pronunciation and accent. [Twelfth embodiment] [Processing of the 12th embodiment] Next, a description will be given of a voice response system according to a twelfth embodiment. In the voice response system according to the twelfth embodiment, the emotion of the user is detected from the voice input by the user, and a response that soothes the user is generated according to the emotion.

[0264] In detail, the emotion determination process shown in Fig. 25 and the emotion response generation process shown in Fig. 26 are executed. The emotion determination process is carried out as detailed processing of S50 described above, and as shown in Fig. 25, first, emotions are scored based on tone of voice, stress at the end of sentences, length of a sentence, speed of speech, unexpected words, etc. (S392), and then the emotions are classified according to the score and recorded in memory (S394).

[0265] When this process is completed, the emotion determination process is terminated. Next, in the process of S56 described above, the emotion response generation process is executed. In detail, as shown in Fig. 26, first, the emotion category set in the emotion determination process is determined (S412). If the emotion category is normal (S412: normal), a normal greeting such as "Hello" is generated as a response (message) (S414).

[0266] If the emotion category is anger (S412: anger), a sentence for calming the other person's emotions, such as "Did that bother you?", is generated as a response (S416). If the emotion category is joy (S412: joy), a greeting with a brighter nuance than normal greetings, such as "Today is a fun day," is generated as a response (S418).

[0267] If the emotion category is "confusion" (S412: "confusion"), a greeting that shows concern for the other person, such as "What's wrong?", is generated as a response (S420). When this process is completed, the emotion response generation process ends.

[0268] [Effects of the twelfth embodiment] In the voice response system 100, the server 90 detects an unexpected voice uttered by the user to detect irritation or agitation, and generates a message for suppressing the irritation or agitation.

[0269] According to the voice response system 100, when the user is irritated or upset, it is possible to suppress these, and therefore it is possible to suppress the user from vocalizing trouble between the user and the surroundings.

[0270] [Thirteenth embodiment] [Processing of the 13th embodiment] Next, a voice response system according to a thirteenth embodiment will be described. In the voice response system according to the thirteenth embodiment, a process is performed to guide a user to an object in a captured image. This process is performed in the server 90 as details of the process of S56 described above.

[0271] When a user inputs a command such as "Please guide me to the tower I can see" into the terminal device 1 by voice, the guidance process is executed in the process of S56. In the guidance process, as shown in Fig. 27, first, the terminal location information is acquired from the GPS receiver 27 of the terminal device 1 or the like (S432).

[0272] Then, based on the voice (text information) and image processing, a target object is identified from among the objects in the captured image, and its position is identified (S434). In this process, the position of the object is identified in map information (which may be acquired from an external source or may be held by the server 90) based on the shape, relative position, etc. of the object. For example, if a tower is captured in the captured image, the tower is identified on the map based on the position of the terminal device 1 and the shape of the tower.

[0273] Next, a route to this object is searched for (S436), and route information is acquired (S438). This process can be realized using a process similar to that in a well-known cloud-based navigation device.

[0274] Then, a response for providing route guidance is generated (S440). In this process, the response generated may be similar to the response provided by the navigation device. When this process is completed, the guidance process is terminated. When the guidance process is performed while the user is moving, the message can be reproduced using the automatic conversation server process, with the reproduction condition being that the user reaches the point to which the guidance should be given.

[0275] [Effects of the thirteenth embodiment] In the voice response system 100, when text information is input, the server 90 generates a response according to a captured image of the surroundings of the voice response system 100, and outputs the response by voice.

[0276] According to the voice response system 100, a response can be output by voice in response to a captured image, which improves usability compared to a configuration in which a response is generated only from text information.

[0277] Furthermore, in the voice response system 100, the server 90 searches for an object contained in the text information from within the captured image by image processing, identifies the position of the searched object, and provides guidance to the position of the object.

[0278] Such a voice response system 100 can guide the user to an object in a captured image. Furthermore, in the voice response system 100, when providing guidance to a destination, the server 90 acquires route information to the destination, such as weather, temperature, humidity, traffic information, road surface conditions, etc., and outputs the route information by voice.

[0279] According to such a voice response system 100, the status (route information) to the destination can be notified to the user by voice. [Modification of the thirteenth embodiment] In addition to the above configuration, text information may be input to respond as to what has been recognized, and what (who) has been recognized from the captured image may be output by voice.

[0280] Furthermore, in the above voice response system 100, the server 90 may acquire a moving image capturing an image of the shape of the user's mouth when inputting text information by voice, instead of the process of S48. In this case, instead of the process of S52, the voice may be converted into text information, and unclear parts of the voice may be estimated based on the moving image to correct the text information.

[0281] According to such a voice response system 100, the contents of the speech can be estimated from the shape of the mouth, so that unclear parts of the speech can be estimated well. [Fourteenth embodiment] [Processing of the 14th embodiment] Next, a voice response system according to a fourteenth embodiment will be described. In the voice response system according to the fourteenth embodiment, a user is requested to perform a predetermined action, and it is determined whether the user performs the action as requested. In this configuration, in the automatic conversation terminal process shown in FIG. 6 and the automatic conversation server process shown in FIG. 7, a movement request process 1 shown in FIG. 28 and a movement request process 2 shown in FIG. 29 are sequentially performed as details of the process of S56 described above.

[0282] First, when the process of S54 ends, movement request process 1 starts, and in movement request process 1, a response (message) instructing to move the line of sight or head to a predetermined position is output (S452) as shown in Fig. 28. When this process ends, movement request process 1 ends.

[0283] Next, when the process of S54 is completed, movement request process 2 is started, and in movement request process 2, as shown in Fig. 29, it is determined whether the line of sight and head position have moved as instructed (S462). In this process, the user's movements are detected by processing the image captured by the camera and using the detection results by various sensors of the terminal device 1. When detecting the line of sight by image processing, a well-known line of sight recognition technology may be adopted.

[0284] If the gaze or head has not moved as instructed (S462: NO), the response generated in S452 is output again (S464).If the gaze or head has moved as instructed (S462: YES), another arbitrary response is generated (S466).

[0285] When this processing is completed, the movement request processing 2 is completed. [Effects of the fourteenth embodiment] In the voice response system 100, the line of sight of the user is detected, and if the user does not move his / her line of sight to a predetermined position in response to a call, a voice is output requesting the user to move his / her line of sight to the predetermined position.

[0286] According to the voice response system 100, the user can be made to look at a specific location, and therefore can reliably check for safety when driving a vehicle. In the above voice response system 100, the server 90 observes the positions of body parts and facial expressions, and if there is little change in response to the call, outputs a voice requesting the user to change the position of the body part or the facial expression.

[0287] According to such a voice response system 100, it is possible to move the position of a part of the user's body to a specific position or induce the user to make a specific facial expression. The present invention can be used when driving a vehicle or during a physical examination.

[0288] [Fifteenth embodiment] [Processing of the 15th embodiment] Next, a voice response system according to a fifteenth embodiment will be described. In the voice response system according to the fifteenth embodiment, when a user inputs a broadcast program or a piece of music as voice, if the broadcast program or the piece of music is interrupted, a process of complementing the input is performed.

[0289] In this configuration, as details of S56 described above, the broadcast music supplement process shown in Fig. 30 is executed. As shown in Fig. 30, the broadcast music supplement process first determines whether the broadcast program or music (the song if sung by the user) has been interrupted (S482).

[0290] If the broadcast is interrupted (S482: YES), the synchronized broadcast program or song is set as the response content in the process of S492 described later (S484), and the broadcast song complementation process ends. If the broadcast is not interrupted (S482: NO), if the broadcast program is being watched, the broadcast program is acquired (S486), and if a song is being played, the corresponding song is acquired (S488).

[0291] Here, in the karaoke DB 116, music and lyrics are recorded in association with each other, and when a music piece is acquired in this process, a music piece with lyrics is acquired. Next, the broadcast program or music piece that the user is viewing is identified (S490). Then, this broadcast program or music piece is acquired and prepared for playback in synchronization with the broadcast program or music piece that the user is viewing (S492), and the broadcast music complementation process is terminated.

[0292] [Effects of the fifteenth embodiment] In the voice response system 100, the server 90 acquires a broadcast program similar to the broadcast program that the user is watching, and when the broadcast program is interrupted, the server 90 outputs the broadcast program that it acquired to complement the interrupted broadcast program.

[0293] According to such a voice response system 100, it is possible to compensate for the interruption of a broadcast program being viewed by a user. In addition, in the voice response system 100, when a user adds lyrics to a song without lyrics and sings it, the server 90 compares the song with lyrics with the lyrics added by the user and outputs the lyrics by voice in the part where only the lyrics of the user are not included.

[0294] According to such a voice response system 100, it is possible to compensate for a portion that a user of a so-called karaoke device cannot sing (a portion where the lyrics are cut off). [Sixteenth embodiment] [Processing of the 16th embodiment] Next, a voice response system according to a sixteenth embodiment will be described. In the voice response system according to the sixteenth embodiment, when a captured image contains characters and a user asks a terminal device 1 how to read the characters, the system acquires information about the characters from an external source and outputs the reading of the characters contained in the information by voice.

[0295] In this configuration, as details of S56 described above, the character explanation process shown in Fig. 31 is performed. In the character explanation process, as shown in Fig. 31, first, it is determined whether or not a question about the reading, such as "how to read", has been received (S502). If a question about the reading has been received (S502: YES), the reading of the image-recognized character is searched for from other servers connected via the Internet network 85 (S504), the obtained reading is set in the response (S506), and the character explanation process ends.

[0296] If it is not a question about reading (S502: NO), then it is a question about "words" such as those in a Japanese dictionary. It is determined whether or not a question about the meaning of "is received (S508)." If a question about the meaning is received, the meaning of the image-recognized characters (words) is searched for from other servers etc. connected via the Internet network 85 (S510), the obtained meaning is set as a response (S512), and the character explanation process is terminated.

[0297] [Effects of the 16th embodiment] According to such a voice response system 100, the reading of characters recognized by image recognition is searched for from other servers, etc., and the obtained reading is set in the response, so that the user can be informed of how to read characters, the meaning of words, etc.

[0298] [Seventeenth embodiment] [Processing of the 17th embodiment] Next, a voice response system according to a seventeenth embodiment will be described. In the voice response system according to the seventeenth embodiment, a server 90 detects abnormal behavior or a state of a user of the terminal device 1 based on a sensor value detected by the terminal device 1, and performs a process of reporting an abnormality when the abnormality is detected.

[0299] In detail, the terminal device 1 performs the behavior response terminal process shown in Fig. 32, and the server 90 performs the behavior response server process. In the behavior response terminal process, as shown in Fig. 32, first, outputs from various sensors mounted on the terminal device 1 are acquired (S522), and an image captured by the camera 41 is acquired (S524). Then, the acquired outputs from the various sensors and the captured image are transmitted as packets to the server 90 (S526), ​​and the behavior response terminal process is terminated.

[0300] Next, in the behavior response server process, first, the above-mentioned processes of S42 to S44 are performed as shown in Fig. 33. Next, based on the location information of the terminal device 1 (detection result by the GPS receiver 27), behavior such as wandering is identified (S532), and the user's environment is detected based on the detection results by the temperature sensors 15, 19, etc. (S534). Then, an abnormality is detected (S536).

[0301] In this process, an abnormality is detected based on the change in the location information and the environment. For example, if the user does not move in a hot or cold place, or if the user is in a place that the user does not usually go to, an abnormality is detected (S536). Alternatively, the location information and the environment are scored, and if this score falls below a reference value (outside the reference range), an abnormality is determined.

[0302] Next, it is determined whether or not an abnormality has been detected (S538). If no abnormality has been detected (S538: NO), the behavior response server process is terminated. If an abnormality has been detected (S538: YES), a message indicating that an abnormality has occurred is generated (S540) and a notification is sent to a predetermined contact point (S542). Then, the processes of S62 to S68 (except S66) described above are performed, and the behavior response server process is terminated.

[0303] [Effects of the 17th embodiment] In the voice response system 100, the server 90 detects the user's behavior and the user's surrounding environment, and generates a message in accordance with the detected behavior and surrounding environment.

[0304] Such a voice response system 100 can notify dangerous places, restricted areas, etc. Also, it can detect abnormal behavior of the user.

[0305] Furthermore, in the voice response system 100, the server 90 receives an image of the user. Based on the image, a health condition is determined and a message is generated in response to the health condition. Such a voice response system 100 makes it possible to manage the health condition of the user.

[0306] Furthermore, in the voice response system 100, the server 90 notifies a predetermined contact point when the health condition falls below a reference value. According to the voice response system 100, when the health condition of the user is below a reference value, a report can be issued, and thus an abnormality can be notified to others at an earlier stage.

[0307] [Other embodiments] The present invention is not limited to the above-described embodiment, and various other forms may be adopted as long as they fall within the technical scope of the present invention.

[0308] For example, the voice response system 100 may mediate communication between two or more parties. In detail, when vehicles need to give way to each other at an intersection or the like, the terminal devices 1 may negotiate with each other as to which vehicle will enter the intersection first. In this case, each terminal device 1 transmits information on the moving direction when approaching the intersection and the approach speed to the intersection to the server 90, and the server 90 sets priorities for each terminal device 1 according to the moving direction and the approach speed, and generates and outputs voices such as "Stop" or "Enter" according to the priority.

[0309] Furthermore, when the terminal device 1 accepts an incoming call (incoming call) for communication such as voice communication that requires a real-time response, the call may be accepted only when it is convenient for the user. Specifically, when the camera 41 captures an image of the user's face, the call may be accepted at a time convenient for the user.

[0310] Furthermore, some people feel annoyed if the other party does not respond when they call during voice communication, etc. In order to suppress such feelings, the other party's situation may be notified to the user who is waiting for a response from the other party. For example, the terminal device 1 may manage the user's schedule, and if the user does not respond to an incoming call, it may search for free time in the user's schedule and notify the user when he or she will be able to respond.

[0311] In addition, if the user does not respond to an incoming call, the location of the user may be communicated to the caller. For example, if the user is connected to the Internet via a smartphone or computer, it is possible to know which device is being operated. It is possible to identify the user's location from this information and communicate it to the caller.

[0312] Furthermore, whether or not the user can respond to an incoming call may be determined using location information using GPS or the like. Based on the location information, it is possible to determine whether or not the user is in a car or at home, and for example, if the user is on the move or in bed, it may be determined that the user is in a public place or asleep and therefore cannot respond to an incoming call. In this way, if the user cannot respond to an incoming call, it is possible to inform the caller of what the user is doing, as described above.

[0313] Another possible configuration for obtaining location information is to use security cameras. In recent years, security cameras have been installed in various places, and the location of the user can be recognized by using these security cameras and a configuration for identifying the user, such as face recognition. Security cameras can also be used to determine the situation, such as what the user is doing (whether or not the user is in a position to answer the phone). Whether or not the user is able to answer an incoming call can also be determined based on conditions such as whether or not the user is using a different landline phone (incoming calls cannot be answered while the landline phone is in use). Cut.

[0314] Furthermore, when a user of the terminal device 1 wants to have a conversation with someone, the results of learning the user's personality may be used to call up a terminal device that is estimated to have a good compatibility with the user among an unspecified number of users. In such a case, the terminal device may be configured to talk to the user about a topic that is likely to generate excitement (a topic that both users are interested in (extracted using the learning results)).

[0315] Also, when the voice response unit is not used for a long time (when the user does not speak for a reference time or longer), the voice response unit may prompt the user with some kind of message. At this time, location information such as GPS may be used to select the words to be spoken.

[0316] [Relationship between each means described in the claims or the means for solving the problems (present invention) and the configurations in the embodiments] The terminal device 1 and the server 90 in the above embodiment correspond to an example of a voice response unit of the present invention. The processes of 22 and S56 in the above embodiment correspond to an example of a response acquisition means of the present invention.

[0317] Furthermore, the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output means of the present invention, and the processes of S2 and S6 in the above embodiment correspond to an example of a voice input means of the present invention.

[0318] Furthermore, the process of S14 in the above embodiment corresponds to an example of a voice transmitting means of the present invention. Also, the answer candidate DB 105 in the above embodiment corresponds to an example of a answer recording means of the present invention.

[0319] Furthermore, the process of S56 in the above embodiment corresponds to an example of a personality information acquiring means of the present invention. Moreover, the processes of S22 and S56 in the above embodiment correspond to an example of a response acquiring means of the present invention.

[0320] Furthermore, the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output means of the present invention. Also, the processes of S254, S258, and S260 in the above embodiment correspond to an example of a first personality information generating means and a second personality information generating means of the present invention. Also, the process of S56 in the above embodiment corresponds to an example of a personality information acquiring means of the present invention.

[0321] Furthermore, the processes of S22 and S56 in the above embodiment correspond to a response acquisition unit of the present invention. Moreover, the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output unit of the present invention.

[0322] Furthermore, the processes of S254, S258, and S260 in the above embodiment correspond to an example of a first personality information generating means and a second personality information generating means of the present invention. Furthermore, the processes of S48 and S56 in the above embodiment correspond to an example of a response generating means of the present invention. Moreover, the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output means of the present invention.

[0323] Furthermore, the process of S48 in the modified example of the above embodiment corresponds to an example of a voice-input moving image obtaining means of the present invention, and the process of S52 in the above embodiment corresponds to an example of a character information converting means of the present invention.

[0324] Furthermore, the preference information generating process in the above embodiment is an example of the preference information generating means of the present invention. The process of S56 in the above embodiment corresponds to an example of a response candidate acquisition unit of the present invention.

[0325] Furthermore, the action character input process in the above embodiment corresponds to an example of the character information generating means of the present invention. Also, the other device information obtaining means (other terminal utilization process) in the above embodiment corresponds to an example of the forwarding means of the present invention.

[0326] Furthermore, the process of S98 in the above embodiment corresponds to an example of a reproduction condition determination means of the present invention, and the process of S100 in the above embodiment corresponds to an example of a message reproduction means of the present invention.

[0327] Furthermore, the process of S116 in the above embodiment corresponds to an example of a no-answer sending means of the present invention, and the process of S372 in the above embodiment corresponds to an example of an utterance accuracy detection means of the present invention.

[0328] Furthermore, the process of S374 in the above embodiment corresponds to an example of the accuracy degree output means of the present invention, and the process of S204 in the above embodiment corresponds to an example of the connection control means of the present invention.

[0329] Furthermore, the process of S50 in the above embodiment corresponds to an example of a feeling determination means of the present invention, and the process of S438 in the above embodiment corresponds to an example of a route information acquisition means of the present invention.

[0330] Furthermore, the process of S462 in the above embodiment corresponds to an example of the line-of-sight detection means of the present invention, and the process of S464 in the above embodiment corresponds to an example of the line-of-sight movement request transmission means of the present invention.

[0331] Furthermore, the process of S464 in the above embodiment corresponds to an example of a change request sending means of the present invention, and the process of S486 in the above embodiment corresponds to an example of a broadcast program acquiring means of the present invention.

[0332] Furthermore, the process of S484 in the above embodiment corresponds to an example of a broadcast program supplementation means and a lyrics addition means of the present invention. Also, the processes of S504 and S506 in the above embodiment correspond to an example of a pronunciation output means of the present invention. Also, the processes of S522 and S524 in the above embodiment correspond to an example of a behavioral environment detection means of the present invention.

[0333] Furthermore, the process of S538 in the above embodiment corresponds to an example of a health condition determining means of the present invention, and the process of S540 in the above embodiment corresponds to an example of a health message generating means of the present invention.

[0334] Furthermore, the process of S542 in the above embodiment corresponds to an example of a reporting means of the present invention. [Explanation of symbols]

[0335] 1...terminal device, 10...behavior sensor unit, 11...two-dimensional acceleration sensor, 13...axial gyro sensor, 15...temperature sensor, 17...humidity sensor, 19...temperature sensor, 21...humidity sensor, 23...illuminance sensor, 25...wetness sensor, 27...GPS receiver, 29...wind speed sensor, 33...electrocardiogram sensor, 35...heart sound sensor, 37...microphone, 39...memory, 41...camera, 50...communication unit, 53...wireless telephone unit, 55...contact memory, 60...alarm unit, 61...display, 63...illumination, 65...speaker, 70...operation unit, 71...touch pad, 73...confirmation button, 75...fingerprint sensor, 77...rescue request lever, 80...communication base station, 85...internet network, 90... server, 100... voice response system, 101... calculation unit, 102... voice recognition DB, 103... predictive conversion DB, 104... voice DB, 105... response candidate DB, 106... personality DB, 107... learning DB, 108... preference DB, 109... news DB, 110... weather DB, 111... playback condition DB, 112... handwritten characters / sign language DB, 113... terminal information DB, 114... emotion determination DB, 115... health determination DB, 116... karaoke DB, 117... report destination DB, 118... sales DB, 119... client DB.

Claims

[Claim 1] A voice response system including a request source device that is a request source of a request, a providing source device that is an information providing source, and a server capable of communicating with the request source device and the providing source device, the providing device is configured to be able to set whether or not to permit information provision; an information acquisition unit configured to acquire a request from the request source device, and if the request includes a request that specifies the providing source device, transmit the request to the specified providing source device, and if the information provision is permitted, acquire the provided information provided by the providing source device; a providing unit configured to generate a voice response based on the provided information as a response to a request from the requesting device, and provide the response to the requesting device; A voice response system comprising:

Images