Information processing system, program product, and information processing method

Abstract

Provided are an information processing system, a program product, and an information processing method, the information processing system being provided with a processor that performs: a process for specifying a notification status, which is a notification relating to the speaking of a person located in the vicinity of a subject and which is a status of the notification performed to the subject; and generating control information for notifying the surrounding person of the determined notification condition.

Description

TECHNICAL FIELD

[0001] The present application relates to an information processing system, a program product, and an information processing method. The present application can also be applied to a program and a program product. BACKGROUND

[0002] A process is disclosed in Patent Literature 1, which determines speech of a first user based on sensing information of a sensor device that senses at least one of the first user and a second user, and controls information output to the first user based on a determination result of the speech of the first user.

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2023-106649 SUMMARY

[0004] If a notification related to speech of a person located in the periphery of the object is made to the object, the object can know the content of the speech of the person in the periphery or the like.

[0005] Here, for example, if a notification is made to the object through a device that can be referred to only by the object, the person in the periphery cannot know the state of the notification. In this case, in a state in which the notification to the object has not ended, a state in which the person in the periphery makes new speech can occur.

[0006] Also, for example, a failure or the like of a device referred to by the object can be assumed. In this case, in a state in which the notification cannot be made to the object, a state in which the person in the periphery makes speech can occur.

[0007] An object of the present application is to enable a person in the periphery to know a notification related to speech of a person located in the periphery of an object and a state of the notification made to the object.

[0008] The application described in Aspect 1 is an information processing system including a processor that performs processing of determining a notification state that is a notification related to speech of a person in the periphery of an object, i.e., a person in the periphery, and that is a state of the notification made to the object, and generating control information that causes the notification of the determined notification state to the person in the periphery.

[0009] The application described in Aspect 2 is the information processing system described in Aspect 1, in which the processor performs processing of determining a progress state of the notification made to the object as the notification state, and generating control information that causes the notification of the progress state to the person in the periphery as the control information.

[0010] The invention described in Solution 3 is the information processing system described in Solution 2, wherein the processor performs display processing of content of speech of the person in the periphery, that is, speech content, and the progress of the display processing in the subject person reference device as the device referred to by the subject person as the progress, and generates control information that causes the progress of the display processing in the subject person reference device to be notified to the person in the periphery as the control information.

[0011] The invention described in Solution 4 is the information processing system described in Solution 3, wherein the processor generates control information that causes the end of the display processing in the subject person reference device to be notified to the person in the periphery as the control information.

[0012] The invention described in Solution 5 is the information processing system described in any one of Solutions 1 to 4, wherein the processor generates control information that causes output in a person in the periphery reference device, which is the device referred to by the person in the periphery, to change according to the notification condition as the control information.

[0013] The invention described in Solution 6 is the information processing system described in Solution 5, wherein the processor generates control information that causes output in the person in the periphery reference device to change with the end of the notification to the subject person as the control information.

[0014] The invention described in Solution 7 is the information processing system described in Solution 6, wherein the processor generates control information that causes the brightness of at least a part of the person in the periphery reference device to change with the end of the notification to the subject person as the control information.

[0015] The invention described in Solution 8 is the information processing system described in Solution 5 or 6, wherein the subject person is notified of content of speech, which is content of speech of the person in the periphery, and notification processing of the content of speech to the subject person is performed, and the processor further generates control information that causes information indicating that speech is possible to be output by the person in the periphery reference device with the end of the notification processing.

[0016] The invention described in Solution 9 is the information processing system described in any one of Solutions 1 to 8, wherein the processor generates control information that causes sound output from a sound source to change according to the condition of the notification to the subject person as the control information.

[0017] The invention described in Solution 10 is the information processing system described in Solution 9, wherein the processor generates control information that causes no sound to be emitted from the sound source with the end of the notification to the subject person, or generates control information that causes sound to be emitted from the sound source with the end of the notification as the control information.

[0018] The invention described in Solution 11 is the information processing system described in Solution 9, wherein the content of speech as the content of speech of the peripheral person is notified to the object, the notification processing of the content of speech of the object is performed, and the processor further generates control information that causes a sound corresponding to information indicating that speech is possible to be output from the sound source as the notification processing ends.

[0019] The invention described in Solution 12 is the information processing system described in any one of Solutions 1 to 11, wherein the processor generates, as the control information, control information that causes the notification of the notification condition to each of the peripheral persons by individual devices that are the devices each possessed by the peripheral persons.

[0020] The invention described in Solution 13 is the information processing system described in Solution 12, wherein the processor generates, as the control information, control information that causes a state of vibration of the individual devices to change according to the condition of the notification to the object.

[0021] The invention described in Solution 14 is the information processing system described in Solution 12 or 13, wherein the processor generates, as the control information, control information that causes display content in the individual devices to change according to the condition of the notification to the object.

[0022] The invention described in Solution 15 is the information processing system described in Solution 14, wherein the content of speech as the content of speech of the peripheral person is notified to the object, the notification processing of the content of speech of the object is performed, and the processor further generates control information that causes information indicating that speech is possible to be displayed on the individual devices as the notification processing ends.

[0023] The invention described in Solution 16 is the information processing system described in any one of Solutions 1 to 15, wherein the content of speech as the content of speech of a person who has spoken, i.e., a speaker, among the peripheral persons is notified to the object, and the processor performs processing of determining, as the notification condition, a notification condition of the content of speech of each of the speakers, and further generates control information that causes a notification manner when the notification condition determined for each of the speakers is notified to the peripheral persons to differ for each of the speakers.

[0024] The invention described in the item 17 is the information processing system described in any one of the items 1 to 16, wherein the notification related to the talk of the pre-registered person among the persons around the object person, that is, the registered person, is made to the object person, the notification related to the talk of the person other than the registered person, that is, the non-registered person, is not made to the object person, the processor performs the process of determining the state of the notification, that is, the notification state, related to the talk of the registered person and made to the object person as the notification state, and generating the control information that causes the notification of the notification state related to the notification made to the registered person to the persons around the object person, that is, the surrounding persons, as the control information.

[0025] The invention described in the item 18 is a program product that records a program for causing a computer to function, the function including a function of determining a notification state that is a notification related to the talk of a person around an object person, that is, a surrounding person, and is a state of the notification made to the object person, and a function of generating control information that causes the notification of the determined notification state to the surrounding person.

[0026] The invention described in the item 19 is an information processing method including a step of determining a notification state that is a notification related to the talk of a person around an object person, that is, a surrounding person, and is a state of the notification made to the object person, and a step of generating control information that causes the notification of the determined notification state to the surrounding person.

[0027] Effects of Invention

[0028] According to the item 1 of the present invention, it is possible to make the surrounding persons know the state of the notification related to the talk of the surrounding person around the object person and made to the object person.

[0029] According to the item 2 of the present invention, it is possible to notify the surrounding persons of the state of the notification made to the object person.

[0030] According to the item 3 of the present invention, it is possible to notify the surrounding persons of the state of the display process in the object person reference device referred to by the object person.

[0031] According to the item 4 of the present invention, it is possible to notify the surrounding persons that the display process in the object person reference device has ended.

[0032] According to the item 5 of the present invention, it is possible to change the output in the surrounding person reference device that is the device referred to by the surrounding person, in accordance with the notification state.

[0033] According to the item 6 of the present invention, it is possible to change the output in the surrounding person reference device as the notification made to the object person ends.

[0034] According to the 7th aspect of the present application, it is possible to notify the surrounding people that the notification to the object person has ended using the change in the brightness of at least a part of the surrounding people reference device.

[0035] According to the 8th aspect of the present application, it is possible to notify the surrounding people that it is possible to speak.

[0036] According to the 9th aspect of the present application, it is possible to notify the surrounding people of the status of the notification to the object person using sound.

[0037] According to the 10th aspect of the present application, it is possible to notify the surrounding people that the notification to the object person has ended using sound.

[0038] According to the 11th aspect of the present application, it is possible to notify the surrounding people that it is possible to speak.

[0039] According to the 12th aspect of the present application, it is possible to individually notify the surrounding people of the status of the notification to the object person.

[0040] According to the 13th aspect of the present application, it is possible to notify the surrounding people of the status of the notification to the object person using the vibration of an individual device.

[0041] According to the 14th aspect of the present application, it is possible to notify the surrounding people of the status of the notification to the object person through the display in an individual device.

[0042] According to the 15th aspect of the present application, it is possible to notify the surrounding people that it is possible to speak through the display in an individual device.

[0043] According to the 16th aspect of the present application, it is possible to notify the surrounding people that a notification is being made to the object person about which of the surrounding people.

[0044] According to the 17th aspect of the present application, it is possible to notify the surrounding people of the status of the notification related to the speech of the pre-registered surrounding people.

[0045] According to the 18th aspect of the present application, it is possible to enable the surrounding people to know the notification related to the speech of the surrounding people located around the object person and the status of the notification made to the object person.

[0046] According to the 19th aspect of the present application, it is possible to enable the surrounding people to know the notification related to the speech of the surrounding people located around the object person and the status of the notification made to the object person. BRIEF DESCRIPTION OF DRAWINGS

[0047] The embodiments of the present application are described in detail below with reference to the following drawings.

[0048] Figure 1 is a diagram showing the overall structure of the notification system;

[0049] Figure 2 This is a diagram showing the structure of the management server;

[0050] Figure 3 It is a diagram showing the hardware structure of the device;

[0051] Figure 4 This is a diagram representing the database stored in the information storage unit of the management server;

[0052] Figure 5 (A) to (C) are diagrams representing an example of the processing performed in the notification system;

[0053] Figure 6 (A) to (H) are diagrams representing a series of processes for display processing;

[0054] Figure 7 (A) to (C) are diagrams representing another treatment example;

[0055] Figure 8 (A) to (D) are diagrams showing processing examples when using a sound source;

[0056] Figure 9 (A) to (C) are diagrams illustrating processing examples when outputting speaking permission information.

[0057] Symbol Explanation

[0058] 1-Notification system, 11a-CPU, 41-Target, 42-Surrounding people, 91-Individual device, 93-Sound source, 200-Device, 300-Management server, 500-Light-emitting device. Detailed Implementation

[0059] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0060] Figure 1 This is a diagram showing the overall structure of the notification system 1 in this embodiment.

[0061] A management server 300 is provided in the notification system 1. Furthermore, a device 200 worn by the recipient (described later) is provided in the notification system 1.

[0062] exist Figure 1 The image shows only one device 200, but multiple devices 200 can be set up depending on the number of subjects. Device 200 is an eyeglass-type device. Device 200 is worn on the head of the subject. The subject uses device 200 to visually perceive their surroundings.

[0063] Further, a separate microphone 600 worn by each of the surrounding persons described later is provided. The separate microphone 600 acquires the voices of the surrounding persons and generates voice information.

[0064] The separate microphone 600 is provided for each of the surrounding persons. In a case where the surrounding persons are a plurality of persons, a plurality of separate microphones 600 are provided.

[0065] Further, a light emitting device 500, which is an example of a surrounding person reference device, is provided.

[0066] In the light emitting device 500, for example, a light source such as an LED and a CPU, which is an example of a processor that controls the output of the light source, are provided.

[0067] The light emitting device 500 notifies the surrounding persons of the notification related to the speech and the state of the notification made to the object person.

[0068] In addition, as described later, the notification to the surrounding persons is also made via a sound source or the like, which is a device other than the light emitting device 500.

[0069] The device 200 is connected to the management server 300 via a communication line 400 such as the Internet. Further, the separate microphone 600 and the light emitting device 500 are also connected to the management server 300 via the communication line 400.

[0070] Structure of Management Server

[0071] Figure 2 is a diagram showing the structure of the management server 300. The management server 300 is realized by a computer.

[0072] The management server 300, which functions as an information processing system, has an arithmetic processing section 111 that performs digital arithmetic processing according to a program. Further, the management server 300 has an information storage section 19 that stores information.

[0073] The information storage section 19 is realized by an existing information storage device. Specifically, the information storage section 19 is realized by, for example, an HDD (Hard Disk Drive). Further, the information storage section 19 is realized by, for example, a semiconductor memory.

[0074] In the arithmetic processing section 111, a CPU 11a, which is an example of a processor, is provided.

[0075] Further, in the arithmetic processing section 111, a RAM 11b used as a work memory or the like of the CPU 11a is provided. Further, in the arithmetic processing section 111, a ROM 11c that stores a program or the like executed by the CPU 11a is provided.

[0076] Also, a nonvolatile memory 11d that can save data even in the case of interruption of power supply is provided in the arithmetic processing section 111.

[0077] The nonvolatile memory 11d is constituted by, for example, SRAM or flash memory backed up by a battery. Various information such as programs executed by the CPU 11a is stored in the information storage section 19.

[0078] The CPU 11a provided in the arithmetic processing section 111 reads the programs stored in the ROM 11c or the information storage section 19. Thus, various processes performed in the management server 300 are executed by the management server 300.

[0079] The programs executed by the CPU 11a can be provided to the management server 300 in a state stored in a recording medium. The recording medium is a recording medium readable by a computer.

[0080] As the recording medium, a magnetic recording medium such as a magnetic tape or a magnetic disk, an optical recording medium such as an optical disk, and the like can be given. Also, as the recording medium, a magneto-optical recording medium, a semiconductor memory, and the like can be given.

[0081] Further, the programs executed by the CPU 11a can be provided to the management server 300 using a communication unit. As the communication unit, for example, a communication unit using an Internet line can be given.

[0082] [Structure of the device]

[0083] Figure 3 is a diagram showing the hardware structure of the device 200.

[0084] The device 200 is provided with an arithmetic processing section 211, an information storage section 212, a sensor 213, and a device camera 214. Also, the device 200 is provided with a device microphone 215, a speaker 216, and a display section 217.

[0085] A CPU 21a that is an example of a processor is provided in the arithmetic processing section 211.

[0086] Also, a RAM 21c used as a work memory or the like of the CPU 21a is provided in the arithmetic processing section 211. Also, a ROM 21b that stores programs executed by the CPU 21a or the like is provided in the arithmetic processing section 211.

[0087] The information storage section 212 is realized by an existing information storage device such as a semiconductor memory.

[0088] As the sensor 213, a GPS sensor, a direction sensor, and the like can be given as an example. By referring to the output from the sensor 213, the current position of the device 200 and the orientation of the device 200 can be determined.

[0089] The device camera 214 is a camera that captures the surroundings of the device 200.

[0090] In a state in which the device 200 is worn by the subject, the device camera 214 captures a front direction of the subject toward the front direction. In other words, the device camera 214 captures a front direction of the subject toward a direction in which the subject is facing.

[0091] The device microphone 215 acquires a voice of the subject and generates voice information.

[0092] The speaker 216 outputs a sound or a voice and performs a notification process to the subject who wears the device 200.

[0093] The display section 217 is a so-called display that displays various information. The display section 217 is disposed in front of the eyes of the subject in a state in which the device 200 is worn by the subject.

[0094] An image obtained by the device camera 214 is displayed on the display section 217. In a state in which the subject wears the device 200, an image that reflects a state in front of the subject is displayed on the display section 217.

[0095] In the present embodiment, the subject performs visual recognition of a front of the subject by referring to the image that appears on the display section 217.

[0096] The device 200 can be understood as a subject reference device that is referred to by the subject.

[0097] In addition, there is also a transmissive device 200.

[0098] In this case, as the display section 217, a transparent display section 217 is provided, which the subject can visually recognize a back of the display section 217.

[0099] The subject performs visual recognition of the back of the display section 217 through the display section 217. In other words, the subject visually recognizes a front of the subject through the display section 217.

[0100] In the transmissive device 200, an image is displayed on the display section 217. The user performs visual recognition of both a real space located at the back of the display section 217 and the image displayed on the display section 217.

[0101] A program executed by the CPU 21a can be provided to the device 200 in a state stored in a recording medium. The recording medium is a recording medium that is readable by a computer.

[0102] As the recording medium, similarly to the above, a magnetic recording medium such as a magnetic tape or a magnetic disk, an optical recording medium such as an optical disk can be given. Further, as the recording medium, a magneto-optical recording medium, a semiconductor memory can be given in addition.

[0103] Further, the program executed by the CPU 21a can be provided to the device 200 using a communication unit. As the communication unit, for example, a communication unit using an Internet line can be given.

[0104] In addition, as the device 200, it is not limited to the eyeglass-type device, and a smartphone or a tablet terminal, and the like can be given in addition. The eyeglass-type device, the smartphone, the tablet terminal, and the like are all devices that the subject can carry.

[0105] A display portion and a device camera are also provided in the smartphone or the tablet terminal. In this case, the subject is able to perform visual recognition of the front thereof by referring to an image captured by the device camera and displayed on the display portion.

[0106] In this case, the subject refers to the display portion provided on the smartphone or the tablet terminal disposed in front of the subject. Thus, the subject is able to perform visual recognition of a space located in front of the subject and a space located behind the smartphone or the tablet terminal.

[0107] In the present embodiment, the subject is subjected to a notification process of the utterance content described later via the eyeglass-type device 200. However, it is not limited thereto, and the notification process can be performed even in the case of using a smartphone or a tablet terminal.

[0108] In the present specification, the processor refers to a broad processor.

[0109] As the processor, a general-purpose processor can be given as an example. As the general-purpose processor, for example, a CPU (Central Processing Unit) can be given.

[0110] Further, as the processor, a special-purpose processor can be given as an example. As the special-purpose processor, for example, a GPU (Graphics Processing Unit) can be given. Further, as the special-purpose processor, for example, an ASIC (Application Specific Integrated Circuit) can be given in addition. Further, as the special-purpose processor, for example, a FPGA (Field Programmable Gate Array), a programmable logic device can be given in addition.

[0111] The actions of the processor are not limited to being performed by one processor. The actions of the processor can be actions that are cooperatively performed by a plurality of processors existing in physically separate locations. Also, the order of the actions of the processor is not limited to the order described in the present embodiment and can be changed.

[0112] (Explanation of Database)

[0113] Figure 4 is a view that shows a database stored in the information storage section 19 of the management server 300 (refer to Figure 2 ).

[0114] As shown in Figure 4 , in the database, information about the surrounding person 42 is registered for each surrounding person 42 (refer to Figure 5 ).

[0115] In the database, an identification ID that is information for identifying each surrounding person 42 is stored for each surrounding person 42.

[0116] Also, in the database, microphone identification information is stored for each surrounding person 42. The microphone identification information is identification information of the individual microphone 600 (refer to Figure 1 ) that each of the surrounding persons 42 has.

[0117] Further, in the database, face information and the like of the surrounding person 42 is registered for each surrounding person 42.

[0118] In the present embodiment, photography of the face of the surrounding person 42 is performed in advance. Then, face information that is information of the face of the surrounding person 42 is registered in the database.

[0119] As the face information, for example, an image of the face of the surrounding person 42 can be given. Also, as the face information, information about a feature amount of the face of the surrounding person 42 can be given in addition. The information about the feature amount is obtained by analyzing the image of the face of the surrounding person 42.

[0120] The video acquired by the device camera 214 (refer to Figure 3 ) and the voice information acquired by the individual microphone 600 (refer to Figure 1 ) are transmitted to the management server 300 (refer to Figure 1 ).

[0121] The CPU 11a (refer to Figure 2 ) of the management server 300 acquires the video or the voice information. Then, the CPU 11a acquires the video or the voice information as information of the surrounding person 42.

[0122] CPU11a determines the surrounding person 42 reflected in the image based on the image acquired by the device camera 214.

[0123] CPU 11a acquires images of the faces of the surrounding people 42 reflected in the image. Then, CPU 11a determines the surrounding people 42 based on the facial images and facial information stored in the database.

[0124] Furthermore, the CPU11a acquires the speech content of the people around it 42 based on the speech information obtained by the individual microphone 600.

[0125] [Description of the processing performed in notification system 1]

[0126] Figure 5 (A) to (C) are diagrams representing an example of the processing performed in notification system 1.

[0127] exist Figure 5 (A) shows the recipient 41 who performs the notification processing of the spoken content. Furthermore, in Figure 5 (A) shows multiple people 42 who are people surrounding the object 41.

[0128] In this embodiment, such as Figure 5 As shown in (A), the recipient 41 wears device 200. Through this device 200, the recipient 41 is informed of the content of the speech given by the person in the vicinity 42.

[0129] It is also possible to imagine a situation where the person 41 is hearing impaired. If the content of the speech 48 is communicated, the hearing impaired person will also be able to determine the content of the speech of the person 42 around them.

[0130] like Figure 5 As shown in (A), the presence of the person in the vicinity 42 is reflected on the display unit 217 provided on the device 200. Furthermore, the speech content 48 of the person in the vicinity 42 is displayed on the display unit 217.

[0131] Each person in the vicinity 42 wears an individual microphone 600. The voice information of the person in the vicinity 42 is acquired by the individual microphone 600. In addition, in this embodiment, the subject 41 also wears an individual microphone 600.

[0132] The device 200 worn by the subject 41 is an eyeglasses-type device 200.

[0133] The device 200 is worn on the head of the subject 41. The subject 41 uses the device 200 to visually identify people 42 around themselves. In other words, the subject 41 uses the device 200 to visually identify people 42 in front of them.

[0134] As described above, the device camera 214 (refer to Figure 3 ) is provided in the device 200. Also, the display section 217 that displays the image obtained by the device camera 214 is provided in the device 200.

[0135] The subject 41 visually recognizes the surrounding person 42 by referring to the surrounding person 42 photographed by the device camera 214 and displayed on the display section 217.

[0136] Further, the case where the device 200 is the above-described transmission-type device 200 is also conceivable. In this case, the subject 41 visually recognizes the surrounding person 42 located behind the display section 217 through the transparent display section 217.

[0137] Also, in the present embodiment, as shown in (A) of Figure 5 , the light emitting device 500 is provided. The light emitting device 500 is provided at a position where the surrounding person 42 can visually recognize. The role of the light emitting device 500 will be described later.

[0138] The CPU 11a of the management server 300 determines the surrounding person 42 who has spoken when acquiring the voice information acquired by the individual microphone 600.

[0139] The microphone identification information is sent from the individual microphone 600 together with the voice information.

[0140] The CPU 11a acquires the microphone identification information sent to the management server 300 together with the voice information.

[0141] The CPU 11a determines the surrounding person 42 on the basis of the acquired microphone identification information and the microphone identification information stored in the database.

[0142] The microphone identification information for identifying each individual microphone 600 is stored in each individual microphone 600. The microphone identification information is sent from the individual microphone 600 to the management server 300 at the same time as the voice information.

[0143] The CPU 11a determines the surrounding person 42 on the basis of the microphone identification information and the microphone identification information registered in the database. The CPU 11a determines the surrounding person 42 who has spoken, which becomes the source of the voice information acquired by the individual microphone 600.

[0144] In the present embodiment, the surrounding person 42 who has spoken is determined each time the person speaks, and for each surrounding person 42.

[0145] The individual microphone 600 is prepared for each surrounding person 42.

[0146] The individual microphone 600 obtains the voice information of the surrounding person 42. The voice information obtained by the individual microphone 600 is transmitted to the management server 300 together with the microphone identification information.

[0147] The CPU 11a determines the surrounding person 42 on the basis of the microphone identification information. Further, the CPU 11a obtains the voice information on the determined surrounding person 42.

[0148] Further, the CPU 11a obtains the speech content 48 of the determined surrounding person 42 on the basis of the obtained voice information. In addition, the obtaining of the speech content 48 on the basis of the voice information can be performed using a publicly known method.

[0149] The voice information is selected by the individual microphone 600 at the time of transmission of the voice information from the individual microphone 600 to the management server 300. In the individual microphone 600, the voice information whose sound pressure exceeds a predetermined threshold is selected from the obtained voice information.

[0150] Then, the selected voice information is transmitted from the individual microphone 600 to the management server 300 together with the microphone identification information.

[0151] Thus, the voice information of the other surrounding person 42 who is not intended to obtain the voice information is suppressed from being transmitted to the management server 300.

[0152] The "other surrounding person 42" refers to the other surrounding person 42 who is different from the surrounding person 42 who wears the individual microphone 600. In other words, the "other surrounding person 42" refers to the surrounding person 42 who does not wear the individual microphone 600.

[0153] In the present embodiment, the voice information of the other surrounding person 42 is suppressed from being transmitted to the management server 300 by the individual microphone 600.

[0154] The other surrounding person 42 is away from the surrounding person 42 who wears the individual microphone 600. In this case, the sound pressure of the voice of the other surrounding person 42 obtained by the individual microphone 600 becomes small. In this case, the voice information of the other surrounding person 42 is not transmitted to the management server 300.

[0155] In the present embodiment, the voice information whose sound pressure exceeds a predetermined threshold is selected. Then, the selected voice information is transmitted to the management server 300. In this case, the voice information of the other surrounding person 42 is not transmitted to the management server 300.

[0156] In addition, the selection of the voice information whose sound pressure exceeds a predetermined threshold can also be performed by the management server 300.

[0157] In this case, the management server 300 selects voice information from the voice information transmitted from the individual microphone 600. The management server 300 selects voice information in which the sound pressure exceeds a predetermined threshold from the transmitted voice information.

[0158] In this case, the management server 300 acquires the selected voice information as voice information of the surrounding person 42 who wears the individual microphone 600.

[0159] Also, further, the CPU 11a can acquire voice information obtained by a terminal device as voice information of the surrounding person 42. As the terminal device, for example, a smart phone or a tablet terminal possessed by each of the surrounding persons 42 can be cited as an example.

[0160] A microphone is provided in the smart phone or the tablet terminal.

[0161] The CPU 11a can also acquire voice information obtained by the microphone of the terminal device as voice information of the surrounding person 42. In this case, the CPU 11a acquires voice information obtained by the microphone and transmitted to the management server 300 as voice information of the surrounding person 42.

[0162] Also, further, a common microphone can be provided. In this case, the CPU 11a acquires voice information obtained by the common microphone. Then, the CPU 11a acquires voice information of each of the surrounding persons 42 on the basis of the voice information.

[0163] In the case of using the common microphone, characteristic information of voice is registered in the database in advance. Specifically, characteristic information of voice of each of the surrounding persons 42 is registered in the database for each of the surrounding persons 42.

[0164] In this case, the CPU 11a acquires voice information of each of the surrounding persons 42 on the basis of the characteristic information registered in the database.

[0165] Also, the CPU 11a of the management server 300 determines each of the surrounding persons 42 who are displayed on the display section 217 (refer to Figure 5 (A)). The CPU 11a determines each of the surrounding persons 42 who are displayed on the display section 217 in addition to the acquisition of voice information of each of the surrounding persons 42.

[0166] The CPU 11a determines each of the surrounding persons 42 on the basis of an image obtained by the device camera 214 (refer to Figure 3 ).

[0167] Specifically, the CPU 11a determines each of the surrounding persons 42 on the basis of an image and face information of the surrounding persons 42.

[0168] The CPU 11a acquires an image of the surrounding person 42 captured by the device camera 214. Then, the CPU 11a determines each surrounding person 42 based on the image of the surrounding person 42 and the face information within the image. The CPU 11a determines each surrounding person 42 based on the image of the surrounding person 42 and the face information registered in the database.

[0169] Then, the CPU 11a generates control information that causes the spoken content 48 to be associated with the surrounding person 42 who spoke. The CPU 11a generates control information that causes the spoken content 48 to be associated with the surrounding person 42 appearing on the display section 217.

[0170] As described above, the CPU 11a determines the surrounding person 42 who spoke and the spoken content 48 of the surrounding person 42. Also, the CPU 11a determines each surrounding person 42 appearing on the display section 217 of the device 200.

[0171] Then, the CPU 11a generates control information that causes the spoken content 48 to be associated with the surrounding person 42. The CPU 11a generates control information that causes the spoken content 48 to be associated with the surrounding person 42 appearing on the display section 217.

[0172] The CPU 11a generates control information containing position information about the display position of the spoken content 48 when generating the control information.

[0173] The CPU 11a determines the position of the surrounding person 42 appearing on the display section 217 as the display position of the spoken content 48. More specifically, the CPU 11a determines the position around the head of the surrounding person 42 as the display position of the spoken content 48.

[0174] Then, the CPU 11a generates control information containing position information about the display position. Also, the CPU 11a contains the determined spoken content 48 in the control information.

[0175] Then, in the present embodiment, the control information containing the position information and the spoken content 48 is transmitted from the management server 300 to the device 200.

[0176] Then, the CPU 21a (see Figure 3 ) of the device 200 performs display control of the display section 217 based on the control information. Thus, the spoken content 48 is displayed at the position determined according to the position information in the display section 217.

[0177] Thus, in the display section 217 of the device 200, for example, the spoken content 48 is displayed at the position around the head of the surrounding person 42 appearing on the display section 217. Figure 5(A) of FIG. 1. In this display, the speech content 48 is displayed in a state in which the speech content 48 is associated in correspondence with the surrounding person 42 who spoke.

[0178] In the present embodiment, the speech content 48 of the surrounding person 42 is displayed on the display section 217 of the device 200. In the present embodiment, if the surrounding person 42 speaks, the speech content 48 of the surrounding person 42 is displayed on the display section 217 of the device 200.

[0179] In the present embodiment, when the speech content 48 is displayed, a display image 45 constituted by an image indicating a dialogue box is displayed. The speech content 48 is displayed inside the display image 45.

[0180] Figure 6 (A) to (H) of FIG. 1 are diagrams indicating a flow of a series of display processes.

[0181] In Figure 6 (A) of FIG. 1 illustrates a state in which the surrounding person 42 does not speak, and in this case, the CPU 11a does not acquire the voice information. Also, in this case, the CPU 11a does not generate the control information that causes the speech content 48 to be displayed.

[0182] In this case, the speech content 48 is not displayed on the display section 217 of the device 200.

[0183] Figure 6 (B) to (H) of FIG. 1 illustrate a state after the surrounding person 42 starts speaking.

[0184] In this case, the CPU 11a acquires the voice information. The CPU 11a acquires the speech content 48 on the basis of the voice information. Then, the CPU 11a generates the control information that causes the speech content 48 to be displayed.

[0185] As a result, as indicated by a symbol 13X, the speech content 48 is displayed on the display section 217 of the device 200. More specifically, the speech content 48 is displayed inside the display image 45 displayed on the display section 217 of the device 200.

[0186] As described above, the speech content 48 is displayed in a state in which the speech content 48 is associated in correspondence with the surrounding person 42 who spoke.

[0187] As a result, the speech content 48 of the surrounding person 42 is notified to the target person 41. In the present embodiment, the notification process of the speech content 48 to the target person 41 is performed as such.

[0188] There is a time difference between the timing at which the voice information is acquired and the timing at which the speech content 48 is displayed.

[0189] The timing for acquiring voice information refers to the timing at which the CPU11a acquires voice information.

[0190] The timing for displaying the spoken content 48 refers to the timing for displaying the spoken content 48 in the start display unit 217.

[0191] In this embodiment, as shown by arrow 14Y, the display of the spoken content 48 begins after the CPU 11a acquires the voice information.

[0192] Figure 6 (F) indicates the timing of the end of the speech of the person in the surrounding area 42.

[0193] In this case, as indicated by symbol 14P, in Figure 6 The display processing of the spoken content 48 ends after the timing in (F). And, in this case, as shown by symbol 14Z, in Figure 6 After the timing in (F), the spoken content 48 disappears.

[0194] Even after the person 42 in the vicinity finishes speaking, the display process continues. Even after the person 42 in the vicinity finishes speaking, the display process continues until the last part of the speech content 48 is displayed.

[0195] If the last part of the spoken content 48 is displayed on the display unit 217, the display processing in the display unit 217 ends. This results in the state shown by symbol 14P. Then, as shown by symbol 14Z, the spoken content 48 disappears.

[0196] [Notification to those in the vicinity]

[0197] CPU 11a determines the status of the notification made to the recipient 41, i.e., the notification status. As described above, the recipient 41 is notified of the content 48 of the speech of the person in the vicinity 42. CPU 11a determines the status of this notification made to the recipient 41, i.e., the notification status.

[0198] CPU11a determines at predetermined intervals Figure 6 The symbol 6X indicates the notification processing status, i.e., the notification status.

[0199] CPU11a determines the notification status at predetermined intervals based on information sent from device 200 at predetermined intervals.

[0200] Device 200 sends information about the status of the aforementioned notification to object 41 to management server 300. This information is sent from device 200 to management server 300 at predetermined intervals.

[0201] The CPU 11a determines the notification condition based on the information transmitted to the management server 300. The CPU 11a determines the notification condition based on the information as the condition of the notification to the target person 41.

[0202] As described above, the notification related to the speech of the surrounding person 42 is made to the target person 41. Specifically, the notification of the speech content 48 of the surrounding person 42 is made to the target person 41.

[0203] The CPU 11a determines the condition of the notification of the speech content 48. The CPU 11a determines the progress condition of the notification of the speech content 48 made to the target person 41 as the notification condition.

[0204] More specifically, the CPU 11a determines the progress condition of the display processing in the device 200 as the progress condition.

[0205] The CPU 11a determines the progress condition of the display processing of the speech content 48 of the surrounding person 42 and the display processing in the device 200.

[0206] Then, the CPU 11a generates control information that causes the determined progress condition to be notified to the surrounding person 42.

[0207] The control information is output to the light emitting device 500 which is an example of the surrounding person reference device.

[0208] Accordingly, the CPU (not shown) provided to the light emitting device 500 performs the control of the light emitting device 500 based on the control information.

[0209] The CPU 11a of the management server 300, for example, generates control information that causes the output in the surrounding person reference device to be changed as the control information. The surrounding person reference device refers to a device such as the light emitting device 500 which is referred to by the surrounding person 42.

[0210] The CPU 11a generates control information that causes the output in the surrounding person reference device to be changed according to the condition of the notification in the device 200.

[0211] The CPU 11a generates control information that causes the output in the surrounding person reference device to be changed as the notification in the device 200 ends.

[0212] More specifically, the CPU 11a generates control information that causes the luminance of at least a part of the surrounding person reference device to be changed. The CPU 11a generates the control information as the notification made to the target person 41 ends.

[0213] The CPU 11a generates control information that causes the surrounding person 42 to be notified that the display processing in the device 200 has ended.

[0214] Then, in the present embodiment, the generated control information is transmitted to the light emitting device 500 (refer to Figure 5 (A)), for example. Then, based on the control information, the lighting control of the light emitting device 500 as an example of a peripheral person reference device is performed.

[0215] Thereby, the peripheral person 42 recognizes the notified condition in the device 200 by referring to the light emitting device 500.

[0216] In Figure 5 (A) to (C), an example of the processing when the output of the light emitting device 500 as an example of a peripheral person reference device is changed is shown.

[0217] In Figure 5 (A), the condition of the start of the speech of the peripheral person 42 shown by the symbol 5A among a plurality of peripheral persons 42 is shown. Also, in Figure 5 (A), the condition of the start of the display processing of the speech content 48 in the display section 217 is shown.

[0218] In this case, the light emitting device 500 is lit. If the display processing of the speech content 48 is started in the display section 217, the light emitting device 500 is lit.

[0219] In addition, in the present embodiment, the configuration is made such that the majority of the light emitting device 500 is lit, but the configuration can also be such that the light emitting device 500 is locally lit.

[0220] In the condition shown in Figure 5 (A), the CPU 11a generates the control information such that the light emitting device 500 is lit. In the condition in which the display processing is being performed by the device 200, the CPU 11a generates the control information such that the light emitting device 500 is lit.

[0221] Thereby, as shown in Figure 5 (A), the light emitting device 500 is lit.

[0222] Figure 5 (B) shows the state just before the speech of the peripheral person 42 shown by the symbol 5A is ended. Even at the timing of Figure 5 (B), the display processing is still being performed in the display section 217 of the device 200.

[0223] In this case, as well as the above, the CPU 11a generates the control information such that the light emitting device 500 is lit.

[0224] Thereby, as shown in Figure 5 (B), the light emitting device 500 continues to be lit.

[0225] Figure 5(C) shows the state where the display processing in the display unit 217 of device 200 has ended. Figure 7 In the situation shown in (C), the people around 42 also stopped talking.

[0226] In this case, CPU 11a generates control information that causes the light-emitting device 500 to turn off. This control information is generated if CPU 11a determines that display processing in device 200 has ended.

[0227] Additionally, "display processing finished" refers to the state where the display of the last part of the acquired speech content 48 has ended.

[0228] If control information is generated that causes the light-emitting device 500 to turn off, then the light-emitting device 500 will turn off.

[0229] In this case, the display processing of the speech content 48 in each of the surrounding people 42 recognition devices 200 has ended. In this case, each of the surrounding people 42 can begin to speak.

[0230] Figure 7 (A) to (C) are diagrams representing another treatment example.

[0231] This processing example illustrates a process in which the output of a separate device 91 is changed based on the status of the notification given to the object 41.

[0232] "Individual equipment 91" refers to the equipment owned by each of the surrounding people 42.

[0233] CPU 11a generates control information that causes the output of individual device 91 to change according to the notification given to object 41. In this processing example, CPU 11a generates control information that causes the vibration state of individual device 91 to change. A vibration source (not shown) is provided in individual device 91.

[0234] It can also notify people 42 in the vibrating state of a separate device 91 that a conversation has started or ended, and display the content 48 of the conversation. Additionally, in Figure 7 In this process, the voices of the surrounding people 42 are acquired by a microphone (not shown) installed in a separate device 91.

[0235] As a standalone device 91, a smartphone or tablet terminal can be cited as an example. Smartphones or tablet terminals typically contain a vibration source. Furthermore, standalone device 91 can be a dedicated device for transmitting vibrations to people 42 in the vicinity.

[0236] In addition, Figure 7 Of these, 91 are smartphones.

[0237] Figure 7(A) of FIG. 7 shows a situation in which the talking of the person 42 indicated by the symbol 7A in the plurality of persons 42 starts.

[0238] In this processing example, in this case, the individual device 91 possessed by each of the persons 42 vibrates.

[0239] In the present embodiment, the vibration of the individual device 91 is started along with the start of the display processing in the device 200.

[0240] In this processing example, the CPU 11a first determines that the display processing in the device 200 has started. Then, the CPU 11a generates control information that causes the individual device 91 to vibrate in accordance with this determination. Thereby, the vibration of the individual device 91 is started.

[0241] Figure 7 (B) of FIG. 7 shows a state just before the talking of the person 42 indicated by the symbol 7A ends.

[0242] In this case, the CPU 11a generates control information that causes the individual device 91 to vibrate as well as in the above. Thereby, the individual device 91 continues to vibrate.

[0243] Figure 8 (C) of FIG. 7 shows a situation in which the display processing in the device 200 has ended.

[0244] In this case, the CPU 11a generates control information that causes the vibration of the individual device 91 to stop. Thereby, the vibration of the individual device 91 stops.

[0245] In this case, each of the persons 42 recognizes that the display processing in the device 200 has ended.

[0246] Each of the persons 42 first recognizes that the individual device 91 no longer vibrates. Accordingly, each of the persons 42 recognizes that the display processing has ended.

[0247] Further, the CPU 11a can also generate control information that causes the sound output from the sound source to change as the control information. In this case, the CPU 11a generates control information that causes the sound to change in accordance with the situation of the notification to the target person 41.

[0248] More specifically, the CPU 11a generates, for example, control information that causes no sound to be emitted from the sound source along with the end of the notification.

[0249] Further, the CPU 11a generates, for example, control information that causes sound to be emitted from the sound source along with the end of the notification.

[0250] Figure 8 (A) to (D) of FIG. 7 are diagrams that show processing examples when the sound source 93 is used.

[0251] Figure 8 (A) shows the state at the beginning of speech of the surrounding person 42, indicated by symbol 8A among the multiple surrounding persons 42. Furthermore, Figure 8 (A) shows the state at which the display of the spoken content 48 in the display unit 217 begins.

[0252] In this case, CPU 11a generates control information to start outputting sound from sound source 93. Thus, sound output from sound source 93 begins.

[0253] In this example, the sound output is a typing sound or a clicking sound, indicating that information input is in progress. In this processing example, if display processing begins on device 200, sound is output from sound source 93.

[0254] CPU 11a generates control information that causes a predetermined specific sound to be output from sound source 93. Thus, the predetermined specific sound, such as a typing sound or a clicking sound, is output from sound source 93.

[0255] In this embodiment, as the display processing in the device 200 begins, sound is output from the sound source 93.

[0256] Figure 8 (B) shows the state just before the person 42, represented by symbol 8A, is about to finish speaking.

[0257] In this case, CPU 11a generates control information that causes a predetermined specific sound to be output from sound source 93. Thus, sound continues to be output from sound source 93.

[0258] Figure 8 (C) shows the status that the display processing in device 200 has ended.

[0259] In this situation, CPU 11a generates control information that prevents sound source 93 from emitting sound. Therefore, no typing or clicking sounds are output.

[0260] Furthermore, in this processing example, CPU 11a generates control information that causes sound source 93 to emit a new sound. This "new sound" refers to a sound different from the sound output from sound source 93 during display processing in device 200.

[0261] Thus, as indicated by symbol 8C, the output indicates a new sound that signifies the end of the processing.

[0262] Then, as Figure 8 As shown in (D), it becomes a state in which no sound from sound source 93 is emitted.

[0263] If become Figure 9In the state shown in (C), each of the surrounding persons 42 recognizes that the display processing in the device 200 has ended. Also, in the state shown in (D), each of the surrounding persons 42 also recognizes that the display processing in the device 200 has ended. Figure 9 In the state shown in (C), each of the surrounding persons 42 recognizes that the display processing in the device 200 has ended. Also, in the state shown in (D), each of the surrounding persons 42 also recognizes that the display processing in the device 200 has ended.

[0264] In the above, the case where two sounds are output is described. Alternatively, only one of the two sounds can be output.

[0265] In the above, one of the two sounds is output during the display processing in the device 200. Specifically, the above typing sound or click sound is output.

[0266] Also, the other of the two sounds is output when the display processing in the device 200 ends. Specifically, the above new sound is output.

[0267] The two sounds do not necessarily have to be output. Alternatively, a configuration in which only one of the two sounds is output can be provided.

[0268] Also, the CPU 11a can further generate control information that causes output of information indicating that speech is possible. Hereinafter, in the present specification, the information indicating that speech is possible is referred to as "speech permission information".

[0269] In this case, the CPU 11a generates control information that causes the speech permission information to be output by the surrounding person reference device along with the end of the notification processing.

[0270] The surrounding person reference device refers to a device that is referenced by the surrounding persons 42. As the surrounding person reference device, the above light emitting device 500, a smartphone, a tablet terminal, a PC (Personal Computer), or the like can be given. Also, as the surrounding person reference device, the above device 200 worn by the target person 41 can be given as an example.

[0271] As for the device 200, not only is it worn by the target person 41, but it is sometimes worn by the surrounding persons 42. Therefore, the device 200 also corresponds to a device that is referenced by the surrounding persons 42, that is, a surrounding person reference device.

[0272] In the following description, a case where the speech permission information is displayed on the device 200 worn by the surrounding persons 42 is described as an example.

[0273] Alternatively, the speech permission information is not limited to being displayed on the device 200, but can be displayed on a smartphone, a tablet terminal, a PC, or the like.

[0274] Here, the device 200, the smartphone, the tablet terminal, the PC, or the like can also be understood as a separate device. The separate device refers to a device that each of the surrounding persons 42 has.

[0275] The aforementioned notification to people in the vicinity via vibration can be considered an example of notification using a separate device.

[0276] Other examples of standalone devices include watch-type devices and ring-type devices. Even watch-type and ring-type devices can send notifications using the light, vibration, or sound described above.

[0277] Figure 9 (A) to (C) are diagrams illustrating processing examples when outputting speaking permission information.

[0278] Figure 9 (A) shows the situation where the speech of the person in the surrounding person 42, indicated by symbol 9A, begins.

[0279] In this processing example, at this time, the device 200 worn by the person in the vicinity 42 displays "Speaking". The device 200 displays information indicating that the speaking content 48 is in progress.

[0280] Figure 9 (B) shows the state just before the person 42, indicated by symbol 9A, is about to finish speaking. In this case, as described above, "Speaking" is displayed on the device 200 worn by the person 42.

[0281] In this case, information indicating that the speech content 48 is being processed is also displayed on the device 200.

[0282] Figure 9 (C) shows the state that the display processing in the device 200 worn by the subject 41 has ended.

[0283] If the display processing on the device 200 worn by the person being addressed (41) has ended, the "speaking" display disappears. The "speaking" display on the device 200 worn by the person in the vicinity (42) also disappears.

[0284] Therefore, in this situation, the notification process of the content of the speech 48 is also completed to the people around 42.

[0285] Moreover, in this processing example, such as Figure 8 As shown in (C), the character "Can Talk" is displayed. In other words, in this processing example, the message "Talking is allowed" is displayed.

[0286] In this example, control information is generated that causes a change in the display content on the device 200 worn by the person in the vicinity 42. In this processing example, according to the control information, the display on the display unit 217 of the device 200 worn by the person in the vicinity 42 changes.

[0287] In the present embodiment, the state in which the sentence "talking" is displayed becomes the state in which the sentence is eliminated.

[0288] Also, the state in which the talking permission information "talkable" is not displayed becomes the state in which the talking permission information is displayed.

[0289] In the present processing example, if the state indicated by (C) in the above becomes the state in which the talking permission information is displayed, the surrounding person 42 also recognizes that the timing at which the surrounding person 42 can talk. ​

[0290] In the present embodiment, the state in which the sentence "talking" is displayed becomes the state in which the sentence is eliminated.

[0291] The sound corresponding to the talking permission information can also be output from the sound source 93 (see FIG. 1). ​

[0292] As the "sound corresponding to the talking permission information", for example, a sound obtained by reading out the character information can be cited as an example. Specifically, a sound obtained by reading out the character information "talkable" can be cited as an example.

[0293] Also, the "sound corresponding to the talking permission information" can be a sound other than the sound obtained by reading out the character information. As the sound other than the sound obtained by reading out the character information, for example, a bell sound can be cited.

[0294] [Notification processing for each talker]

[0295] The CPU 11a can also generate control information that causes the notification manner at the time of notifying the surrounding person 42 of the notification situation to be different. More specifically, the CPU 11a can generate control information that causes the notification manner to be different for each talker.

[0296] In the present embodiment, the object person 41 is notified of the talking content 48 of the person who talks in the surrounding person 42, that is, the talker. In the present embodiment, the talking content 48 is notified of the object person 41 for each talker.

[0297] In the present embodiment, the CPU 11a determines the notification situation for each talker. The CPU 11a determines the notification situation of the talking content 48 of the talker for each talker.

[0298] The CPU 11a causes the notification manner at the time of notifying the surrounding person 42 of the notification situation determined for each talker to be different for each talker. The CPU 11a generates control information that causes the notification manner to be different for each talker. ​​

[0299] Specifically, the CPU 11a generates, for example, control information that causes the emission color in the above-described light emitting device 500 to be different for each speaker. The light emitting device 500 is a device for notifying the notification situation.

[0300] The CPU 11a generates control information that causes the emission color in the light emitting device 500 used in the notification of the notification situation to be different. Specifically, the CPU 11a generates control information that causes the emission color to be different for each speaker.

[0301] In this case, the color of the light emitted from the light emitting device 500 is changed every time the speaker changes.

[0302] In this case, the surrounding person 42 is able to determine which person's speech content 48 the speech content 48 notified to the object person 41 is. Also, in this case, the surrounding person 42 is able to determine that the notification made to himself or herself is a notification about which speaker.

[0303] In addition, the change in the notification method can be made in a manner other than a manner in which the emission color is made different.

[0304] For example, the emission pattern of the light emitting device 500 can be made different for each speaker.

[0305] Also, furthermore, the sound output from the sound source 93 can also be made different for each speaker.

[0306] Also, furthermore, for example, the vibration pattern of the vibration source can also be made different for each speaker.

[0307] Also, it can be configured that every time the speaker is different, new information about the speaker is displayed on the object person reference device or a separate device.

[0308] 〔Notification related to speech of registered surrounding person〕

[0309] In the present embodiment, the object person 41 is notified of the speech of the registered surrounding person 42 who is a surrounding person 42 registered in advance. In the present embodiment, the object person 41 is not notified of the speech of a person other than the registered surrounding person 42, i.e., a non-registered person.

[0310] In the present embodiment, the non-registered person is not registered in the database, and the non-registered person does not have a separate microphone 600.

[0311] In addition, a case in which the voice information of the non-registered person is acquired by the separate microphone 600 of the registered surrounding person 42 is also conceivable. In this case, the sound pressure of the voice information of the non-registered person becomes small, and the voice information is not transmitted to the management server 300.

[0312] It is also conceivable that the voice information of the non-registered person is acquired by the individual microphone 600 of the registered surrounding person 42. However, in this case, the voice information is not transmitted to the management server 300.

[0313] In this case, the CPU 11a determines the notification condition of the notification related to the speech of the registered surrounding person 42 as the notification condition.

[0314] The CPU 11a determines the notification condition of the notification related to the speech of the registered surrounding person 42 and the notification to the object person 41.

[0315] The notification regarding the speech of the non-registered person is not made to the object person 41. In this case, the CPU 11a does not determine the notification condition regarding the speech of the non-registered person as the notification condition.

[0316] In this case, the CPU 11a generates the control information that causes the notification of the notification condition related to the speech of the registered surrounding person 42. The CPU 11a generates the control information that causes the notification to the surrounding person 42 of the notification condition regarding the registered surrounding person 42.

[0317] The CPU 11a does not generate the control information that causes the notification of the notification condition regarding the notification related to the speech of the non-registered person. The CPU 11a does not generate the control information that causes the notification to the surrounding person 42 of the notification condition regarding the non-registered person.

[0318] 〔Others〕

[0319] The above notifications can be combined.

[0320] In the above, the case where the light emission of the light emitting device 500, the vibration of the vibration source, the output of the sound from the sound source 93, and the display of the speech permission information are individually performed is described. In the above, the case where the plurality of types of notifications are individually performed is described.

[0321] The notification method is not limited thereto. Two or more of the plurality of types of notifications can be made to the surrounding person 42.

[0322] Furthermore, the above notification to the surrounding person 42 can be made via the device 200 worn by the object person 41. For example, the notification to the surrounding person 42 can be made by making the output of the light source (not shown) provided to the outer surface of the device 200 different.

[0323] Furthermore, in the above, the case where the start of the notification of the speech content 48 or the end of the notification of the speech content 48 is notified to the surrounding person 42 is described.

[0324] Further, the surrounding person 42 can be notified of a situation in which the notification itself cannot be performed, a situation in which the quality of the notification is degraded, and the like. In other words, the surrounding person 42 can be notified of a situation in which an adverse situation has occurred in the notification to the target person 41.

[0325] A situation in which the notification itself cannot be performed, a situation in which the quality of the notification is degraded, and the like can occur due to a failure of the device 200 worn by the target person 41 and the like.

[0326] The surrounding person 42 can be notified of a situation in which the notification itself cannot be performed, a situation in which the quality of the notification is degraded, and the like. In other words, the surrounding person 42 can be notified of a situation in which an adverse situation has occurred in the notification to the target person 41.

[0327] A situation in which an adverse situation has occurred in the notification to the target person 41 is assumed to be notified to the surrounding person 42.

[0328] In this case, the surrounding person 42 is less likely to speak in a situation in which an adverse situation has occurred in the notification to the target person 41.

[0329] When a situation in which an adverse situation has occurred in the notification to the target person 41 is notified, the light emitting device 500 is caused to emit light, as described above. Alternatively, the vibration of the vibration source, the output of the sound from the sound source 93, the display of information, and the like are performed.

[0330] Specifically, for example, the light emitting device 500 is caused to blink. Also, for example, the light emitting device 500 is caused to emit light in a predetermined specific color such as red.

[0331] Also, for example, the vibration of the vibration source is made larger than in a situation in which the adverse situation has not occurred.

[0332] Also, for example, a sound indicating that the adverse situation has occurred is output from the sound source 93.

[0333] Also, for example, information indicating that the adverse situation has occurred is displayed on the display portion 217 of the device 200.

[0334] Further, as described above, the surrounding person 42 can be notified of two or more kinds of information.

[0335] Further, in the above, the determination of the notification situation and the generation of the control information are described as being performed by the management server 300 as an example of an information processing system.

[0336] The processing of the determination of the notification situation and the generation of the control information can be implemented by one device such as one management server 300. Also, the processing of the determination of the notification situation and the generation of the control information can be implemented by a plurality of devices.

[0337] The information processing system can be constituted by a single device or by a plurality of devices.

[0338] (Notes) (1)

[0340] An information processing system including a processor,

[0341] The processor performs processing of:

[0342] determining a notification condition, which is a notification related to a speech of a person located in a periphery of a target person, i.e., a peripheral person, and which is a condition of the notification to the target person; and

[0343] generating control information that causes the notification of the determined notification condition to the peripheral person. (2)

[0345] The information processing system according to (1), wherein

[0346] The processor performs processing of:

[0347] determining a progress condition of the notification to the target person as the notification condition; and

[0348] generating, as the control information, control information that causes the notification of the progress condition to the peripheral person. (3)

[0350] The information processing system according to (2), wherein

[0351] The processor performs processing of:

[0352] determining a display processing of a content of a speech of the peripheral person, i.e., a speech content, and a progress condition of the display processing in a target person reference device, which is a device referred to by the target person, as the progress condition; and

[0353] generating, as the control information, control information that causes the notification of the progress condition of the display processing in the target person reference device to the peripheral person. (4)

[0355] The information processing system according to (3), wherein

[0356] The processor generates, as the control information, control information that causes the notification of the display processing in the target person reference device having ended to the peripheral person. (5)

[0358] The information processing system according to any one of (1) to (4), wherein

[0359] The processor generates, as the control information, control information that causes an output in a peripheral person reference device, which is a device referred to by the peripheral person, to change in accordance with the notification situation. (6)

[0361] The information processing system according to (5), in which

[0362] The processor generates, as the control information, control information that causes an output in the peripheral person reference device to change with an end of the notification to the target person. (7)

[0364] The information processing system according to (6), in which

[0365] The processor generates, as the control information, control information that causes a luminance of at least a part of the peripheral person reference device to change with an end of the notification to the target person. (8)

[0367] The information processing system according to (5) or (6), in which

[0368] The target person is notified of a content of speech, which is a content of speech of the peripheral person, and a notification process of the content of speech to the target person is performed,

[0369] The processor further generates control information that causes information indicating that speech is possible to be output by the peripheral person reference device with an end of the notification process. (9)

[0371] The information processing system according to any one of (1) to (8), in which

[0372] The processor generates, as the control information, control information that causes a sound output from a sound source to change in accordance with a situation of the notification to the target person. (10)

[0374] The information processing system according to (9), in which

[0375] The processor generates, as the control information, control information that causes no sound to be emitted from the sound source with an end of the notification to the target person, or generates control information that causes a sound to be emitted from the sound source with the end of the notification. (11)

[0377] The information processing system according to (9), in which

[0378] a speech content as a content of speech of the person in the periphery is notified to the object, and a notification process of the speech content to the object is performed,

[0379] The processor further generates control information that causes a sound corresponding to the information indicating that speech is possible to be output from the sound source as the notification process ends. (12)

[0381] The information processing system according to any one of (1) to (11), in which

[0382] The processor generates, as the control information, control information that causes the notification situation to be notified to each of the persons in the periphery by a separate device that is a device possessed by each of the persons in the periphery. (13)

[0384] The information processing system according to (12), in which

[0385] The processor generates, as the control information, control information that causes a state of vibration of the separate device to change according to the situation of the notification to the object. (14)

[0387] The information processing system according to (12) or (13), in which

[0388] The processor generates, as the control information, control information that causes display content in the separate device to change according to the situation of the notification to the object. (15)

[0390] The information processing system according to (14), in which

[0391] a speech content as a content of speech of the person in the periphery is notified to the object, and a notification process of the speech content to the object is performed,

[0392] The processor further generates control information that causes an information indicating that speech is possible to be displayed on the separate device as the notification process ends. (16)

[0394] The information processing system according to any one of (1) to (15), in which

[0395] a speech content as a content of speech of a person who has spoken, that is, a speaker, among the persons in the periphery is notified to the object,

[0396] The processor performs the following process:

[0397] determining, as the notification condition, a notification condition regarding the speech content of each of the speakers as the notification condition; and

[0398] control information is also generated that causes the notification manner when the notification condition determined for each of the speakers is notified to the surrounding people to differ for each of the speakers. (17)

[0400] The information processing system according to any one of (1) to (16), in which

[0401] the object person is notified of the speech of a pre-registered surrounding person, that is, a registered surrounding person, among the surrounding people, and is not notified of the speech of a person other than the registered surrounding person, that is, a non-registered person,

[0402] the processor performs the following processing:

[0403] determining, as the notification condition, a notification condition of the notification regarding the speech of the registered surrounding person and the notification of the object person,

[0404] control information is generated that causes the notification condition of the notification regarding the speech of the registered surrounding person to be notified to the surrounding people as the control information. (18)

[0406] A program for causing a computer to implement functions including:

[0407] determining a notification condition of a notification regarding the speech of a surrounding person located in the vicinity of an object person and a condition of the notification of the object person; and

[0408] generating control information that causes the determined notification condition to be notified to the surrounding people.

[0409] The information processing system according to (1) enables the surrounding people to know the condition of the notification of the object person regarding the speech of the surrounding person located in the vicinity of the object person.

[0410] The information processing system according to (2) enables the surrounding people to be notified of the condition of the notification of the object person.

[0411] The information processing system according to (3) enables the surrounding people to be notified of the condition of the display processing in the object person reference device referred to by the object person.

[0412] The information processing system according to (4) enables the surrounding people to be notified that the display processing in the object person reference device has ended.

[0413] According to the information processing system according to (5), it is possible to change the output in the surrounding person reference device that is a device referred to by the surrounding person, in accordance with the notification situation.

[0414] According to the information processing system according to (6), it is possible to change the output in the surrounding person reference device as the notification to the object person ends.

[0415] According to the information processing system according to (7), it is possible to notify the surrounding person that the notification to the object person has ended, using a change in the brightness of at least a part of the surrounding person reference device.

[0416] According to the information processing system according to (8), it is possible to notify the surrounding person that it is possible to speak.

[0417] According to the information processing system according to (9), it is possible to notify the surrounding person of the situation of the notification to the object person, using sound.

[0418] According to the information processing system according to (10), it is possible to notify the surrounding person that the notification to the object person has ended, using sound.

[0419] According to the information processing system according to (11), it is possible to notify the surrounding person that it is possible to speak.

[0420] According to the information processing system according to (12), it is possible to individually notify each surrounding person of the situation of the notification to the object person.

[0421] According to the information processing system according to (13), it is possible to notify the surrounding person of the situation of the notification to the object person, using vibration of an individual device.

[0422] According to the information processing system according to (14), it is possible to notify the surrounding person of the situation of the notification to the object person, by display in an individual device.

[0423] According to the information processing system according to (15), it is possible to notify the surrounding person that it is possible to speak, by display in an individual device.

[0424] According to the information processing system according to (16), it is possible to notify the surrounding person that a notification is being made to the object person about which of the surrounding persons the notification is about.

[0425] According to the information processing system according to (17), it is possible to notify the surrounding person of the situation of a notification related to the speaking of a pre-registered surrounding person.

[0426] According to the program according to (18), it is possible to enable a surrounding person to know a notification related to the speaking of a surrounding person located in the vicinity of an object person and the situation of the notification to the object person.

[0427] The embodiments of the present application described above are provided for the purpose of illustration and description. It is apparent to a person skilled in the art that various modifications and changes can be made to the embodiments without departing from the scope of the present application. The present application is not limited to the embodiments disclosed, and the embodiments of the present application do not comprehensively and exhaustively include the present application, and the present application is not limited to the disclosed modes. It is obvious to a person skilled in the art that various modifications and changes are self-evident. The present embodiments are selected and described in order to most easily explain the principles of the present application and its application. Thus, other skilled persons in the art can understand the present application by various modifications to the specific use assumed to be optimized for various embodiments. The scope of the present application is defined by the above claims and their equivalents.

Claims

1. An information processing system including a processor, the processor performs processing of: determining a notification condition, the notification condition being a notification related to a speech of a person located in a periphery of a target person, i.e., a peripheral person, and being a condition of the notification to the target person; and generating control information that causes the notification of the determined notification condition to the peripheral person.

2. The information processing system according to claim 1, wherein the processor performs processing of: determining a progress condition of the notification to the target person as the notification condition; and generating control information that causes the notification of the progress condition to the peripheral person as the control information.

3. The information processing system according to claim 2, wherein the processor performs processing of: determining a display processing of a content of the speech, i.e., a speech content, and a progress condition of the display processing in a target person reference device, which is a device referenced by the target person, as the progress condition; and generating control information that causes the notification of the progress condition of the display processing in the target person reference device to the peripheral person as the control information.

4. The information processing system according to claim 3, wherein the processor generates control information that causes the notification of the end of the display processing in the target person reference device to the peripheral person as the control information.

5. The information processing system according to any one of claims 1 to 4, wherein the processor generates control information that causes an output in a peripheral person reference device, which is a device referenced by the peripheral person, to change according to the notification condition as the control information.

6. The information processing system according to claim 5, wherein the processor generates control information that causes the output in the peripheral person reference device to change with the end of the notification to the target person as the control information.

7. The information processing system according to claim 6, wherein the processor generates control information that causes a luminance of at least a part of the peripheral person reference device to change with the end of the notification to the target person as the control information.

8. The information processing system according to claim 5 or 6, wherein a notification processing of a speech content, which is a content of the speech of the peripheral person, to the target person is performed, the processor further generates control information that causes information indicating that the speech is possible to be output by the peripheral person reference device with the end of the notification processing.

9. The information processing system according to any one of claims 1 to 8, wherein the processor generates control information that causes a sound output from a sound source to change according to a condition of the notification to the target person as the control information.

10. The information processing system according to claim 9, wherein the processor generates control information that causes no sound to be emitted from the sound source with the end of the notification to the target person, or generates control information that causes a sound to be emitted from the sound source with the end of the notification as the control information. ​ 11. The information processing system according to claim 9, wherein the notification processing of the talk content to the object person is performed, the processor further generates control information that causes a sound corresponding to information indicating that it is possible to talk to be output from the sound source as the notification processing ends.

12. The information processing system according to any one of claims 1 to 11, wherein the processor generates, as the control information, control information that causes the notification situation to be notified to each of the surrounding persons by individual devices that are each a device possessed by the surrounding persons.

13. The information processing system according to claim 12, wherein the processor generates, as the control information, control information that causes a state of vibration of the individual devices to change according to the situation of the notification to the object person.

14. The information processing system according to claim 12 or 13, wherein the processor generates, as the control information, control information that causes display content in the individual devices to change according to the situation of the notification to the object person.

15. The information processing system according to claim 14, wherein the notification processing of the talk content to the object person is performed, the processor further generates control information that causes information indicating that it is possible to talk to be displayed on the individual devices as the notification processing ends.

16. The information processing system according to any one of claims 1 to 15, wherein the notification processing of the talk content to the object person is performed, the processor performs the following processing: determines, as the notification situation, a notification situation of the talk content to the object person for each of the talkers; and further generates control information that causes the notification situation determined for each of the talkers to be notified to the surrounding persons in a notification manner that differs for each of the talkers.

17. The information processing system according to any one of claims 1 to 16, wherein the notification to the object person is performed in relation to talk of a pre-registered surrounding person, i.e., a registered surrounding person, among the surrounding persons, and the notification to the object person is not performed in relation to talk of a person other than the registered surrounding person, i.e., a non-registered person, the processor performs the following processing: determines, as the notification situation, a situation of the notification to the object person in relation to talk of the registered surrounding person; and generates, as the control information, control information that causes the notification situation of the notification in relation to talk of the registered surrounding person to be notified to the surrounding persons.

18. A program article that records a program for causing a computer to implement functions including: a determination function that determines a notification situation, the notification situation being a notification in relation to talk of a person, i.e., a surrounding person, located in a periphery of an object person, and being a situation of the notification to the object person; and a notification function that performs the notification to the object person in relation to the talk of the surrounding person. generating a control information that causes the notification of the determined notification situation to the person in the periphery.

19. An information processing method comprising steps of: determining a notification situation, the notification situation being a notification related to a speech of a person in a periphery of a target person, i.e., the person in the periphery, and being a situation of the notification to the target person; and generating a control information that causes the notification of the determined notification situation to the person in the periphery.

Images