Notification system, notification method, and notification device

Abstract

The present invention provides a notification device comprising: a notification unit that notifies people in the vicinity of the output status of acquired sound, which is sound acquired by the acquisition unit from people in the vicinity, to the user by the output unit; and a control unit that causes the notification unit to send notifications according to the output status of the acquired sound by the output unit. [Solution] The notification device according to one embodiment is characterized by comprising: a notification unit that notifies people in the vicinity of the output status of acquired sound, which is sound acquired by the acquisition unit from people in the vicinity, to the user by the output unit; and a control unit that causes the notification unit to issue a notification according to the output status of the acquired sound by the output unit.

Description

Technical Field

[0001] The present invention relates to a notification system, a notification method, and a notification device.

Background Art

[0002] Conventionally, there is known a technique that enables a user to hear ambient sounds while wearing a voice output device such as headphones or earphones. In such a technique, there is known a portable music playback device provided with a notification means for notifying from headphones that an external sound matches a predetermined phrase when the external sound matches the predetermined phrase (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the external sound matches the predetermined phrase, the music playback device described in Patent Document 1 notifies from the headphones that the match has occurred. A user using the music playback device can hear the external sound without removing the headphones. However, when the external sound is a voice directed from a person around to the user, a person around cannot recognize whether the external sound is being output to the user (that is, whether the user is listening). An object of the present invention is to provide a notification device that notifies a person around of the output status of the acquired external sound to the user.

Means for Solving the Problems

[0005] To solve the above-mentioned problems, a notification device according to one embodiment is characterized by comprising a notification unit that notifies the surrounding people of the output status of acquired sound, which is sound acquired by the acquisition unit from surrounding people, by the output unit, and a control unit that causes the notification unit to provide notifications according to the output status of the acquired sound by the output unit.

[0006] Furthermore, a notification system according to one embodiment is characterized by comprising a notification unit that notifies the surrounding people of the output status of acquired sound, which is sound acquired by the acquisition unit from surrounding people, by the output unit, and a control unit that causes the notification unit to send a notification according to the output status of the acquired sound by the output unit.

[0007] Furthermore, one embodiment of the notification method is a notification method in which a notification unit notifies the surrounding people of the output status of the sound acquired by the sound acquisition output unit, which is sound acquired from the surrounding people, to the user. The notification unit provides notification according to the output status of the sound acquired by the output unit. It is characterized by including. [Effects of the Invention]

[0008] According to one embodiment, a notification device, notification system, or notification method can be provided that notifies people in the surrounding area of ​​the status of sound output to the user, which has been acquired by the acquisition unit from people in the surrounding area. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram illustrating the schematic configuration of the notification system 1 in Example 1. [Figure 2] This is a flowchart illustrating the processing flow of notification system 1. [Figure 3] This is a diagram illustrating the schematic configuration of the notification system 2 in Example 2. [Figure 4] This is a diagram illustrating the schematic configuration of the notification system 3 in Example 3. [Figure 5] This is a diagram illustrating the schematic configuration of the notification system 4 in Example 4. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described below with reference to the drawings. In each figure, the same reference numerals indicate components having the same or equivalent function. The configurations, numerical values, processing flow, functions, and elements described in the following embodiments are merely examples, and they can be freely modified or changed, and the technical scope of the present invention is not intended to be limited to the following description.

[0011] <Example 1> Figure 1 is a diagram illustrating the schematic configuration of the notification system 1 in Example 1.

[0012] The notification system 1 includes a notification device 100 and a control device 200. The notification device 100 and the control device 200 may be configured as separate devices or as a single device.

[0013] The notification device 100 is a device that outputs sound to the user. The notification device 100 is, for example, an earphone. The notification device 100 may also be a bone conduction earphone. The notification device 100 is not limited to an earphone, but may also be headphones, a headset, smart glasses, VR goggles, a hearing aid, etc. The notification device 100 may have a noise cancellation function and may also have an ambient sound pass-through function. The noise cancellation function is a function that cancels out and reduces noise generated around the user so that the user does not hear it. The ambient sound pass-through function is a function that outputs ambient sounds collected by a microphone or the like through a speaker to the user to hear.

[0014] The general configuration of the notification device 100 will now be described. The notification device 100 includes an acquisition unit 110, an output unit 120, a notification unit 130, a first communication unit 140, and a first control unit 150.

[0015] The acquisition unit 110 is, for example, a microphone. The acquisition unit 110 acquires ambient sound and converts it into an audio signal. In this specification, the sound acquired by the acquisition unit 110 may be referred to as acquired sound. The acquired sound includes, for example, the sound uttered by people around the user.

[0016] The acquisition unit 110 may be provided outside the notification device 100 and configured to be communicable with one or both of the notification device 100 and the control device 200 by wire or wirelessly. The acquisition unit 110 may be provided in the control device 200.

[0017] The output unit 120 is, for example, a speaker and outputs sound to the user.

[0018] The notification unit 130 may be configured to include at least one of a display unit and a light emitting unit. The notification unit 130 outputs a notification to people around the notification device 100. That is, the notification unit 130 outputs a notification to people around the user wearing the notification device 100.

[0019] The notification unit 130 may be provided outside the notification device 100 and configured to be communicable with one or both of the notification device 100 and the control device 200 by wire or wirelessly. The notification unit 130 may be provided in the control device 200. When the notification unit 130 is provided in the control device 200, the notification unit 130 can output a notification to people around the control device 200.

[0020] The acquisition unit 110 and the notification unit 130 may be provided in the same device outside the notification device 100.

[0021] The display unit is, for example, a display disposed on the surface of the notification device 100. The display unit displays an image or video and notifies people around the user.

[0022] The light emitting unit may be, for example, an LED (Light Emitting Diode) or the like disposed on the surface of the notification device 100. The light emitting unit emits light and notifies people around the user.

[0023] The first communication unit 140 is, for example, a communication module that supports any communication standard. The communication standard may be, for example, a wired communication standard or a short-range wireless communication standard including Bluetooth®, infrared, and NFC. The first communication unit 140 communicates with the connected control device 200.

[0024] For example, the notification device 100 can transmit the sound signal of the acquired sound to the control device 200 via the first communication unit 140. The notification device 100 can receive various information, including the sound signal, from the control device 200 via the first communication unit 140.

[0025] The first control unit 150 controls the acquisition unit 110, the output unit 120, the notification unit 130, and the first communication unit 140. The first control unit 150 may, for example, control the output unit 120 to output sound from the output unit 120.

[0026] Next, the general configuration of the control device 200 will be described.

[0027] The control device 200 may be a terminal capable of controlling the notification device 100. The control device 200 may be, for example, a smartphone, a smartwatch, smart glasses, etc.

[0028] The control device 200 may include a second communication unit 260, an input unit 270, a storage unit 280, and a second control unit 290. One or more components included in the control device 200 are included in the notification device 100. The control device 200 may be configured as follows: One or more components included in the control device 200 may be located outside the control device 200 and configured to be connectable to at least one of the control device 200 and the notification device 100 by wire or wireless connection. For example, the storage unit 280 may be located on a remote server configured to communicate with the control device 200 via a network system or the like.

[0029] The second communication unit 260 is, for example, a communication module that corresponds to the same communication standard as the first communication unit 140. The second communication unit 260 is used for communication between the notification device 100 and the control device 200. By connecting the second communication unit 260 and the first communication unit 140, communication can be performed between the notification device 100 and the control device 200.

[0030] For example, the control device 200 can receive the sound signal of the acquired sound transmitted from the notification device 100 via the second communication unit 260. The control device 200 can transmit the sound signal to the notification device 100 via the second communication unit 260.

[0031] The input unit 270 includes an input interface capable of receiving input from the user. The input unit 270 may include, for example, one or more buttons, a touch panel, etc. The input unit 270 may be used by the user to select a process to be executed by the notification device 100 or the control device 200.

[0032] The storage unit 280 is a storage medium including ROM (Read Only Memory) or RAM (Random Access Memory), etc. The storage unit 280 may store a program executed by the notification system 1. The program may include a program executed by the notification system 1.

[0033] The memory unit 280 includes a storage unit 281 and a holding unit 282.

[0034] The storage unit 281 stores the sound signal of the acquired sound as sound data. The storage unit 281 may store the sound from the start to the end of the ambient sound generation as a single sound data. Sound detection The start time of ambient sound generation may be defined as the time when sound generation (or sound at or above the predetermined volume) is detected after a set period of time has elapsed since the start of ambient sound generation (in other words, when no ambient sound is being generated). After the start time of ambient sound generation has been identified (in other words, after ambient sound generation has occurred), if a period of time has elapsed since the start of ambient sound generation has elapsed since the start of ambient sound generation, that point in time may be defined as the end time of ambient sound generation.

[0035] The storage unit 281 may store one or more sound data sets, each associated with a unique ID. The sound data may consist of sounds from a single sound source. The storage unit 281 may temporarily store the sound data and, after a predetermined time has elapsed, delete the oldest sound data sequentially. The storage unit 281 may be configured, for example, as a ring buffer. The storage unit 281 may sequentially store the sound data in chronological order. When the sound data stored in the ring buffer exceeds the capacity of the ring buffer, the storage unit 281 may sequentially delete the oldest sound data.

[0036] The holding unit 282 acquires and holds the sound data stored in the storage unit 281. The holding unit 282 may, for example, store the sound characteristics recognized by the second control unit 290, the sound source, the words contained in the sound, and a summary of the sound, associating them with the sound data.

[0037] The sound data acquired by the holding unit 282 from the storage unit 281 may be deleted from the storage unit 281. The holding unit 282 may retain the sound data stored in the storage unit 281 from the start time to the end time of sound generation. The holding unit 282 may retain a portion of the sound data from the start time to the end time of sound generation. The holding unit 282 may associate the IDs stored in the storage unit 281 with the sound data.

[0038] The second control unit 290 is configured to include at least one processor, at least one dedicated circuit, or a combination thereof. The processor may be a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA or ASIC. The second control unit 290 executes processes related to the operation of the notification system 1 while controlling each component of the control device 200.

[0039] The second control unit 290 includes an output control unit 291 and a notification control unit 292.

[0040] Each component included in the second control unit 290 may consist of hardware, software, or both. The second control unit 290 may control each component included in the control device 200. The second control unit 290 can cause the control device 200 to perform various processes in response to user input from the input unit 270.

[0041] The memory unit 280 stores setting conditions. Setting conditions are conditions that indicate sounds with features of high importance to the user. Setting conditions may be conditions set by the user. Sounds that satisfy the setting conditions may be sounds that the user does not want to miss. The memory unit 280 may store at least one condition as a setting condition. For example, the memory unit 280 may store as a setting condition that the sound contains a pre-registered word. The memory unit 280 may store as a setting condition that the sound comes from a pre-registered sound source. Pre-registered words may be, for example, the user's name or a greeting ("Excuse me," etc.). Pre-registered sound sources may be, for example, a specific person, a traffic light, a telephone, a car, a visiting chime. It may be an announcement. The memory unit 280 may also store as a setting condition that the sound is a repeated identical sound. A repeated identical sound may be, for example, the sound of a car horn, such as "beep beep". A repeated identical sound may be, for example, the sound of spoken language, such as "Mr. A, Mr. A". The interval between repeated sounds may be set to a predetermined time. In this specification, an acquired sound that satisfies the setting condition may be referred to as the first sound.

[0042] The memory unit 280 may store the importance level in association with the set conditions. Table 1 shows the set conditions and This table illustrates the correspondence between importance levels. Sounds that the user wants to hear in real time or near real time may be considered high-importance sounds. Sounds that the user does not need to hear in real time may be considered low-importance sounds. For example, sounds that the user does not need to hear in real time may be sounds that the user wants to record to listen to later. In Table 1, the memory unit 280 stores a high importance level associated with the setting condition that the sound contains the user's name. The memory unit 280 also stores a low importance level associated with the setting condition that the sound is from person A. Importance levels can be set to any number of levels, not just "high" and "low".

[0043] [Table 1]

[0044] The output control unit 291 determines whether the acquired sound includes the first tone.

[0045] If the output control unit 291 determines that the acquired sound includes a first tone, it acquires the importance level corresponding to that first tone based on the set conditions. If the output control unit 291 determines that the acquired sound includes a first tone with a high importance level, it causes the output unit 120 to output the acquired sound that satisfies the set conditions set in advance by the user. The output control unit 291 causes the output unit 120 to output the acquired sound in real time or near real time. Outputting to the output unit 120 in real time or near real time means that the acquisition unit 110 acquires ambient sound and outputs it immediately (for example, within 0.5 seconds). If the output unit 120 is outputting sound, the output control unit 291 may output the acquired sound after the output of the currently outputting sound has finished. The output control unit 291 may output the acquired sound after the user is able to hear the acquired sound. The output control unit 291 may output the acquired sound at a time that is delayed from real time. The output control unit 291 may change the output speed of the acquired sound before outputting it to the output unit 120. The output control unit 291 may adjust the volume of the acquired sound and output it to the output unit 120.

[0046] If the output control unit 291 determines that the acquired sound includes a first sound of low importance, it may hold the output of the acquired sound by the output unit 120 and have the acquired sound held by the holding unit 282.

[0047] If the output control unit 291 determines that the acquired sound does not contain the first tone, it will not output the acquired sound to the output unit 120. At a predetermined timing after the holding unit 282 has acquired the acquired sound, the output control unit 291 will output the acquired sound held by the holding unit 282. The timing may be such that the user is able to hear the acquired sound. For example, the predetermined timing may be when there is input from the user to the input unit 270.

[0048] The notification control unit 292 causes a notification to be sent to people in the vicinity if the acquired sound includes the first tone. The notification control unit 292 causes the notification unit 130 to send a notification according to the output status of the acquired sound by the output unit 120. The notification control unit 292 causes a notification to be sent to people in the vicinity based on the output status of the acquired sound including the first tone by the output unit 120. The notification unit 130 is controlled by the notification control unit 292 and can notify people in the vicinity of the output status of the acquired sound, which is sound acquired by the acquisition unit 110 from people in the vicinity, by the output unit 120.

[0049] Table 2 is used to explain the correspondence between the output status of acquired sounds, including the first tone, and the notification method. Table 2 is an example table illustrating the correspondence between the output status of acquired sounds, including the first tone, and the notification method. The notification control unit 292 causes the notification unit 130 to notify people in the vicinity when the output control unit 291 determines that "the acquired sound is a sound that includes the first tone, which is of high importance" (i.e., when the acquired sound is output in near real-time). The notification control unit 292 causes the notification unit 130 to notify people in the vicinity when the output control unit 291 determines that "the acquired sound is a sound that includes the first tone, which is of low importance" (i.e., when the output unit 120 of the acquired sound is withheld from outputting).

[0050] [Table 2]

[0051] If the output control unit 291 determines that "the acquired sound includes a first sound of high importance, and the acquired sound should be output", the notification control unit 292 may cause different notifications to be sent before, during, and after the output unit 120 outputs the acquired sound. Specifically, before the output unit 120 outputs the first sound and after the acquisition unit 110 has acquired the first sound, the notification control unit 292 may cause the notification unit 130 to send a first notification. After the decision to have the output unit 120 output the acquired sound and before the output of the acquired sound begins, the notification control unit 292 may cause the notification unit 130 to send a first notification. While the output unit 120 is outputting the first sound, the notification control unit 292 may cause the notification unit 130 to send a second notification. After the output unit 120 has finished outputting the first sound, the notification control unit 292 may cause the notification unit 130 to send a third notification. The notification control unit 292 may issue a third notification immediately before the output unit 120 finishes outputting the first sound. The notification control unit 292 may issue a third notification after the output unit 120 has output the acquired sound, but before a predetermined time has elapsed since the output was terminated. It is desirable that the first, second, and third notifications be of different forms. The notification for the first sound may be, for example, a red light flashing. The notification for the second sound may be a flashing red light. The notification for the third sound may be, for example, a green light flashing. Instead of the first to third notifications, the notification control unit 292 may display a progress bar indicating the progress of the output of the acquired sound, a counter indicating the time until output is completed, etc., on the display unit. It could also be used as a notification. In this case, people around the user can speak to the user, and then continue speaking only after confirming that the first sound has finished being output to the user.

[0052] If the output control unit 291 determines that the acquired sound includes a first sound of low importance and the output unit 120 of the acquired sound is withheld, the notification control unit 292 causes the notification unit 130 to issue a fourth notification. The form of the fourth notification may be different from the forms of the first to third notifications. The form of the fourth notification may be, for example, an orange light emission.

[0053] If the output control unit 291 holds the output of the acquired sound by the output unit 120, the second control unit 290 may have the sound data of the acquired sound, including the first sound, held in the holding unit 282. By having the holding unit 282 hold the sound data of the acquired sound, the user can output the acquired sound at any timing of their choosing and listen to it again. When the holding unit 282 has finished holding the sound data including the acquired sound, the notification control unit 292 may have the notification unit 130 issue a fifth notification, which is a notification indicating that the saving of the acquired sound has been completed. The form of the fifth notification may be different from the form of the first to fourth notifications. The form of the fifth notification may be, for example, the orange light turning off. In this case, people around the user can confirm that the content of the utterance they spoke to the user has been saved.

[0054] If the acquired sound does not include the first tone, the notification control unit 292 causes the notification unit 130 to issue a sixth notification. The notification control unit 292 may also choose not to issue a notification to people in the vicinity if the acquired sound does not include the first tone. The form of the sixth notification may be different from the forms of the first to fifth notifications.

[0055] If the notification unit 130 is a light-emitting unit, the notification control unit 292 may distinguish between the first to sixth notification forms by varying the form of light emission. The form of light emission may be the color of the light emission or the interval between flashes.

[0056] If the notification unit 130 is a display unit, the notification control unit 292 may distinguish between the first to sixth notifications by varying the content and form of the display. The display unit may distinguish between the first to sixth notifications by using a display that shows the output status of the first sound, such as a diagram or text.

[0057] Table 2 illustrates the correspondence between the output status of acquired sounds and the notification method. However, the correspondence is not limited to that shown in Table 1, and various modifications and changes are possible within the scope of its essence.

[0058] Next, we will explain the output method, which is the processing flow of the notification system 11, using Figure 2. Figure 2 is a flowchart for explaining the processing flow of the notification system 11.

[0059] The acquisition unit 110 acquires the sound, which is the ambient sound (Step 1).

[0060] The storage unit 281 stores the acquired sound as sound data (step 2).

[0061] The output control unit 291 determines whether or not the sound data stored by the storage unit 281 includes the first sound (step 3).

[0062] In step 3, if the output control unit 291 determines that the sound data contains a first tone, the output control unit 291 determines whether or not to output the first tone to the output unit 120 (step 4). The output control unit 291 determines the importance of the first tone. By determining the importance of the first tone, it is possible to determine whether or not to output the first tone.

[0063] In step 4, the output control unit 291 determines that it will output the first tone to the output unit 120. If this occurs, the notification control unit 292 causes the notification unit 130 to send the first notification (step 5).

[0064] After the notification control unit 292 causes the notification unit 130 to send the first notification, the output control unit 291 causes the output unit 120 to start outputting the first sound (step 6).

[0065] After the output control unit 291 starts outputting the first tone, that is, while the output control unit 291 is outputting the first tone, the notification control unit 292 causes the notification unit 130 to send a second notification (step 7).

[0066] After the output control unit 291 has finished outputting the first sound, the notification control unit 292 instructs the notification unit 130 to stop sending the second notification (step 8).

[0067] The notification control unit 292 instructs the notification unit 130 to stop the second notification and then to send the third notification (step 9).

[0068] In step 4, if the output control unit 291 determines that the output of the first sound by the output unit 120 should be suspended, the notification control unit 292 causes the notification unit 130 to send a fourth notification (step 10).

[0069] After the notification control unit 292 issues a fourth notification, the control unit has the sound data stored in the storage unit 281 held in the holding unit 282 (step 11). The sound data held in the holding unit 282 may be sound data that includes the first sound.

[0070] After the holding unit 282 has finished holding the sound data including the first, the notification control unit 292 causes the notification unit 130 to notify the fifth notification (step 12).

[0071] In step 3, if the output control unit 291 determines that the sound data does not contain the first sound, the notification control unit 292 causes the notification unit 130 to send a sixth notification.

[0072] In Example 1, an example was shown in which the first to third notifications are output from before the output of the first sound until after the output is completed. However, it is not necessary to output all of the first to third notifications. For example, the notification unit 130 may not give the first notification but give the second and third notifications, or it may not give the second notification but give only the first and third notifications. It may also give only the second notification.

[0073] In Example 1, the notification unit 130 was shown to notify the fourth and fifth notifications before the holding of the first sound was completed, but either the fourth or fifth notification may be notified.

[0074] <Example 2> Figure 3 is a diagram illustrating the schematic configuration of the notification system 2 in Example 2.

[0075] Notification system 2 includes output device 300 and notification device 400.

[0076] The output device 300 is a device that outputs sound to the first user. The output device 300 is, for example, an earphone. The output device 300 may also be a hearing aid. The output device 300 is not limited to earphones, but may also be headphones, a headset, smart glasses, VR goggles, etc.

[0077] The output device 300 may include an output unit 320, a first control unit 350, and a first communication unit 340. Each component included in the output device 300 may operate in the same manner as the configuration of the corresponding embodiment 1.

[0078] One or more components included in the output device 300 may be located outside the output device 300 and configured to be connectable to each other by wire or wireless connection.

[0079] The notification device 400 notifies the second user of the output status of the sound acquired by the output device 300 for the first user. The notification device 400 may be a terminal capable of controlling the output device 300. The notification device 400 may be, for example, a smartphone, a smartwatch, smart glasses, etc.

[0080] The notification device 400 may include an acquisition unit 410, a notification unit 430, a second communication unit 460, an input unit 470, a storage unit 480, and a second control unit 490. Each component included in the notification device 400 may operate in the same manner as the configuration of the corresponding embodiment 1.

[0081] In the notification system 2 of Embodiment 2, the acquisition unit 410 acquires the sound emitted by the second user as the acquired sound. The notification unit 430 notifies the second user of the output status of the acquired sound, which is the sound emitted by the second user, from the output unit 320.

[0082] One or more components included in the notification device 400 may be located outside the notification device 400 and configured to be connectable to each other by wired or wireless means. For example, the notification unit 430 may be located outside the notification device 400 and may be configured to be located within the output device 300. For example, the storage unit 480 may be located on a remote server configured to communicate with the notification device 400 via a network system or the like. For example, an application downloaded to the notification device 400 may control each component included in the notification device 400 and the output device 300.

[0083] In this embodiment, the output device 300 is used by the first user, and the notification device 400 is used by the second user.

[0084] The first user may be someone who has difficulty hearing, such as an elderly person, someone with an auditory processing disorder, or a foreigner. The second user is the person with whom the first user communicates. The second user may be, for example, someone who converses with multiple people, including the first user. For example, the first user may be a student, and the second user may be, for example, a teacher giving a lecture to the student.

[0085] In notification system 2, the voice spoken by the second user is acquired by the notification device 400, output from the output device 300, and heard by the first user. The output status of the voice from the output device 300 can be checked by the notification unit 430 of the notification device 400, so the second user can communicate while checking the sound output status of the output device to the first user.

[0086] The method of using the notification system 2 according to this embodiment will be described in more detail. In this embodiment, the setting condition is set so that the sound data spoken by the second user is stored in the storage unit 482. The first user can listen back to the second user's speech by outputting the sound data stored in the storage unit 482 at any time. When using such a notification system 2, for example, if a teacher (second user) gives a lecture to a student (first user) using the notification device 400, the student can listen back to the teacher's speech from the output device 300 at any time. Therefore, if the student misses part of the teacher's lecture, they can listen back to that part. On the other hand, the teacher can see from the notification unit 430 of the notification device 400 whether the student is asking for repetition, and can take measures such as temporarily stopping speech or adjusting the speed of speech, thereby supporting communication between the two. can.

[0087] <Example 3> Figure 4 is a diagram illustrating the schematic configuration of the notification system 3 in Example 3.

[0088] Notification system 3 includes output device 500, notification device 600, and notification device 700. Output device 500 and notification device 600 are devices used by a third user, and notification device 700 is a device used by a fourth user.

[0089] The output device 500 is a device that outputs sound to a third user, and is, for example, an earphone. The output device 500 is not limited to an earphone, but may be headphones, a headset, smart glasses, VR goggles, etc.

[0090] The output device 500 may include an output unit 520, a first communication unit 540, and a first control unit 550. Each component included in the output device 500 may operate in the same manner as the configuration of the corresponding embodiment 1.

[0091] One or more components included in the output device 500 may be located outside the output device 500 and configured to be connectable to the output device 500 and / or the notification device 600 by wire or wireless connection.

[0092] The notification devices 600 and 700 may be, for example, a personal computer, a smartphone, a smartwatch, smart glasses, etc. The notification devices 600 and 700 may be configured to include acquisition units 610, 710, notification units 630, 730, second communication units 660, 760, input units 670, 770, storage units 680, 780, and second control units 690, 790. Each component included in the notification devices 600 and 700 may operate in the same manner as the configuration of the corresponding embodiment 1. The notification device 600 may be a terminal capable of controlling the output device 500. The second control unit 690 may determine whether to output the sound emitted by the fourth user, acquired by the acquisition unit 710, to the output unit 520. The second control unit 690 may make a determination on the acquired sound in the same manner as in the first embodiment.

[0093] The notification device 600 obtains the output status of the sound acquired by the output device 500 to the third user. The notification device 600 transmits the output status of the sound acquired to the third user to the notification device 700.

[0094] The notification device 700 receives from the notification device 600 the output status of the acquired sound for the third user. The notification device 700 then notifies the fourth user of the output status of the acquired sound for the third user from the output device 500 that it received.

[0095] The notification device 700 may notify the fourth user by displaying a mark on the display indicating the output status of the acquired sound for the third user. For example, the mark may be displayed with a different color or display format depending on the output status of the acquired sound.

[0096] One or more components included in the notification device 600 or notification device 700 may be located outside the notification device and configured to be connectable to each other by wired or wireless means. For example, the notification unit 730 may be located outside the notification device 700. For example, the storage unit 780 may be located on a remote server configured to communicate with the notification device 700 via a network system or the like. For example, applications downloaded to the notification device 600 and notification device 700 may control each component included in the notification system 3.

[0097] An example of how to use the notification system 3 in this embodiment will be described.

[0098] In this embodiment, the output device 500 and the notification device 600 are used by the third user, and the notification device 700 is used by the fourth user. The third user and the fourth user may communicate via the notification devices 600 and 700.

[0099] At some point in the conversation between the third user and the fourth user, the fourth user is the speaker, and the third user is the listener of the fourth user's speech. The acquisition unit 710 of the notification device 700 acquires the sound that is the speech of the fourth user, who is the speaker. The notification device 600 receives the fourth user's speech via the second communication units 660 and 760. The output unit 520 outputs the fourth user's speech to the third user. Through this process, the third user can hear the speech spoken by the fourth user. At this time, the fourth user can check the output status of the output unit 520's speech to the third user via the notification device 700.

[0100] In the notification system 3 of the second embodiment, the fourth user, who is conversing with the third user, can use the notification device to check the output status of their speech to the third user (i.e., whether the sound they spoke is being output to the third user), thereby enabling them to converse efficiently.

[0101] <Example 4> Figure 5 is a diagram illustrating the schematic configuration of the notification system 4 in Example 4.

[0102] Notification system 4 includes output device 800, notification device 900, and notification device 1000. Output device 800 and notification device 900 are devices used by the fifth user. Notification device 1000 is a device used by the sixth user.

[0103] The output device 800 is a device that outputs sound to a fifth user, and is, for example, an earphone. The output device 800 is not limited to an earphone, but may be headphones, a headset, smart glasses, VR goggles, etc.

[0104] The output device 800 may include an acquisition unit 810, an output unit 820, a first communication unit 840, and a first control unit 850. Each component included in the output device 800 may operate in the same manner as the configuration of the corresponding embodiment 1.

[0105] One or more components included in the output device 800 may be located outside the output device 800 and configured to be connectable to the output device 800 and / or the notification device 900 by wire or wireless connection.

[0106] Notification devices 900 and 1000 may be, for example, a personal computer, a smartphone, a smartwatch, smart glasses, etc. Notification devices 900 and 1000 may be configured to include notification units 930, 1030, second communication units 960, 1060, input units 970, 1070, storage units 980, 1080, and second control units 990, 1090. Each component included in notification devices 900 and 1000 may operate in the same manner as the configuration of the corresponding embodiment 1. Notification device 900 may be a terminal capable of controlling output device 800. The second control unit 990 may determine whether to output the acquired sound, which is a sound emitted by the sixth user acquired by the acquisition unit 810, to the output unit 820. The second control unit 990 may determine whether to have the sound emitted by the sixth user acquired by the acquisition unit 810 be held in the holding unit 982. The second control unit 990 may perform a determination on the acquired sound in the same manner as in the first embodiment.

[0107] The notification device 1000 obtains the output status of the sound acquired by the output device 800 for the fifth user. The notification device 900 transmits the obtained output status of the sound acquired for the fifth user to the notification device 1000.

[0108] The notification device 1000 receives from the notification device 900 the output status of the acquired sound for the fifth user. The notification device 1000 notifies the sixth user of the output status of the acquired sound for the fifth user from the output device 800 that it received. The notification device 1000 may also notify the sixth user that the sound emitted by the sixth user has been held in the holding unit 982.

[0109] The notification device 1000 may notify the sixth user by displaying a mark on its display indicating the output status of the acquired sound for the fifth user. For example, the mark may be displayed with a different color or display format depending on the output status of the acquired sound.

[0110] One or more components included in the notification device 900 or notification device 1000 may be located outside the notification device and configured to be connectable to each other by wired or wireless means. For example, the notification unit 1030 may be located outside the notification device 1000. For example, the storage unit 1080 may be located on a remote server configured to communicate with the notification device 1000 via a network system or the like. For example, applications downloaded to the notification device 900 and notification device 1000 may control each component included in the notification system 4.

[0111] An example of how to use the notification system 4 according to this embodiment will be described.

[0112] In this embodiment, for example, the fifth user is performing some kind of task. In such a situation, the settings are configured so that sound data from a third party's speech is not output in real time from the output device 800 but is instead stored in the storage unit 982. In this case, if the sixth user speaks directly to the fifth user, the fifth user is unable to respond because they are performing a task, but the acquisition unit 810 of the output device 800 acquires the speech from the sixth user. The output device 800 transmits the sixth user's speech to the notification device 900 via the first communication unit 840 and the second communication unit 960 of the notification device 900. The storage unit 982 of the notification device 900 stores the sound data of the sixth user's speech that it has received. At this time, the output unit 820 of the output device 800 may issue the fourth and fifth notifications. By recognizing the fourth and fifth notifications, the sixth user can know that their speech has been stored in the notification device 900, even if the fifth user does not respond. Furthermore, the notification device 900 may notify the notification device 1000 that the holding unit 982 has held the sound uttered by the sixth user. The notification unit 1030 of the notification device 1000 may notify the sixth user that the sound uttered by the sixth user has been held. As a result, the sixth user can also be recognized by the notification device 1000 that their utterance has been held by the notification device 900.

[0113] Subsequently (for example, after the fifth user has finished their work), if the second control unit 990 causes the output unit 820 to output the sixth user's utterance held in the holding unit 982 in response to the fifth user's operation, the notification device 900 notifies the notification device 1000. Upon receiving the notification, the notification device 1000 notifies the sixth user, for example, via the display, that the sixth user's utterance has been output from the output unit 800 (i.e., the fifth user has heard the sixth user's utterance). At this time, while the sixth user's utterance is being output from the output unit 820, the notification may be in the same form as the second notification. After the output of the sixth user's utterance from the output unit 820 has finished, the notification may be in the same form as the third notification.

[0114] As described above, the sixth user can be informed by the notification device 1000 that their utterance to the fifth user has been heard by the fifth user. Therefore, even if the sixth user leaves the area after initially speaking to the fifth user, they can confirm that their utterance reached the fifth user.

[0115] <Effects of this embodiment> The notification device according to the present invention includes a notification unit that notifies the surrounding people of the status of the output of acquired sound, which is sound acquired by the acquisition unit from surrounding people, to the user by the output unit, and a control unit that causes the notification unit to send a notification according to the output status of the acquired sound by the output unit, thereby enabling the acquisition unit to notify the surrounding people of the status of sound acquired by the surrounding people.

[0116] According to one embodiment, a notification device, notification system, or notification method can be provided that notifies people in the surrounding area of ​​the output status of sounds acquired by the acquisition unit from people in the surrounding area to the user.

[0117] Furthermore, this technology can also be configured as follows.

[0118] (1) The notification device includes a notification unit that notifies the surrounding people of the status of the output of acquired sound, which is sound acquired by the acquisition unit from surrounding people, to the user by the output unit, and a control unit that causes the notification unit to send a notification according to the output status of the acquired sound by the output unit.

[0119] (2) In the notification device described in (1) above, the control unit causes the notification unit to issue a first notification when the output unit outputs the acquired sound.

[0120] (3) In the notification device described in (2) above, the control unit causes the notification unit to issue a second notification different from the first notification if the output unit does not output the acquired sound.

[0121] (4) In the notification device described in (2) or (3) above, the control unit causes the notification unit to issue a first notification after deciding to have the output unit output the acquired sound, but before starting the output.

[0122] (5) In the notification device described in (4) above, the control unit causes the notification unit to issue a third notification different from the first notification after the output of the acquired sound has finished.

[0123] (6) In the notification device described in (4) above, the control unit causes a third notification to be issued after the output unit has output the acquired sound and before a predetermined time has elapsed since the output unit has elapsed.

[0124] (7) In the notification device described in any of (1) to (6) above, the control unit causes the acquired sound that satisfies the setting conditions set in advance by the user to be output from the output unit.

[0125] (8) In the control system described in (3) above, if the output unit does not output the acquired sound, the control unit causes the acquired sound to be held in the holding unit, and in response to input from the user to the input unit, the control unit causes the acquired sound held in the holding unit to be output.

[0126] (9) In the notification device described in (8) above, the control unit causes the notification unit to issue a fourth notification different from the first notification when the holding unit holds the acquired sound.

[0127] (10) The notification system comprises a notification unit that notifies the surrounding people of the status of the output of acquired sound, which is sound acquired by the acquisition unit from surrounding people, to the user by the output unit, and a control unit that causes the notification unit to send a notification according to the output status of the acquired sound by the output unit.

[0128] (11) The notification method is a notification method in which a notification unit notifies the surrounding people of the output status of the acquired sound, which is sound obtained from the surrounding people, to the user, and the notification unit provides a notification according to the output status of the acquired sound by the output unit.

[0129] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the device can be configured as a state determination system by functionally or physically distributing and integrating them in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of Symbols]

[0130] Notification systems 1, 2, 3, 4 Notification device 100 Control device 200 Output devices: 300, 500, 800 Notification device 400, 600, 700, 900, 1000 Acquisition part 110, 410, 610, 710, 810 Output section: 120, 320, 520, 820 Notification section 130, 430, 630, 730, 930, 1030 First Communications Department: 140, 340, 540, 840 First control unit: 150, 350, 550, 850 Second Communications Department: 260, 460, 660, 760, 960 Inputs: 270, 470, 670, 770, 970 Storage 280, 480, 680, 780, 980, 1080 Storage units 281, 481, 681, 781, 981, 1081 Holding part 282, 482, 682, 782, 982, 1082 Second control unit: 290, 490, 690, 790, 990, 1090 Output control units 291, 491, 691, 791, 991, 1091 Notification control unit 292, 492, 692, 792, 992, 1092

Claims

1. A notification unit notifies the surrounding people of the status of the output of acquired sound, which is sound acquired by the acquisition unit from surrounding people, to the user by the output unit. A notification device comprising a control unit that causes the notification unit to provide notification according to the output status of the sound acquired by the output unit.

2. The control unit causes the notification unit to issue a first notification when the output unit outputs the acquired sound. The notification device according to claim 1.

3. The control unit, when the output unit does not output the acquired sound, causes the notification unit to issue a second notification different from the first notification. The notification device according to claim 2.

4. The control unit, after deciding to have the output unit output the acquired sound, but before starting the output, has the notification unit issue a first notification. The notification device according to claim 2.

5. The control unit, after the output of the acquired sound has finished, causes the notification unit to issue a third notification different from the first notification. The notification device according to claim 4.

6. The control unit causes a third notification, different from the first notification, to be issued after the output unit has output the acquired sound and before a predetermined time has elapsed since the output has been terminated. The notification device according to claim 4.

7. The control unit causes the acquired sound that satisfies the setting conditions set in advance by the user to be output from the output unit. The notification device according to claim 1.

8. The control unit, when the output unit does not output the acquired sound, has the acquired sound held in the holding unit, and outputs the acquired sound held in the holding unit in response to input from the user to the input unit. The notification device according to claim 3.

9. The control unit, when the holding unit holds the acquired sound, causes the notification unit to issue a fourth notification different from the first notification. The notification device according to claim 8.

10. A notification unit that notifies the surrounding people of the status of the output of acquired sound, which is sound acquired by the acquisition unit from surrounding people, to the user by the output unit, and a control unit that causes the notification unit to send a notification according to the output status of the acquired sound by the output unit, A notification system equipped with this feature.

11. A notification method in which a notification unit notifies the surrounding people of the output status of acquired sound, which is sound acquired from the surrounding people, to the user, The notification unit provides notification according to the output status of the sound acquired by the output unit. Notification methods, including those mentioned above.

Images