Control system, control method, and control program

Abstract

A control system according to one embodiment is characterized by comprising a control unit that, when a first sound satisfying a preset condition is included in an acquired surrounding sound, determines whether or not to cause an output device to output the surrounding sound including the first sound to a user on the basis of the state of the user.

Description

Control System, Control Method, and Control Program Cross - Reference to Related Applications

[0001] This application claims the priority of Japanese Patent Application No. 2024 - 187596 filed in Japan on October 24, 2024, and incorporates the entire disclosure of the prior application herein for reference.

[0002] This disclosure relates to a control system, a control method, and a control program.

[0003] Conventionally, there is known a technology that enables a user to hear ambient sounds while wearing a voice output device such as headphones or earphones. In such a technology, there is known a portable music playback device provided with a notification means for notifying from the headphones that the external sound and a predetermined phrase match when they match (Patent Document 1).

[0004] Japanese Patent Application Laid - Open No. 2001 - 256771

[0005] A control system according to an embodiment includes a control unit that determines whether to output to a user, via an output device, ambient sounds including a first sound that satisfies a preset condition in the acquired ambient sounds, based on the state of the user.

[0006] A control method according to an embodiment includes, when the acquired ambient sounds include a first sound that satisfies a preset condition, the control unit determining whether to output to the user the ambient sounds including the first sound based on the state of the user.

[0007] A control program according to an embodiment causes a control unit to determine whether to output to the user the ambient sounds including the first sound based on the state of the user when the acquired ambient sounds include a first sound that satisfies a preset condition.

[0008] It is a diagram for explaining the schematic configuration of the control system 1. It is a flowchart for explaining the flow of processing of the control system 1.

[0009] The embodiments of this disclosure will be described below with reference to the drawings. In each drawing, the same reference numerals indicate components having the same or equivalent function. The configurations, numerical values, processing flow, functions, and elements described in the following embodiments are merely examples, and they can be freely modified or changed. This does not mean that the technical scope of this disclosure is limited to the following description.

[0010] <Example> Figure 1 is a diagram illustrating the schematic configuration of the control system 1 in the example.

[0011] The control system 1 includes an output device 100 and a control device 200. The output device 100 and the control device 200 may be configured as separate devices or as a single device.

[0012] The output device 100 is a device that outputs sound to the user. The output device 100 is, for example, an earphone. The output device 100 may also be a bone conduction earphone. The output device 100 is not limited to an earphone, but may also be headphones, a headset, a speaker, a smartphone, etc. The output device 100 may have a noise cancellation mode and may also have an ambient sound mode. The noise cancellation mode is a mode that cancels out and reduces ambient sounds, which are sounds generated around the user, so that the user does not hear them. The ambient sound mode is a mode that outputs ambient sounds collected by a microphone, etc., through a speaker to the user. The output device 100 may also be a device that visually displays sound, for example, a terminal such as a smartphone that includes a display.

[0013] The general configuration of the output device 100 will now be described. The output device 100 includes an acquisition unit 110, a first output unit 120, a first communication unit 130, and a first control unit 140.

[0014] The acquisition unit 110 is, for example, a microphone. The acquisition unit 110 acquires ambient sounds and converts them into sound signals. Ambient sounds include, for example, sounds made by the user, sounds made by people around the user, and sounds emitted from sound sources (speakers, etc.) around the user.

[0015] The acquisition unit 110 is located outside the output device 100 and may be configured to communicate with one or both of the output device 100 and the control device 200 via wired or wireless means.

[0016] The first output unit 120 is, for example, a speaker. The first output unit 120 outputs sound to the user. The first output unit 120 may function in a so-called noise cancellation mode by canceling out the sound acquired by the acquisition unit 110 and outputting it. The first output unit 120 may function in a so-called ambient sound capture mode by outputting the sound acquired by the acquisition unit 110 directly from the first output unit 120 in real time. The first output unit 120 may output sound based on the sound signal received from the control device 200.

[0017] The first communication unit 130 is, for example, a communication module, which communicates with the control device 200. The communication module is a communication module that corresponds to any communication standard. The communication standard is, for example, a wired communication standard, or a short-range wireless communication standard including Bluetooth®, infrared and NFC, etc.

[0018] The output device 100 transmits an audio signal to the control device 200 via the first communication unit 130. The output device 100 receives an audio signal from the control device 200 via the first communication unit 130.

[0019] The first control unit 140 may control the first output unit 120 to output sound in either noise cancellation mode or ambient sound capture mode. The first control unit 140 may operate the first output unit 120 in either noise cancellation mode or ambient sound capture mode, depending on the user's selection.

[0020] Next, the general configuration of the control device 200 will be described.

[0021] The control device 200 may be, for example, a smartphone, smartwatch, smart glasses, or personal computer, and may be a terminal capable of controlling the output device 100.

[0022] The control device 200 includes a second communication unit 210, an input unit 220, a second output unit 240, a storage unit 250, and a second control unit 260. One or more components included in the control device 200 may be included in the output device 100. One or more components included in the control device 200 may be provided outside the control device 200 and configured to be connectable to one or both of the control device 200 and the output device 100 by wired or wireless means. For example, the storage unit 250 may be located on a remote server configured to communicate with the control device 200 via a network system or the like.

[0023] The second communication unit 210 is, for example, a communication module, which is connected to the first communication unit 130 and used for communication between the output device 100 and the control device 200. The communication module may support the same communication standard as the first communication unit 130. By connecting the second communication unit 210 and the first communication unit 130, communication is established between the output device 100 and the control device 200. The second communication unit 210 receives sound signals transmitted from the output device 100 via the first communication unit 130. The second communication unit 210 transmits sound signals transmitted from the control device 200 via the first communication unit 130 to the output device 100.

[0024] The input unit 220 includes an input interface capable of receiving input from the user. The input unit 220 may include, for example, one or more buttons, a touch panel, etc. The input unit 220 may be used by the user to select the processing to be performed by the output device 100. The input unit 220 may be used by the user to select whether the first output unit 120 operates in noise cancellation mode or ambient sound mode.

[0025] The second output unit 240 outputs a notification to the user. The second output unit 240 may be configured to include one or more of the following: a display unit 241, a light-emitting unit 242, a vibration unit 243, and a sound-producing unit 244.

[0026] The display unit 241 is, for example, a display located on the surface of the control device 200. The display unit 241 displays images or videos and notifies the user.

[0027] The display unit 241 may be configured as a touch panel display and equipped with the function of an input unit 220.

[0028] The light-emitting unit 242 may be, for example, an LED (Light Emitting Diode) placed on the surface of the control device 200. The light-emitting unit 242 notifies the user by emitting light.

[0029] The vibration unit 243 includes, for example, a vibration element such as a piezoelectric element. The vibration unit 243 notifies the user through vibration.

[0030] The sound-producing unit 244 is, for example, a speaker. The sound-producing unit 244 notifies the user by outputting sound.

[0031] The storage unit 250 is a storage medium including ROM (Read Only Memory) or RAM (Random Access Memory), etc. The storage unit 250 may store a program executed by the control system 1. The program may include a sound processing program executed by the control system 1.

[0032] The memory unit 250 includes a storage unit 251 and a holding unit 252.

[0033] The storage unit 251 stores the sound signals of ambient sounds acquired by the acquisition unit 110 as sound data. The storage unit 251 may store the period from the start to the end of ambient sound generation as a single sound data. The start time of ambient sound generation may be when sound generation (or sound at a predetermined volume or higher) is detected after a set time or longer has passed during which no sound is detected (or ambient sound is below a predetermined volume) (in other words, when no ambient sound is being generated). After the start time of ambient sound generation has been identified (in other words, after the generation of an external sound), if the period during which no sound is detected (or ambient sound is below a predetermined volume) exceeds the set time, the end time of ambient sound generation may be when sound is no longer detected.

[0034] The storage unit 251 may store one or more sound data associated with a unique ID. The sound data may consist of sounds from a single sound source. Here, a single sound source refers to each of the two speakers, for example, if there are two speakers. In other words, even if two speakers are speaking simultaneously, the storage unit 251 stores each speaker's speech separately. The storage unit 251 may store sound data for a predetermined time and, after the predetermined time has elapsed, delete the oldest sound data sequentially. The storage unit 251 may be configured as, for example, a ring buffer. The storage unit 251 may store sound data sequentially in chronological order. When the sound data stored in the ring buffer exceeds the capacity of the ring buffer, the storage unit 251 may sequentially delete the older sound data and store new sound data.

[0035] The holding unit 252 acquires and holds the sound data stored in the storage unit 251. The holding unit 252 may associate and hold, for example, one or more of the following with the acquired sound data: the sound characteristics recognized by the second control unit 260, the source from which the sound originated, the words contained in the sound, or a summary of the sound. The holding unit 252 may also associate the IDs stored in the storage unit 251 with the sound data.

[0036] When the holding unit 252 acquires sound data from the storage unit 251, the sound data may be deleted from the storage unit 251. The holding unit 252 may retain sound data stored in the storage unit 251 from the start to the end of ambient sound generation. The holding unit 252 may retain a portion of the sound data from the start to the end of ambient sound generation.

[0037] The second control unit 260 is configured to include at least one processor, at least one dedicated circuit, or a combination thereof. The processor may be a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA or ASIC. The second control unit 260 executes processes related to the operation of the control system 1 while controlling each component of the control system 1.

[0038] The memory unit 250 stores information indicating the correspondence between the importance of a sound and the output method corresponding to the user's state. An example of the correspondence between the importance of a sound and the output method corresponding to the user's state is shown in Table 1. In Table 1, high, medium, and low are defined as information indicating the importance of sound data determined by the importance determination unit 262. Also in Table 1, first state, second state, and third state are defined as information indicating the user's state determined by the state determination unit 263. Furthermore, the output method to be selected according to the combination of the importance of the sound data and the user's state is defined. In this embodiment, the output method includes, for example, a "first output method," a "second output method," and a "third output method." The "first output method" is an output method that outputs a sound including the ambient sound to the user at a near real-time timing immediately after the acquisition unit 110 acquires the ambient sound. The "second output method" is an output method that outputs a sound including the ambient sound at a later timing than the first output method. Specifically, it may be an output method that outputs after the end of the generation of ambient sound is recognized. The "third output method" is an output method that outputs sound, including ambient sounds, at an arbitrary timing in response to user operation, rather than at a predetermined timing. The importance determination unit 262 and the state determination unit 263 will be described later. Note that the first to third output methods may define not only the timing of sound output, but also the volume, sound output speed, and sound pitch, or a combination thereof.

[0039]

[0040] The memory unit 250 stores an importance determination table used to determine the importance of acquired ambient sounds. Table 2 is an example of an importance determination table. The importance determination table defines the importance of a sound in relation to a first setting condition that indicates the characteristics of the sound. The importance determination table defines the importance in relation to the first setting condition. The first setting condition defines a condition that indicates the characteristics of the sound. The first setting condition may be set in advance by the user. In this embodiment, for example, the first setting condition is defined as a sound that includes a pre-registered word and a sound that comes from a pre-registered sound source. The pre-registered word may be, for example, the user's name or a call. The call may be, for example, "Excuse me". The pre-registered sound source may be, for example, a specific person, a traffic light, a telephone, a car, a visitor chime, or an announcement. The first setting condition may also be a sound in which the same sound is repeated. A sound in which the same sound is repeated may be, for example, the sound of a car horn, such as "beep beep". A repeated sound may be, for example, the voice of someone nearby speaking to person A, such as "Person A, Person A." The interval between repeated sounds may be set to a predetermined time. In this specification, ambient sounds that satisfy the first setting condition may be referred to as the first sound.

[0041] The importance level indicates the priority given to notifying the user of the first sound. There may be multiple levels of importance, with higher importance levels indicating a higher priority for notifying the user. In this embodiment, the importance level is set to three levels, including "high," "medium," and "low." "High" is the highest level of importance among the three levels. "Medium" is the middle level of importance among the three levels. "Low" is the lowest level of importance among the three levels. For example, the importance level determination table may define an importance level of "high" as associated with the first setting condition, "the type of sound source is an announcement sound." For example, the importance level determination table may define an importance level of "low" as associated with the type of sound source, which is the sound of a traffic light.

[0042]

[0043] The second control unit 260 includes a state determination unit 263, an importance determination unit 262, and an output control unit 261.

[0044] Each component included in the second control unit 260 may be configured by hardware, software, or both. Each component included in the second control unit 260 may be controlled by the second control unit 260. The second control unit 260 may cause the control system 1 to execute various processes in response to an input by a user from the input unit 220. Each component included in the second control unit 260 may be configured to be included within the output device 100.

[0045] The state determination unit 263 detects the state of the user. The state determination unit 263 categorizes the state of the user using a predetermined determination method (described later).

[0046] The state determination unit 263 categorizes the state of the user according to the degree to which the user can receive ambient sound or the first sound. The state of the user may be categorized into a plurality, and in this embodiment, it is categorized into three including a "first state", a "second state", and a "third state". The "first state" may be a state in which the user can receive the first sound or ambient sound. The "second state" may be a state in which the user cannot receive the first sound. The "third state" is a state at an intermediate level between the first state and the second state among the three levels of states.

[0047] The state determination unit 263 may detect the state of the user and categorize the state of the user into a first state to an nth state. The method for determining the state of the user may be set by the user and stored in the storage unit 250. The state determination unit 263 may determine the state of the user based on the state of the control device 200 or the output device 100 connected to the control system 1. The control device 200 or the output device 100 connected to the control system 1 may be a terminal used by the user. Examples of methods by which the state determination unit 263 determines the state of the user are shown below.

[0048] <Determination Method 1> The state determination unit 263 may determine which of the first to third states the user is in based on the current time, day of the week, and other date and time information. The user may pre-set the time, day of the week, and other date and time information to correspond to the user's states from the first to the third state. For example, the "first state" may correspond to the current time being between 9:00 and 18:00, the "second state" may correspond to the current time being after 20:00, and the "third state" may correspond to the current time being between 7:00 and 9:00.

[0049] <Determination Method 2> The state determination unit 263 detects whether the user is using an app on the terminal the user is using, or which app the user is using. The state determination unit 263 may determine which of the first to third states the user is in, depending on the user's app usage status. The user may pre-set app usage statuses associated with the user's states from the first to the third state. For example, the "first state" may be the state in which the user is not using an app, the "second state" may be the state in which the user is using an online meeting app, and the "third state" may be the state in which the user is using a music playback app.

[0050] <Determination Method 3> The state determination unit 263 detects the frequency of input to the terminal being used by the user. Here, the input frequency may be the number of inputs made per predetermined unit time. The input may be input using a keyboard or touch on a touch panel. Based on the frequency of input to the terminal being used by the user, the state determination unit 263 may determine which of the first to third states the user is in. The user may pre-set the input frequency to correspond to the user's states from the first to the third state. For example, the "first state" may be a state in which the user is not making any inputs, the "second state" may be a state in which the user is making inputs at a predetermined frequency or higher, and the "third state" may be a state in which the user is making inputs at a predetermined frequency or lower.

[0051] <Determination Method 4> The state determination unit 263 detects the state of the output device 100 being used by the user. The state determination unit 263 may, for example, detect whether the output device 100 is outputting sound, the type of sound being output and the words contained in the sound, whether the output device 100 is acquiring sound, and the type of sound acquired by the output device 100, the words contained in the sound, or a combination thereof, as the state of the output device 100. Based on the state of the output device 100, the state determination unit 263 may determine which of the first to third states the user is in. The user may pre-set the state of the output device 100 in association with the user's states from the first to third states. For example, the "first state" is a state in which the output device 100 is not outputting sound and the sound acquired by the output device 100 is not a sound uttered by the user; the "second state" is a state in which the output device 100 is outputting sound and the sound acquired by the output device 100 is a sound uttered by the user; and the "third state" may be a state in which the output device 100 is outputting sound and the sound acquired by the output device 100 is not a sound uttered by the user.

[0052] <Determination Method 5> The state determination unit 263 detects the user's location using a detection means such as GPS. Based on the user's location, the state determination unit 263 may determine which of the first to third states the user is in. The user may pre-set their location in association with the user's states from the first to the third state. For example, the "first state" may be the user at their workplace, the "second state" may be the user at home, and the "third state" may be the user on a train.

[0053] <Determination Method 6> The state determination unit 263 uses a biosensor to detect the user's pulse rate or heart rate per predetermined time period. By detecting the pulse rate or heart rate, it can determine whether the user is in a relaxed state or a tense state. Based on the detected pulse rate or heart rate, the state determination unit 263 may determine whether the user is in a first state or a second state. The user may pre-set pulse rate or heart rate to correspond to the user's state in the first state and the second state. For example, the "first state" may be a state in which the user is relaxed and has a pulse rate below a predetermined level. For example, the "second state" may be a state in which the user is tense and has a pulse rate above a predetermined level. The state determination unit 263 may estimate the respiratory rate from the detected pulse rate and determine the user's state based on the number of breaths per predetermined time period.

[0054] The state determination unit 263 may determine the user's state by combining multiple determination methods 1 to 6. For example, the user may pre-assign scores to the user's state detected by determination methods 1 to 6. The state determination unit 263 may use determination methods 1 to 6 to detect the user's state and determine the level of the state based on the score calculated by adding the scores corresponding to the detected user's state.

[0055] The importance determination unit 262 determines the importance of the ambient sounds stored by the storage unit 251 based on the speech recognition results and the importance determination table. The importance determination unit 262 may perform speech recognition processing to recognize the speech contained in the ambient sound data. Based on the results of the speech recognition processing, the importance determination unit 262 determines whether or not a first sound has been detected. Based on the speech recognition processing, the importance determination unit 262 may perform word detection included in the sound data. Based on the results of the speech recognition processing, the importance determination unit 262 may recognize the type of sound source. Based on the results of the speech recognition processing, the importance determination unit 262 may recognize the characteristics of the sound.

[0056] The importance determination unit 262 may determine the importance to be "high" if, for example, the ambient sound satisfies the first setting condition that "the type of sound source is an announcement sound." For example, the importance determination unit 262 may determine the importance to be "low" if the ambient sound satisfies the first setting condition that "the sound of a traffic light is present."

[0057] The output control unit 261 determines the output method for outputting the first sound. The output control unit 261 may determine the output method based on the importance determined by the importance determination unit 262 and the user's state determined by the state determination unit 263. If the acquired ambient sound includes a first sound that satisfies a preset condition, the output control unit 261 may determine an output method, including whether or not to have the output device 100 output a sound containing the first sound to the user, based on the user's state.

[0058] The output control unit 261 may determine the output method for outputting sound data based on the correspondence table shown in Table 1. For example, the output control unit 261 may determine the output method to the first output method when the importance level of the sound data is high and the user's state is in the first state. The method for determining the output method for a given combination of importance level and user state may be set by the user.

[0059] The output control unit 261 may output sound data to the first output unit 120 using an output method determined based on the importance of the sound data and the user's state. For example, the output control unit 261 may output the first sound to the output device 100 when the user's state is a preset first state, but may not output the first sound to the output device 100 when the user's state is a second state different from the first state.

[0060] In the first output method, the output control unit 261 may cause the second output unit 240 to notify the user that the first sound has been acquired before causing the first output unit 120 to output the first sound. The output control unit 261 may retroactively output ambient sounds from the time the ambient sounds including the first sound started to be generated. The output control unit 261 may reduce the volume of the second sound that the output device 100 was outputting before outputting the sound including the first sound. The second sound may be, for example, a sound output by an application that the user was using before the control system 1 caused or acquired the sound including the first sound.

[0061] In the second output method, the output control unit 261 may output the first sound at a time when the first output unit 120 is not outputting the second sound. For example, if the user is using a music application and the output device 100 is outputting music, the output control unit 261 may output the first sound at the time when the music being output switches or when there is silence. For example, if the user is playing a game, the output control unit 261 may output the first sound at a time when the input frequency falls below a predetermined frequency. For example, before the output control unit 261 outputs the first sound to the first output unit 120, the output control unit 261 may have the second output unit 240 notify the user that the first sound has been acquired. The output control unit 261 may output ambient sounds, including the first sound, from the time they start to be generated.

[0062] In the third output method, the output control unit 261 may have the second output unit 240 notify the user and the holding unit 252 hold the first sound. That is, in the third output method, the output control unit 261 does not have the first output unit 120 automatically output the first sound, but rather outputs it at any timing according to the user's operation, as described above. The output control unit 261 may have the holding unit 252 hold the first sound so that the user can later confirm the first sound. The output control unit 261 may notify the user that the first sound has been acquired when the user's state is in the second state.

[0063] Next, we will explain the output method, which is the processing flow of the control system 1, using Figure 2. Figure 2 is a flowchart for explaining the processing flow of the control system 1.

[0064] The second control unit 260 determines whether or not the acquisition unit 110 is acquiring ambient sound (step 1).

[0065] In step 1, if the second control unit 260 determines that the acquisition unit 110 has not acquired ambient sound, it repeats the process in step 1.

[0066] In step 1, if the second control unit 260 determines that the acquisition unit 110 is acquiring ambient sound, the storage unit 251 stores the ambient sound (step 2).

[0067] The second control unit 260 determines whether or not the first sound is included in the ambient sound being stored by the storage unit 251 (step 3).

[0068] In step 3, if the second control unit 260 determines that the first sound is not included, the process in step 1 is repeated.

[0069] In step 3, if the second control unit 260 determines that the first sound is included, it determines the importance of the first sound (step 4). The importance determination unit 262 determines whether the first sound corresponds to an "high," "medium," or "low" importance level.

[0070] The status determination unit 263 determines the user's status (steps 5-7).

[0071] The output control unit 261 determines the method for outputting the first sound based on the importance level determined in step 4 and the user status determined in steps 5 to 7, and outputs the first sound (steps 8 to 16).

[0072] The control system 1 may perform the processing in steps 1 to 16 when the output device 100 is set to noise cancellation mode. That is, when the output device 100 is set to noise cancellation mode, the second control unit may determine whether or not to output a sound including the first sound to the user.

[0073] The control system 1 may perform the processing in steps 1 to 16 when the output device 100 is set to ambient sound mode. That is, when the output device 100 is set to ambient sound mode, the second control unit may determine whether or not to output a sound including the first sound to the user.

[0074] In the above embodiment, importance is determined in step 4, but this step is optional. For example, in step 3, if the second control unit 260 determines that there is a first sound (step 3: YES) and determines that the user's state is the first state, the first output method may be used. In other words, the first sound may be output at a timing close to real time. For example, in step 3, if the second control unit 260 determines that there is a first sound (step 3: YES) and determines that the user's state is the second state, the third output method may be used. In other words, the first sound may not be output automatically, but at any timing according to the user's operation.

[0075] <Effects of this embodiment> The control system according to this embodiment includes a control unit that, when the acquired ambient sound contains a first sound that satisfies a preset condition, determines, based on the user's state, whether or not to output ambient sound including the first sound to the user from the output device. This allows ambient sound to be output at an appropriate timing, making it possible for the user to hear the first sound when they are in a state where they can accept ambient sound.

[0076] Furthermore, this technology can also be configured as follows.

[0077] (1) The control system includes a control unit that, when the acquired ambient sound includes a first sound that satisfies a preset condition, determines, based on the user's state, whether or not to have the output device output the ambient sound including the first sound to the user.

[0078] (2) In the control system described in (1) above, the control unit acquires the user's status based on the status of the output device.

[0079] (3) In the control system described in (1) above, the control unit acquires the user's status based on the status of the terminal connected to the control system.

[0080] (4) In the control device described in (3) above, the terminal is the terminal used by the user.

[0081] (5) In the control system described in (1) to (4) above, the control unit causes the ambient sound, including the first sound, to be output to the output device when the user's state is a preset first state, and does not cause the ambient sound, including the first sound, to be output to the output device when the user's state is a second state.

[0082] (6) In the control system described in (5) above, the first state is a state in which the user is not making any input to the terminal connected to the control system, and the second state is a state in which the user is making input to the terminal.

[0083] (7) In the control system described in (5) or (6) above, the control unit notifies the user that it has acquired the first sound when the user's state is the second state.

[0084] (8) In the control system described in (5) to (7) above, the control unit reduces the volume of the second sound that the output device was outputting before outputting the ambient sound including the first sound.

[0085] (9) In the control system described in (5) to (8) above, the control unit outputs the ambient sound including the first sound at a later time than when the user is in the first state, if the user is in the third state.

[0086] (10) In the control system described in (1) to (9) above, the control unit acquires the user's status based on the frequency of user input to the terminal connected to the control system.

[0087] (11) In the control system described in (1) to (10) above, the control unit determines whether or not to output the ambient sound including the first sound to the user, based on the user's state, when the output device is set to a noise cancellation mode that cancels out ambient sound and the first sound is included in the ambient sound.

[0088] (12) The control method includes determining whether or not to output the ambient sound including the first sound to the user based on the user's state when the acquired ambient sound includes a first sound that satisfies a preset condition.

[0089] (13) The control program causes the control unit to determine, based on the user's state, whether or not to output the ambient sound including the first sound to the user if the acquired ambient sound includes a first sound that satisfies a preset condition.

[0090] Although the present disclosure has been described above using embodiments, the technical scope of the present disclosure is not limited to the scope described in the embodiments above, and various modifications and changes are possible within the scope of its gist. For example, all or part of the device may be configured as a state determination system by functionally or physically distributing and integrating them in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present disclosure. The effects of the new embodiments resulting from the combinations will also have the effects of the original embodiments.

[0091] Control System 1 Output Device 100 Acquisition Unit 110 First Output Unit 120 First Communication Unit 130 First Control Unit 140 Control Device 200 Second Communication Unit 210 Input Unit 220 Second Output Unit 240 Display Unit 241 Light-emitting Unit 242 Vibration Unit 243 Sound-producing Unit 244 Memory Unit 250 Storage Unit 251 Holding Unit 252 Second Control Unit 260 Output Control Unit 261 Importance Determination Unit 262 State Determination Unit 263

Claims

1. A control system comprising a control unit that, when the acquired ambient sound includes a first sound that satisfies a preset condition, determines, based on the user's state, whether or not to have the output device output the ambient sound including the first sound to the user.

2. The control system according to claim 1, wherein the control unit acquires the user's status based on the status of the output device connected to the control system.

3. The control system according to claim 1, wherein the control unit acquires the user's status based on the status of a terminal connected to the control system.

4. The control system according to claim 3, wherein the terminal is a terminal used by the user.

5. The control system according to any one of claims 1 to 4, wherein the control unit causes the ambient sound, including the first sound, to be output to the output device when the user's state is a preset first state, and does not cause the ambient sound, including the first sound, to be output to the output device when the user's state is a second state.

6. The control system according to claim 5, wherein the first state is a state in which the user is not making any input to the terminal connected to the control system, and the second state is a state in which the user is making input to the terminal.

7. The control system according to claim 5 or 6, wherein the control unit notifies the user that the first sound has been acquired when the user's state is the second state.

8. The control system according to any one of claims 5 to 7, wherein the control unit reduces the volume of the second sound that the output device was outputting before outputting the first sound.

9. The control system according to any one of claims 5 to 8, wherein when the user's state is a third state, the control unit outputs the ambient sound including the first sound at a later time than when the user's state is a first state.

10. The control system according to any one of claims 1 to 9, wherein the control unit acquires the user's status based on the frequency of user input to a terminal connected to the control system.

11. The control system according to any one of claims 1 to 10, wherein the control unit determines, based on the user's state, whether or not to output the ambient sound including the first sound to the user when the output device is set to a noise cancellation mode that cancels out ambient sound and the ambient sound includes the first sound.

12. A control method comprising the control unit determining, based on the user's state, whether or not to output the ambient sound, including the first sound, to the user if the acquired ambient sound includes a first sound that satisfies a preset condition.

13. A control program that causes a control unit to determine, based on the user's state, whether or not to output the ambient sound containing the first sound to the user if the acquired ambient sound includes a first sound that satisfies a preset condition.

Images