Sound input / output control device, sound input / output control method, and recording medium

Abstract

The sound input and output control device (200) comprises: a posture calculation unit (201) for calculating the posture of each of a plurality of sound output devices (100) independently worn on a user; a synchronization determination unit (202) for determining whether the posture changes of at least two of the plurality of sound output devices (100) are synchronized; an action control unit (203) for controlling the actions of each of the sound output devices (100); and an action control unit (203) for controlling the sound output devices (100) to perform a first action when the posture changes are synchronized, and for controlling the sound output devices (100) to perform a second action when it is determined that the posture changes are not synchronized.

Description

Technical Field

[0001] The present invention relates to a sound input / output control device, a sound input / output control method and a program. Background Art

[0002] Patent document 1 describes the following technology: measuring the angles of the left and right earphones of the headset, and judging whether the headset is worn on the user's head and which earphone covers the user's left ear or right ear according to the measured angles. In addition, Patent document 1 describes switching between music playback and standby, or switching which earphone outputs which sound signal for the left ear and the right ear according to the above judgment result.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent document 1: Japanese Patent Application Publication No. 2019-180082. Summary of the invention

[0006] In addition, in recent years, fully wireless headphones (TWS: True Wireless Stereo) in which the left and right earphones are completely wireless have become popular. Fully wireless headphones output sound in stereo through the left and right earphones. The inventors of the present invention have come up with the following technology: it is possible to set a working mode other than the normal working mode such as stereo sound output in multiple sound output devices that can be used independently, like fully wireless headphones.

[0007] An object of the present embodiment is to provide a sound input / output control device, a sound input / output control method, and a program that can provide a wide range of uses through a sound output device.

[0008] The sound input and output control device in this embodiment is a sound input and output control device that controls multiple sound output devices, which are independently worn by a user and output sound to the user. The sound input and output control device includes: a posture calculation unit, which calculates the posture of each of the sound output devices based on a posture signal detected in each of the sound output devices; a synchronization determination unit, which determines whether the posture changes of at least two of the multiple sound output devices calculated by the posture calculation unit are synchronized; and an action control unit, which controls the action of each of the sound output devices. When the synchronization determination unit determines that the posture changes of at least two of the multiple sound output devices are synchronized, the action control unit controls each of the sound output devices to perform a first action, and when the synchronization determination unit determines that the posture changes of at least two of the multiple sound output devices are not synchronized, the action control unit controls each of the sound output devices to perform a second action.

[0009] The sound input and output control method in this embodiment is a sound input and output control method for controlling multiple sound output devices that are independently worn on a user and output sound to the user by a sound input and output control device, wherein the sound input and output control device calculates the posture of each sound output device based on a posture signal detected in each of the sound output devices, determines whether the calculated posture changes of at least two of the multiple sound output devices are synchronized, controls each of the sound output devices to perform a first action when it is determined that the posture changes of at least two of the multiple sound output devices are synchronized, and controls each of the sound output devices to perform a second action when it is determined that the posture changes of at least two of the multiple sound output devices are not synchronized.

[0010] The program in this embodiment is a program that causes a sound input / output control device to execute processing for controlling a plurality of sound output devices, wherein the plurality of sound output devices are independently worn by a user and output sound to the user, and the program causes the sound input / output control device to execute the following processing: processing for calculating the posture of each of the sound output devices based on posture signals detected in each of the sound output devices; processing for determining whether the calculated posture changes of at least two of the plurality of sound output devices are synchronized; and processing for controlling each of the sound output devices to perform a first action when it is determined that the posture changes of at least two of the plurality of sound output devices are synchronized, and controlling each of the sound output devices to perform a second action when it is determined that the posture changes of at least two of the plurality of sound output devices are not synchronized.

[0011] According to the present embodiment, it is possible to provide a sound input / output control device, a sound input / output control method, and a program that can provide a wide range of uses through a sound output device. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a block diagram schematically showing the configuration of the sound input / output control device according to Embodiment 1 of the present invention.

[0013] Figure 2 This is a block diagram schematically showing the structure of the sound input and output control device according to the second embodiment of the present invention.

[0014] Figure 3 This is a flowchart showing an example of the voice input and output control method according to Embodiment 1 or 2 of the present invention.

[0015] Figure 4 This is a flowchart showing another example of the sound input and output control method according to Embodiment 1 or 2 of the present invention.

[0016] Figure 5 This is a flowchart showing another example of the sound input and output control method according to Embodiment 1 or 2 of the present invention.

[0017] Figure 6 This is a flowchart showing another example of the sound input and output control method according to Embodiment 1 or 2 of the present invention. DETAILED DESCRIPTION

[0018] Implementation Method 1

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0020] Figure 1 2 is a block diagram schematically showing the structure of the sound input / output control device 200 according to the first embodiment of the present invention. Figure 1 Specifically, the sound input / output control device 200 is independently worn by a user and controls a plurality of sound output devices 100 that output sound to the user. Thus, the sound input / output control device 200 can give the sound output device 100 a wider range of uses.

[0021] The sound output device 100 is a completely independent headset such as a fully wireless headset, a fully wireless bone conduction headset, etc., for example, independently worn on the left and right ears of the user. That is, multiple sound output devices 100 are worn by the user completely independently of each other. Usually, a fully wireless headset, a fully wireless bone conduction headset, etc. is a group of two completely independent headsets. However, in the present invention, more than three sound output devices 100 can also be a group. Multiple sound output devices 100 can be worn independently on the left and right cheekbones of the user, etc., and can also be neck speakers or wearable speakers. The sound output device 100 is not limited to a fully wireless type, and a part of it can also be a wired connection.

[0022] like Figure 1 As shown, the sound output device 100 of the first embodiment includes a posture sensor 101, a sound output unit 102, a sound input unit 103, a communication unit 104, a wearable sensor 105, and the like. In addition, as long as at least one sound output device 100 in a group of sound output devices 100 includes the sound input unit 103, the other sound output devices 100 may not include the sound input unit 103. In addition, the sound output device 100 may not include the wearable sensor 105. In addition, a mode in which none of the sound output devices 100 in a group include the sound input unit 103 is also possible.

[0023] The attitude sensor 101 is a sensor for detecting the attitude of the sound output device 100 having the attitude sensor 101, and is, for example, a 3-axis azimuth sensor (geomagnetic sensor), a 3-axis angular velocity sensor (gyro sensor), a 3-axis acceleration sensor, a GNSS (Global Navigation Satellite System) receiver, etc. The attitude signal detected by the attitude sensor 101 is transmitted to the sound input and output control device 200 through the communication unit 104. In the case where the attitude sensor 101 is a 3-axis azimuth sensor, the attitude sensor 101 detects an attitude signal related to the orientation of the sound output device 100. In addition, in the case where the attitude sensor 101 is a 3-axis angular velocity sensor, the attitude sensor 101 detects an attitude signal related to the rotation angle of the sound output device 100 around three axes. In addition, in the case where the attitude sensor 101 is a 3-axis acceleration sensor, the attitude sensor 101 detects an attitude signal related to the rotation angle of the sound output device 100 around two axes that are orthogonal to the gravitational acceleration direction. In addition, when the attitude sensor 101 is a GNSS receiver, the attitude sensor 101 detects an attitude signal related to the moving direction of the sound output device 100 .

[0024] The sound output unit 102 includes a speaker or a bone conduction speaker, etc., and outputs sound to the user according to the sound signal obtained from a portable device such as a smartphone, a DAP (Digital Audio Player), a music playback device such as a CD player (not shown). The sound signal from the music playback device can be received by the communication unit 104 via the sound input and output control device 200, or can be received directly by the communication unit 104 without passing through the sound input and output control device 200.

[0025] In addition, the sound output unit 102 may also output sound to the user according to the sound signal received by the communication unit 104 from the other sound output device 100. Specifically, the sound input unit 103 of the other sound output device 100 obtains the sound signal by picking up the sound, and the sound output unit 102 outputs the sound according to the sound signal received by the communication unit 104 from the other sound output device 100.

[0026] The sound input unit 103 is a microphone or the like, and collects the sound of the user wearing the sound output device 100 to obtain a sound signal. In addition, the sound input unit 103 can collect the user's voice when the user wears the sound output device 100 on the ear. The sound signal obtained by the sound input unit 103 is transmitted to the sound input and output control device 200 or other sound output devices 100 through the communication unit 104. In addition, the sound input unit 103 can be provided in all sound output devices 100 in a group of sound output devices 100, or in at least one sound output device 100. In the case where only one sound output device 100 in a group of sound output devices 100 includes the sound input unit 103, the sound output device 100 including the sound input unit 103 can be used as a master device, and the other sound output devices 100 can be used as slave devices.

[0027] The communication unit 104 performs communication between the sound output device 100 included in the communication unit 104 and other sound output devices 100 and the sound input / output control device 200 , or the above-mentioned music playback device (not shown).

[0028] The communication between the sound output devices 100 is near field magnetic induction (NFMI) communication, wireless LAN such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), infrared communication, NFC (Near Field Communication) and other short-range wireless communication, etc. Which communication is used for the communication between the sound output devices 100 may be switched according to the distance between the sound output devices 100.

[0029] The communication between the sound output device 100 and the sound input / output control device 200 is wireless LAN such as Bluetooth (registered trademark), Wi-Fi (registered trademark), infrared communication, short-distance wireless communication such as NFC, etc. Which communication method is used for the communication between the sound output device 100 and the sound input / output control device 200 may be switched according to the distance between the sound output device 100 and the sound input / output control device 200.

[0030] The wearing sensor 105 is a sensor that detects that the sound output device 100 is worn by the user. Specifically, the wearing sensor 105 is an infrared sensor, a pressure-sensitive sensor, etc. A wearing detection signal detected by the wearing sensor 105 is transmitted to the sound input / output control device 200 via the communication unit 104 .

[0031] As described above, the sound input / output control device 200 is independently worn by the user and controls a plurality of sound output devices 100 that output sound to the user. For example, the sound input / output control device 200 controls at least two sound output devices 100 in a group of sound output devices 100. Figure 1 As shown, the sound input / output control device 200 includes a posture calculation unit 201, a synchronization determination unit 202, a motion control unit 203, a sound input / output processing unit 204, a communication unit 205, and the like.

[0032] The posture calculation unit 201 calculates the posture of each sound output device 100 based on the posture signal detected in each sound output device 100. That is, the posture calculation unit 201 obtains the posture signal from each of a group of sound output devices 100 individually, and calculates the posture of each of the sound output devices 100 based on the posture signal.

[0033] Specifically, when the attitude sensor 101 of the sound output device 100 is a three-axis orientation sensor, the attitude calculation unit 201 calculates the orientation of each sound output device 100. In addition, when the attitude sensor 101 of the sound output device 100 is a three-axis angular velocity sensor, the attitude calculation unit 201 calculates the angular velocity of the movement of each sound output device 100. In addition, when the attitude sensor 101 of the sound output device 100 is a three-axis acceleration sensor, the attitude calculation unit 201 calculates the attitude of each sound output device 100 with respect to the gravity direction, or the acceleration of the movement of each sound output device 100. In addition, when the attitude sensor 101 is a GNSS receiver, the attitude calculation unit 201 calculates the travel direction, movement distance, and movement speed of each sound output device 100.

[0034] The synchronization determination unit 202 determines whether the posture changes of the plurality of sound output devices 100 calculated by the posture calculation unit 201 are synchronized. In other words, the synchronization determination unit 202 determines whether the posture changes of the plurality of sound output devices 100 calculated by the posture calculation unit 201 are synchronized.

[0035] When multiple sound output devices 100 are worn on the left and right ears of the same user, for example, when the user is walking, the multiple sound output devices 100 detect motions vibrating at substantially the same rhythm in the vertical direction for motions (posture changes) at the same speed and in the same direction in the front-back direction when viewed from the user, and almost no posture changes are detected in the left-right direction. In addition, for example, when the user twists his neck, the multiple sound output devices 100 detect rotational motions such as moving in exactly opposite directions at the same speed in the front-back and left-right directions when viewed from the user, and almost no posture changes are detected in the vertical direction. By the differences in the user's respective motions such as lowering the head, jumping, and sleeping, it is possible to determine motions at the same speed and in the same direction, or at the same speed and in exactly opposite directions, and determine them as synchronized posture changes.

[0036] In addition, even when multiple sound output devices 100 are worn on the left and right ears of the same user, there may be situations where the speeds of movements such as oblique twisting and rotation around the user's left and right shoulders are inconsistent, so it is preferable to have a function to improve the judgment accuracy through machine learning, etc.

[0037] Here, "posture change" refers to the change in the postures of the multiple sound output devices 100 when the multiple sound output devices 100 are worn by the same user. For example, the posture calculation unit 201 may detect the orientations of the sound output devices 100 worn in a manner directly opposite to the left and right ears of the same user as being exactly opposite orientations. In addition, there may be a situation where the axial direction of the user's right ear hole is inconsistent with the axial direction of the left ear hole. Therefore, it is preferred that the synchronization determination unit 202 includes a calibration function, which pre-stores the initial values ​​of the postures of the two sound output devices 100 when they are worn on the right and left ears of the user, and based on the initial values ​​of the postures, corrects the values ​​of the postures of the two sound output devices 100 calculated by the posture calculation unit 201. Specifically, when the posture calculation unit 201 calculates the orientation of the sound output device 100, the synchronization determination unit 202 corrects the calculated value of the sound output device 100 worn on one ear by using the difference of the initial value, thereby being able to correct the orientation difference associated with the left-right difference of the user, and when two sound output devices 100 are worn on the right ear and the left ear of the same user, it can be determined that they are in the same posture. Thus, the synchronization determination unit 202 can compare the posture changes of each of the multiple sound output devices 100 when the multiple sound output devices 100 are worn by the same user with high accuracy. Similarly, the synchronization determination unit 202 also uses the initial value to correct other postures calculated by the posture calculation unit 201.

[0038] Specifically, for example, when the orientation of each sound output device 100 is calculated by the orientation calculation unit 201, the synchronization determination unit 202 determines whether the orientation changes of the plurality of sound output devices 100 are synchronized based on whether the orientation changes of the plurality of sound output devices 100 are consistent. In addition, when the angular velocity of the motion of each sound output device 100 is calculated by the orientation calculation unit 201, the synchronization determination unit 202 determines whether the orientation changes of the plurality of sound output devices 100 are synchronized based on whether the angular velocity of the motion of the plurality of sound output devices 100 is consistent. In addition, when the orientation calculation unit 201 calculates the orientation of each sound output device 100 and the acceleration of the motion, the synchronization determination unit 202 determines whether the orientation changes of the plurality of sound output devices 100 are synchronized based on whether the orientation changes of the plurality of sound output devices 100 and the acceleration of the motion are consistent. When the posture calculation unit 201 calculates the movement direction of each sound output device 100 , the synchronization determination unit 202 determines whether the posture changes of the plurality of sound output devices 100 are synchronized based on whether the changes in the movement directions, movement distances, and movement speeds of the plurality of sound output devices 100 are consistent.

[0039] The operation control unit 203 controls the operation of each of the sound output devices 100 .

[0040] Specifically, when the synchronization determination unit 202 determines that the posture changes of the plurality of sound output devices 100 are synchronized, the action control unit 203 controls each sound output device 100 to perform a first action suitable for a case where each sound output device 100 is worn by the same user, for example, an action of reproducing a sound signal acquired from a music reproduction device in a stereo mode. The action control unit 203 may be configured such that, when it is determined that the posture changes acquired from each sound output device 100 are synchronized, the action control unit 203 does not perform a determined action as a first action, but performs an action specified by the user or continues a normal action. In other words, the first action may not be a specially specified action.

[0041] In addition, when the synchronization determination unit 202 determines that the posture changes of the multiple sound output devices 100 are not synchronized, the action control unit 203 controls the sound output devices 100 respectively to perform a second action suitable for the case where each sound output device 100 is worn by a different user, such as an action of reproducing the sound signal obtained from the music reproduction device in monaural mode.

[0042] The details of the first operation and the second operation will be described later.

[0043] Therefore, when a set of sound output devices 100 is composed of two sound output devices 100, the sound input / output control device 200 of the first embodiment determines that the two sound output devices 100 are worn by the same user when the posture changes of the two sound output devices 100 are synchronized. Then, the sound input / output control device 200 controls the two sound output devices 100 to perform the above-mentioned first operation.

[0044] In addition, when a set of sound output devices 100 is composed of two sound output devices 100, if the posture changes of the two sound output devices 100 are not synchronized, the sound input / output control device 200 determines that the two sound output devices 100 are worn by different users. Then, the sound input / output control device 200 controls the two sound output devices 100 to perform the second operation.

[0045] In addition, when a group of sound output devices 100 is composed of three or more sound output devices 100 and the posture changes of two sound output devices 100 in the group of sound output devices 100 are synchronized, the sound input and output control device 200 determines that the two sound output devices 100 are worn by the same user. At the same time, when there are other sound output devices 100 in the group of sound output devices 100 whose posture changes are not synchronized, the sound input and output control device 200 determines that the other sound output devices 100 in the group of sound output devices 100 are worn by different users. And the sound input and output control device 200 controls the two sound output devices 100 to perform the above-mentioned first action, and controls the other sound output devices 100 to perform the above-mentioned second action.

[0046] In addition, when a group of sound output devices 100 is composed of three or more sound output devices 100 and the posture changes of these sound output devices 100 are not synchronized, the sound input and output control device 200 determines that these sound output devices 100 are worn by different users. Then, the sound input and output control device 200 controls these sound output devices 100 to perform the above-mentioned second action.

[0047] The sound input / output processing unit 204 performs predetermined processing on the sound signal received by the communication unit 205 from the music playback device (not shown), and generates a sound signal to be sent to each sound output device 100. For example, the sound input / output processing unit 204 generates a stereo sound signal or a monaural sound signal. Then, the sound signal generated by the sound input / output processing unit 204 is sent to each sound output device 100 through the communication unit 205.

[0048] For example, when a set of sound output devices 100 is composed of a sound output device 100 for the right ear and a sound output device 100 for the left ear, and these sound output devices 100 are worn by the same user, as a first operation, the sound input / output processing unit 204 generates a sound signal for the right ear and a sound signal for the left ear, and the communication unit 205 transmits the sound signal for the right ear generated by the sound input / output processing unit 204 to the sound output device 100 for the right ear, and transmits the sound signal for the left ear generated by the sound input / output processing unit 204 to the sound output device 100 for the left ear. In other words, the first operation is an operation of reproducing a sound signal in a stereo system (single-person mode).

[0049] In addition, when a set of sound output devices 100 are worn by different users, as a second operation, the sound input and output processing unit 204 generates a monaural sound signal, and the communication unit 205 transmits the monaural sound signal to each sound output device 100. In other words, the second operation is an operation of reproducing the monaural sound signal (two-person mode).

[0050] When a group of sound output devices 100 are worn by different users, as a second action, the sound input / output processing unit 204 may also obtain the sound signal picked up by the sound input unit 103, and the communication unit 205 may send the picked up sound signal to other sound output devices 100. In other words, the second action may also be a communication action (transceiver mode) between different users of the sound output device 100. Furthermore, the transceiver mode may be only unidirectional transmission and reception, or may be capable of bidirectional transmission and reception.

[0051] As described later, the second operation may be an operation mode in which the volume or sound quality of the sound output can be individually set.

[0052] The communication unit 205 performs communication between the sound input / output control device 200 and the sound output device 100 and a music playback device (not shown).

[0053] The communication between the sound input / output control device 200 and the sound output device 100 and the music playback device is wireless LAN such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), infrared communication, short-range wireless communication such as NFC, etc. The communication between the sound input / output control device 200 and the sound output device 100 and the music playback device can be switched according to the distance between the sound input / output control device 200 and the sound output device 100 and the music playback device.

[0054] In addition, the sound input / output control device 200 may be integrated with a portable device such as a smartphone, a DAP, a music playback device such as a CD player, etc. In this case, the communication unit 205 performs communication between the sound input / output control device 200 and the sound output device 100. In addition, an application program for executing the above-mentioned functions of the sound input / output control device 200 may be installed in the music playback device.

[0055] In addition, as the attitude sensor 101, two or more sensors such as a 3-axis orientation sensor (geomagnetic sensor), a 3-axis angular velocity sensor (gyro sensor), a 3-axis acceleration sensor, and a GNSS receiver may be mounted on the sound output device 100. By mounting two or more sensors as the attitude sensor 101 in the sound output device 100, the attitudes of the plurality of sound output devices 100 can be detected with higher accuracy. For example, when only a 3-axis angular velocity sensor is mounted as the attitude sensor 101, even if two users each wear one sound output device 100, if the two users stay at a predetermined position, the change in the angular velocity of the movement of the two sound output devices 100 is zero. Therefore, the synchronization determination unit 202 synchronizes the attitude changes of the two sound output devices 100, and the sound input / output control device 200 determines that the two sound output devices 100 are worn by the same user. However, when the posture sensor 101 is equipped with not only a 3-axis angular velocity sensor but also a 3-axis azimuth sensor, if two users are facing different directions, even if the two users stay at a predetermined position, it is possible to detect that the changes in the directions of the two sound output devices 100 are inconsistent.

[0056] In addition, as the attitude sensor 101, only a three-axis orientation sensor or a three-axis acceleration sensor may be mounted.

[0057] In addition, when the posture sensor 101 includes a three-axis acceleration sensor, the posture calculation unit 201 may calculate the movement of the sound output device 100 based on the three-axis movement caused by the position change of the sound output device 100. Furthermore, the synchronization determination unit 202 may determine whether the posture changes of the plurality of sound output devices 100 are synchronized based on whether the movements of the plurality of sound output devices 100 are consistent.

[0058] Implementation Method 2

[0059] Figure 21 is a block diagram schematically showing the structure of a hearable device 300 as a sound input / output control device according to Embodiment 2 of the present invention. The hearable device 300 according to Embodiment 2 is an integrated device of the sound output device 100 and the sound input / output control device 200 according to Embodiment 1. Therefore, the functions of the posture sensor 301, the sound output unit 302, the sound input unit 303, the wearing sensor 304, the posture calculation unit 305, the synchronization determination unit 306, the motion control unit 307, and the sound input / output processing unit 308 included in the hearable device 300 are similar to those of the embodiment 1. Figure 1 The posture sensor 101, the sound output unit 102, the sound input unit 103, the wearing sensor 105, the posture calculation unit 201, the synchronization determination unit 202, the motion control unit 203, and the sound input and output processing unit 204 shown in FIG. In addition, the function of the communication unit 309 included in the hearable device 300 is the same as that of the hearable device 300. Figure 1 The functions of the communication unit 104 shown are the same as the functions of the communication unit 205 combined. Therefore, among the functions of the posture sensor 301, the sound output unit 302, the sound input unit 303, the wear sensor 304, the posture calculation unit 305, the synchronization determination unit 306, the motion control unit 307, the sound input and output processing unit 308, and the communication unit 309, the functions that are the same as the posture sensor 101, the sound output unit 102, the sound input unit 103, the wear sensor 105, the posture calculation unit 201, the synchronization determination unit 202, the motion control unit 203, the sound input and output processing unit 204, the communication unit 104, and the communication unit 205 are omitted.

[0060] In addition, the hearable device 300 is similar to the sound output device 100 of embodiment 1, and is independently worn by the user and outputs sound to the user. Specifically, the hearable device 300 is completely independent headphones such as fully wireless headphones and fully wireless bone conduction headphones, etc., which are independently worn on the left and right ears of the user. That is, multiple hearable devices 300 are worn by the user completely independently of each other. Usually, fully wireless headphones, fully wireless bone conduction headphones, etc. are a group of two completely independent headphones. However, in the present invention, more than three hearable devices 300 can also be a group.

[0061] like Figure 2As shown, the hearable wearable device 300 of the second embodiment includes a posture sensor 301, a sound output unit 302, a sound input unit 303, a wearing sensor 304, a posture calculation unit 305, a synchronization determination unit 306, a motion control unit 307, a sound input and output processing unit 308, a communication unit 309, etc. In addition, in a group of hearable wearable devices 300, as long as at least one hearable wearable device 300 includes the sound input unit 303, the posture calculation unit 305, the synchronization determination unit 306, and the motion control unit 307, the other hearable wearable devices 300 may not include the sound input unit 303, the posture calculation unit 305, the synchronization determination unit 306, and the motion control unit 307. In a group of hearable wearable devices 300, when only one hearable wearable device 300 includes the sound input unit 303, the posture calculation unit 305, the synchronization determination unit 306, and the motion control unit 307, the hearable wearable device 300 including the sound input unit 303, the posture calculation unit 305, the synchronization determination unit 306, and the motion control unit 307 may be used as a master device, and the other hearable wearable devices 300 may be used as slave devices.

[0062] In addition, the hearable device 300 may not include the wearing sensor 304 .

[0063] The posture signal detected by the posture sensor 301 is input to the posture calculation unit 305 . In addition, the posture signal detected by the posture sensor 301 is transmitted to other hearable devices 300 via the communication unit 309 .

[0064] The sound output unit 302 outputs sound to the user based on the sound signal received by the communication unit 309 from the portable device 400 which is a music reproduction device.

[0065] In addition, the sound output unit 302 outputs sound to the user according to the sound signal received by the communication unit 309 from the other hearable device 300 .

[0066] The sound signal acquired by the sound input unit 303 is transmitted to other hearable devices 300 via the communication unit 309 .

[0067] The posture calculation unit 305 calculates the posture of each of the hearable devices 300 based on the posture signal detected by the posture sensor 301 in each of the group of hearable devices 300 .

[0068] For example, when all of a group of hearable devices 300 include the posture calculation unit 305, the posture calculation unit 305 calculates the posture of the hearable device 300 based on the posture signal detected by the posture sensor 301 of the hearable device 300 included in the posture calculation unit 305. Then, the posture of the hearable device 300 calculated by the posture calculation unit 305 is sent to other hearable devices 300 through the communication unit 309.

[0069] In addition, when only one of the hearable devices 300 in the group of hearable devices 300 includes the posture calculation unit 305, the posture calculation unit 305 calculates the posture of the hearable device 300 based on the posture signal detected by the posture sensor 301 of the hearable device 300 included in the posture calculation unit 305. In addition, the posture calculation unit 305 calculates the posture of the other hearable device 300 based on the posture signal detected by the posture sensor 301 of the other hearable device 300 received by the communication unit 309.

[0070] The synchronization determination unit 306 determines whether the posture changes of each of the group of hearable devices 300 are synchronized based on the postures of each of the group of hearable devices 300 calculated by the posture calculation unit 305. The synchronization determination by the synchronization determination unit 306 may be performed separately in all of the group of hearable devices 300. In addition, when only one hearable device 300 in the group of hearable devices 300 includes the posture calculation unit 305 and the synchronization determination unit 306, the posture calculated by the posture calculation unit 305 may be sent from the other hearable devices 300 to the one hearable device 300, and the synchronization determination may be performed by the synchronization determination unit 306 of the one hearable device 300.

[0071] The motion control unit 307 controls the motion of the hearable wearable device 300 .

[0072] In all of the hearable devices 300 in the group, the motion control unit 307 may perform motion control. For example, in two hearable devices 300 in the group, when the posture change of one hearable device 300 is synchronized with the posture change of the other hearable device 300, the motion control unit 307 of the two hearable devices 300 controls the hearable devices 300 to perform the first motion. In addition, the motion control unit 307 of each hearable device 300 other than the two hearable devices 300 in the group controls the hearable device 300 to perform the second motion.

[0073] In addition, in a case where the posture changes of the hearable devices 300 in all of the group of hearable devices 300 are not synchronized, the motion control unit 307 of each hearable device 300 controls the hearable device 300 to perform the second motion.

[0074] In addition, when only one of the hearable wearable devices 300 in the group includes the posture calculation unit 305, the synchronization determination unit 306, and the motion control unit 307, the motion control unit 307 of the one hearable wearable device 300 may also control the motion of the other hearable wearable devices 300. For example, when the posture change of one hearable wearable device 300 is synchronized with the posture change of the other hearable wearable device 300 in the group of hearable wearable devices 300, the motion control unit 307 controls the two hearable wearable devices 300 to perform the above-mentioned first motion. In addition, the motion control unit 307 controls the hearable wearable devices 300 other than the above-mentioned two hearable wearable devices 300 in the group of hearable wearable devices 300 to perform the above-mentioned second motion.

[0075] In addition, in a case where the posture changes of the hearable devices 300 in all of the group of hearable devices 300 are not synchronized, the motion control unit 307 controls the hearable devices 300 to perform the second motion.

[0076] Here, the first operation and the second operation are the same operations as those in the first embodiment.

[0077] The audio input / output processing unit 308 performs predetermined processing on the audio signal received by the communication unit 309 from the portable device 400 as a music playback apparatus, and generates an audio signal to be input to the audio output unit 302 .

[0078] For example, when a set of hearable wearable devices 300 includes a hearable wearable device 300 for the right ear and a hearable wearable device 300 for the left ear, and these hearable wearable devices 300 are worn by the same user, the sound input / output processing unit 308 of the hearable wearable device 300 for the right ear generates a sound signal for the right ear, and the sound input / output processing unit 308 of the hearable wearable device 300 for the left ear generates a sound signal for the left ear.

[0079] Furthermore, when a set of hearable devices 300 are worn by different users, the audio input / output processing unit 308 of each hearable device 300 generates a monaural audio signal.

[0080] The communication unit 309 performs communication between the hearable device 300 included in the communication unit 309 and other hearable devices 300. In addition, the communication unit 309 performs communication between the hearable device 300 included in the communication unit 309 and a portable device 400 as a music playback device.

[0081] The communication between the hearable devices 300 is near field magnetic induction communication, wireless LAN such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), infrared communication, short-range wireless communication such as NFC, etc. Which communication is used for the communication between the hearable devices 300 can be switched according to the distance between the hearable devices 300.

[0082] The communication between the hearable device 300 and the portable device 400 is wireless LAN such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), infrared communication, short-range wireless communication such as NFC, etc. Which communication is used for the communication between the hearable device 300 and the portable device 400 may be switched according to the distance between the hearable device 300 and the portable device 400.

[0083] In addition, the music playback device may be not only the portable device 400 such as the smartphone listed above, but may also be a DAP, a CD player, etc. In addition, the wearable device 300 may be integrated with the music playback device.

[0084] Next, refer to Figures 3 to 6 , an example of the sound input and output control method in Embodiment 1 or 2 of the present invention is described. In addition, in the following description, a case where a set of sound output devices 100 or a set of hearable devices 300 is composed of two sound output devices 100 or two hearable devices 300, one sound output device 100 or hearable device 300 is a master device, and the other sound output device 100 or hearable device 300 is a slave device is described as an example.

[0085] In the following description, the processing flow in the sound input and output control device 200 of Embodiment 1 is described as an example. For the description of the processing flow in the hearable wearable device 300 of Embodiment 2, if the processing of the posture sensor 101, the sound output unit 102, the sound input unit 103, the wear sensor 105, the posture calculation unit 201, the synchronization determination unit 202, the motion control unit 203, the sound input and output processing unit 204, the communication unit 104, and the communication unit 205 are replaced by the processing of the posture sensor 301, the sound output unit 302, the sound input unit 303, the wear sensor 304, the posture calculation unit 305, the synchronization determination unit 306, the motion control unit 307, the sound input and output processing unit 308, and the communication unit 309, respectively, they are the same, so the description is omitted.

[0086] First of all, Figure 3 The example of the sound input and output control method shown is described below.

[0087] First, the posture calculation unit 201 calculates the posture of each sound output device 100 based on the posture signal of the sound output device 100 detected by each of the group of sound output devices 100 (step S101). That is, the posture calculation unit 201 calculates the posture of each of the sound output device 100 for the right ear and the sound output device 100 for the left ear.

[0088] Next, the synchronization determination unit 202 determines whether the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear calculated by the posture calculation unit 201 are synchronized (step S102 ).

[0089] In step S102, when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are synchronized (step S102: Yes), the action control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the single-player mode (first action) (step S103).

[0090] Then, the sound output device 100 for the right ear and the sound output device 100 for the left ear perform normal operations (step S104). Specifically, the sound input / output processing unit 204 generates a sound signal for the right ear and a sound signal for the left ear, and the communication unit 205 sends the sound signal for the right ear to the sound output device 100 for the right ear and sends the sound signal for the left ear to the sound output device 100 for the left ear. Then, the sound output unit 102 of the sound output device 100 for the right ear outputs sound to the user according to the sound signal for the right ear, and the sound output unit 102 of the sound output device 100 for the left ear outputs sound to the user according to the sound signal for the left ear.

[0091] In step S102, when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are not synchronized (step S102: No), the action control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the two-person mode (second action) (step S105).

[0092] Then, the music is reproduced in monaural mode in the sound output device 100 for the right ear and the sound output device 100 for the left ear (step S106). Specifically, the sound input and output processing unit 204 generates a monaural sound signal, and the communication unit 205 transmits the monaural sound signal to the sound output device 100 for the right ear and the sound output device 100 for the left ear. Then, the sound output unit 102 of the sound output device 100 for the right ear and the sound output unit 102 of the sound output device 100 for the left ear respectively output the sound to the user according to the monaural sound signal.

[0093] Next, Figure 4 The example of the sound input and output control method shown in FIG. Figure 4 In the sound input and output control method shown in FIG. 1 , the processing of step S201, step S202, and step S204 are respectively Figure 3 The processing of step S101, step S102, and step S104 of the voice input / output control method shown is the same, and thus the description thereof is omitted.

[0094] In step S202, when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are synchronized (step S202: Yes), the action control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the sound output mode (first action) (step S203).

[0095] In step S202, when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are not synchronized (step S202: No), the action control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the transceiver mode (second action) (step S205).

[0096] Then, the user's voice signal picked up by the microphone (sound input unit) 103 of one sound output device 100 is sent to the other sound output device 100, and the other sound output device 100 receives the voice signal, and the sound output unit 102 of the other sound output device 100 outputs the voice to the user according to the voice signal (step S206). For example, the voice of the user wearing the sound output device 100 for the right ear is picked up by the microphone (sound input unit) 103 of the sound output device 100 for the right ear and sent to the sound output device 100 for the left ear. Moreover, the sound output unit 102 of the sound output device 100 for the left ear outputs the voice to the user wearing the sound output device 100 for the left ear according to the voice signal. Conversely, the voice of the user wearing the sound output device 100 for the left ear is picked up by the microphone (sound input unit) 103 of the sound output device 100 for the left ear and sent to the sound output device 100 for the right ear. Then, the sound output unit 102 of the sound output device 100 for the right ear outputs sound according to the sound signal to the user wearing the sound output device 100 for the right ear. Thus, when the sound output device 100 for the right ear and the sound output device 100 for the left ear are worn by different users, the sound output device 100 for the right ear and the sound output device 100 for the left ear can be used as a transceiver.

[0097] Furthermore, in the above-mentioned transceiver mode, the action control unit 203 may also control the sound output device 100 so as to stop the sound output unit 102 from outputting the sound based on the sound signal, or to reduce the volume of the sound output based on the sound signal. Furthermore, in the above-mentioned transceiver mode, the action control unit 203 may also control the sound output device 100 so as to stop the sound input unit 103 of either the sound output device 100 for the right ear or the sound output device 100 for the left ear from picking up the sound.

[0098] Next, Figure 5 The example of the sound input and output control method shown in FIG. Figure 5 In the sound input and output control method shown in FIG. 1 , the processing of step S301, step S302, step S303, and step S305 are respectively the same as Figure 3 The processes of step S101 , step S102 , step S103 , and step S105 of the voice input / output control method shown are the same, and thus the description thereof is omitted.

[0099] In step S303, after the sound output devices 100 for the right ear and the left ear are controlled to operate in the single-player mode (first operation), the sound output devices 100 for the right ear and the left ear are simultaneously adjusted in volume or sound quality (normal operation) (step S304). Specifically, the sound input and output processing unit 204 generates a sound signal for the right ear and a sound signal for the left ear, and the communication unit 205 sends the sound signal for the right ear to the sound output device 100 for the right ear and sends the sound signal for the left ear to the sound output device 100 for the left ear. In addition, the operation control unit 203, for example, simultaneously adjusts the volume and sound quality of the sound output by the sound output units 102 in the sound output devices 100 for the right ear and the left ear so that the volume and sound quality are substantially the same. Moreover, the sound output unit 102 of the sound output device 100 for the right ear outputs sound to the user according to the sound signal for the right ear, and the sound output unit 102 of the sound output device 100 for the left ear outputs sound to the user according to the sound signal for the left ear.

[0100] In step S305, after the sound output devices 100 for the right ear and the left ear are controlled to operate in the two-person mode (second operation), the volume and sound quality are adjusted individually in the sound output devices 100 for the right ear and the left ear (step S306). Specifically, the sound input / output processing unit 204 generates a sound signal for the right ear and a sound signal for the left ear, and the communication unit 205 sends the sound signal for the right ear to the sound output device 100 for the right ear and the sound signal for the left ear to the sound output device 100 for the left ear. In addition, the operation control unit 203 can, for example, individually adjust the volume and sound quality of the sound output by the sound output unit 102 in the sound output devices 100 for the right ear and the left ear. For example, the sound input / output control device 200 or the music playback device displays a setting screen for adjusting the volume or sound quality for the right ear and a setting screen for adjusting the volume or sound quality for the left ear on its own display unit (not shown). Thus, the user wearing the sound output device 100 for the right ear and the user wearing the sound output device 100 for the left ear can set the volume and sound quality of the sound output device 100 for the right ear and the sound output device 100 for the left ear to the volume and sound quality of the sound output device 100 for the right ear and the sound output device 100 for the left ear to the volume and sound quality of the sound output device 100 for the right ear and the sound output device 100 for the left ear to the user's preference by operating the respective setting screens. Then, the sound output unit 102 of the sound output device 100 for the right ear outputs sound to the user according to the sound signal for the right ear, and the sound output unit 102 of the sound output device 100 for the left ear outputs sound to the user according to the sound signal for the left ear.

[0101] Next, Figure 6 The example of the sound input and output control method shown in FIG. Figure 6 In the sound input and output control method shown in FIG. 1 , the processing of step S401 and step S402 are respectively Figure 3 The processing of step S101 and step S102 of the voice input / output control method shown is the same, so the description thereof is omitted.

[0102] In step S402 , when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are synchronized (step S402 : Yes), the process proceeds to step S103 , S203 , or S303 (step S403 ).

[0103] In step S402, when the posture changes of the sound output device 100 for the right ear and the sound output device 100 for the left ear are not synchronized (step S402: No), the motion control unit 203 determines whether the sound output device 100 for the right ear and the sound output device 100 for the left ear are worn on the same side of different users (step S404). Specifically, the motion control unit 203 determines whether the sound output device 100 for the right ear and the sound output device 100 for the left ear are worn on the same side of different users based on the wearing detection signal detected by the wearing sensor 105 and the postures of the sound output devices 100 for the right ear and the left ear calculated by the posture calculation unit 201. In addition, the motion control unit 203 may make a determination based only on the postures of the sound output devices 100 for the right ear and the left ear calculated by the posture calculation unit 201 without using the wearing detection signal detected by the wearing sensor 105.

[0104] In step S404, when the sound output device 100 for the right ear and the sound output device 100 for the left ear are worn on different sides of different users (step S404: No), the operation control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the two-person mode (step S405). Then, the process proceeds to step S106 or S306 (step S406).

[0105] In step S404, when the sound output device 100 for the right ear and the sound output device 100 for the left ear are worn on the same side of different users (step S404: Yes), the operation control unit 203 controls the sound output device 100 for the right ear and the sound output device 100 for the left ear to operate in the transceiver mode (step S407). Then, the sound signal of the user picked up by the microphone (sound input unit) 103 of one sound output device 100 is sent to the other sound output device 100, the other sound output device 100 receives the sound signal, and the sound output unit 102 of the other sound output device 100 outputs sound to the user according to the sound signal (step S408).

[0106] According to the sound input / output control device 200 in the first embodiment described above, the synchronization determination unit 202 determines whether the posture changes of the sound output devices 100 calculated by the posture calculation unit 201 are synchronized. In addition, when the synchronization determination unit 202 determines that the posture changes of the sound output devices 100 are synchronized, the action control unit 203 controls the sound output devices 100 to perform the first action suitable for the case where the sound output devices 100 are worn by the same user. In addition, when the synchronization determination unit 202 determines that the posture changes of the sound output devices 100 are not synchronized, the action control unit 203 controls the sound output devices 100 to perform the second action suitable for the case where the sound output devices 100 are worn by different users.

[0107] In addition, when the posture changes of a group of sound output devices 100 are not synchronized, the sound input and output control device 200 determines that the sound output devices 100 are worn by different users. Then, the sound input and output control device 200 controls the sound output devices 100 to perform the second operation.

[0108] Therefore, it is possible to give a plurality of sound output devices 100 independently worn by a user a second action suitable for each sound output device 100 worn by a different user in addition to the first action suitable for outputting stereo sound or other normal actions. Therefore, the sound output device 100 can be given a wider range of uses.

[0109] The same is true for the hearable wearable device 300 according to the second embodiment.

[0110] In the above-mentioned embodiment, the present invention is described as a hardware configuration, but the present invention is not limited thereto. The present invention can also be implemented by causing a CPU (Central Processing Unit) to execute a computer program. Figures 3 to 6 The processing steps are recorded in the flowchart.

[0111] In addition, the above-mentioned program can be stored using various types of non-transitory computer readable media and provided to the computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic recording media (e.g., optical magnetic disks), CD-ROMs (Read Only Memory: Read Only Memory) CD-R, CD-R / W, semiconductor memories (e.g., mask ROM, PROM (Programmable ROM: Programmable ROM), EPROM (Erasable PROM: Erasable PROM), flash ROM, RAM (Random Access Memory: Random Access Memory)). In addition, the program can also be provided to the computer via various types of transitory computer readable media (transitory computer readable medium). Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to the computer via wired communication paths such as wires and optical fibers or wireless communication paths.

[0112] In addition, the present invention is not limited to the above-mentioned embodiment, and can be appropriately modified within a range not departing from the gist.

[0113] This application claims the benefit of priority based on Japanese patent application No. 2020-52621, filed on March 24, 2020, and incorporates the entire disclosure of that application herein.

[0114] Industrial Applicability

[0115] It is possible to provide a sound input / output control device, a sound input / output control method, and a program that can provide a sound output device with a wider range of uses.

[0116] Explanation of symbols:

[0117] 100 sound output devices

[0118] 200 Sound input and output control device

[0119] 300 Hearable devices (sound input and output control devices)

[0120] 101, 301 attitude sensor

[0121] 102, 302 sound output unit

[0122] 103, 303 sound input unit

[0123] 104, 309 Ministry of Communications

[0124] 201, 305 Attitude Calculation Department

[0125] 202, 306 Synchronicity determination unit

[0126] 203, 307 motion control unit

[0127] 204, 308 sound input and output processing unit

[0128] 205, 309 Ministry of Communications

[0129] 400 Portable equipment (music reproduction device)

Claims

1. A sound input and output control device that controls a plurality of sound output devices, wherein the plurality of sound output devices are independently worn by a user and output sound to the user, wherein the sound input and output control device include: a posture calculation unit that calculates the posture of each of the sound output devices based on the posture signal detected in each of the sound output devices; a synchronization determination unit that determines whether posture changes of at least two of the plurality of sound output devices calculated by the posture calculation unit are synchronized; and an action control unit, controlling the action of each of the sound output devices, At least one of the sound output devices includes a sound input unit that picks up a sound of a user wearing the sound output device. The action control unit When the synchronization determination unit determines that the posture changes of the plurality of sound output devices are not synchronized with each other, based on the postures of the respective sound output devices calculated by the posture calculation unit, it is determined whether the respective sound output devices are worn on the left ear or the right ear of the respective user, and If at least two of the sound output devices are worn on different sides of different users, at least one of the following controls is performed: control to make each of the sound output devices output sound in a monaural manner; or control to adjust the volume of each of the sound output devices individually, If the sound output devices are worn on the same side of different users, control is performed, The sound picked up by the sound input unit is output from one of the sound output devices including the sound input unit to the other sound output devices.

2. The sound input and output control device according to claim 1, in, The posture signal is a signal related to the orientation of the sound output device. The posture calculation unit calculates the orientation of each of the sound output devices.

3. A sound input and output control method, wherein a sound input and output control device controls a plurality of sound output devices, wherein the plurality of sound output devices are independently worn by a user and output sound to the user, The sound input and output control device calculating the posture of each of the sound output devices based on the posture signal detected in each of the sound output devices, determining whether the calculated posture changes of at least two of the plurality of sound output devices are synchronized, controlling the actions of each of the sound output devices, At least one of the sound output devices includes a sound input unit that picks up a sound of a user wearing the sound output device. When the sound output device determines that the posture changes of the plurality of sound output devices are not synchronized with each other, it is determined whether each of the sound output devices is worn on the left ear or the right ear of a different user based on the calculated postures of each of the sound output devices, and If at least two of the sound output devices are worn on different sides of different users, at least one of the following controls is performed: control to make each of the sound output devices output sound in monaural mode; or control to adjust the volume of each of the sound output devices individually. If the sound output devices are worn on the same side of different users, control is performed so that the sound picked up by the sound input unit of one of the sound output devices having the sound input unit is output to the other sound output devices.

4. A storage medium for storing a computer-readable program for causing a sound input / output control device to execute a process of controlling a plurality of sound output devices, the plurality of sound output devices being independently worn by a user and outputting sound to the user, The program causes the sound input and output control device to execute the following processing: calculating the posture of each of the sound output devices based on the posture signal detected in each of the sound output devices; determining whether the calculated posture changes of at least two of the plurality of sound output devices are synchronized; as well as controlling the actions of each of the sound output devices, At least one of the sound output devices includes a sound input unit that picks up a sound of a user wearing the sound output device. Make the sound output device When it is determined that the posture changes of the plurality of sound output devices are not synchronized with each other, it is determined whether each of the sound output devices is worn on the left ear or the right ear of a different user based on the calculated postures of each of the sound output devices, and If at least two of the sound output devices are worn on different sides of different users, at least one of the following controls is performed: control to make each of the sound output devices output sound in monaural mode; or control to adjust the volume of each of the sound output devices individually. If the sound output devices are worn on the same side of different users, control is performed so that the sound picked up by the sound input unit of one of the sound output devices having the sound input unit is output to the other sound output devices.

Images