Sound input device

Abstract

The present application provides a sound input device that can properly acquire sound including a conversation between a wearer and two conversationalists. The solution of the present application is a sound input device 100 that includes: a first arm portion 10 and a second arm portion 20 that can be arranged at a position sandwiching an object sound source; and a plurality of sound collecting portions 41-46 provided on each of the first arm portion 10 and the second arm portion 20 in three or more places (six or more places in total).

Description

Technical Field

[0001] This invention relates to a voice input device. More specifically, this invention relates to a neckband-type device that can be worn around the wearer's neck and is suitable for acquiring the voice of the wearer and their conversation partner. Background Technology

[0002] In recent years, wearable devices that can be worn anywhere on a user's body and sense the user's state and the state of their surroundings have attracted much attention. As wearable devices, various forms are known, such as those that can be worn on the user's wrist, eyes, ears, neck, or on clothing. By analyzing the user information collected by such wearable devices, information useful to the wearer or others can be obtained.

[0003] Furthermore, as a type of wearable device, there are known devices that can be worn around a user's neck and record the voice emitted by the wearer or their interlocutor (Patent Document 1, Patent Document 2). Patent Document 1 discloses a sound processing system comprising a wearable part for use by a user, the wearable part having at least three sound acquisition units (microphones) that acquire sound data for beamforming. In the system described in Patent Document 1, sound acquisition units are provided on each of the left and right arm sections positioned to clamp the wearer's mouth. In particular, Patent Document 1 proposes a preferred embodiment in which three sound acquisition units are arranged on the left arm section and one sound acquisition unit is arranged on the right arm section. Further, in order to suppress noise (road noise) coming from the ground direction, Patent Document 1 preferably arranges one of the four sound acquisition units facing towards the wearer's feet.

[0004] Furthermore, Patent Document 2 describes a device that is a wearable device worn around a user's neck and has a sound acquisition unit (microphone array) with two locations on each of the left and right arms (a total of four locations). This device in Patent Document 2 discloses the acquisition of sound from the mouth of a person speaking to the wearer via the microphone array.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2019-134441

[0008] Patent Document 2: U.S. Patent Application Publication No. 2019 / 0138603 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] However, since the system in Patent Document 1 has multiple microphones arranged around the wearer's mouth, beamforming can be used to extract the wearer's voice. However, because this system has three microphones on one arm and only one on the other, the range of sound emitted around the wearer (ambient noise or the voice of a conversation partner, etc.) that can be appropriately captured is biased towards one area to the left or right of the wearer. That is, in order to emphasize a specific sound (the wearer's or conversation partner's voice, etc.) or suppress other noises through beamforming, the specific sound needs to reach at least three microphones in a straight line. For example, if a specific sound is generated on the wearer's right side, the specific sound will reach the three microphones on the right arm in a straight line. However, if a specific sound is generated on the wearer's left side, since only one microphone is located on the left arm, the sound may be blocked by the wearer's head, etc., depending on the location of the specific sound, making it difficult for the specific sound to reach more than three microphones in a straight line. In this case, it is very likely that beamforming cannot be properly applied to the sound generated on the wearer's left side. Thus, the system in Patent Document 1 is based on the premise of biasing multiple microphones. Although it can capture the sound emitted from the wearer's mouth, it is not a design suitable for capturing other ambient sounds.

[0011] Furthermore, the device in Patent Document 2 is believed to be able to beamform the sounds emitted by the wearer and the interlocutor more effectively than the system in Patent Document 1 because it has two microphones on each of the left and right arms (a total of four). However, the device in Patent Document 2 only assumes that the wearer is having a conversation with one interlocutor. Specifically, for a conversation involving three people (the wearer and two interlocutors), the device in Patent Document 2 has difficulty beamforming the sounds emitted by each person individually. That is, in order to properly beamform, for a sound source, at least three microphones are needed to acquire the sound, and the spatial position of the sound source is determined from the sound components acquired by each microphone, emphasizing the sound components from that sound source or suppressing other sound components. Here, assuming that there are two interlocutors facing the wearer and that both interlocutors are speaking simultaneously, if only four microphones are provided as in the device in Patent Document 2, then for at least two microphones, unless the sound components emitted by the two interlocutors are separated from the sound components acquired by the microphones, the sound source positions of the two interlocutors cannot be accurately determined. Therefore, in conversation scenarios involving the wearer and two interlocutors, the device in Patent Document 2 sometimes fails to perform proper beamforming. Furthermore, assuming the interlocutors are located to the left or right rear of the wearer, if only four microphones are provided as in the device of Patent Document 2, the wearer's head can sometimes become an obstruction, preventing the interlocutor's voice from reaching the three microphones in a straight line. Therefore, there is also a problem: a wide area behind the wearer where beamforming is impossible (an inaudible area).

[0012] Therefore, the main objective of the present invention is to provide a sound input device that can appropriately acquire the sound of the conversation between the wearer and two interlocutors.

[0013] Technical solutions to the problem

[0014] The inventors of this invention, through in-depth research into solutions to the problems existing in prior art, arrived at the following insight: by arranging three or more (totaling six or more) sound-collecting parts (microphones) on each of the two arms positioned where the target sound source is held, the sound of the conversation, including that of the wearer and two interlocutors, can be simultaneously acquired. Furthermore, based on the above insight, the inventors conceived of solutions to the problems of prior art and thus completed this invention. Specifically, this invention has the following configuration.

[0015] This invention relates to a sound input device. The sound input device of this invention includes two arms and multiple sound collection points. The two arms are configured to be positioned to hold a target sound source. An example of the target sound source is the wearer's mouth. Each of the two arms has three or more sound collection points. Specifically, one arm has three or more sound collection points, and the other arm has three or more sound collection points. Alternatively, each arm may have four or more, or five or more sound collection points. A preferred embodiment of the sound input device of this invention is a neckband-type device worn around the user's neck. However, as long as the sound input device has arms that can be positioned to hold the target sound source (the wearer's mouth), other forms such as eyeglasses or ear-hook types may also be used.

[0016] As described above, by providing three or more sound-collecting sections on each of the two arms, not only the sound emitted from the target sound source (e.g., the wearer's mouth) held between the arms, but also the sound generated around the target sound source can be simultaneously acquired. In particular, even when two interlocutors are speaking simultaneously, the voice of the first interlocutor can be emphasized based on the sound acquired from the three sound-collecting sections on one arm, and the voice of the second interlocutor can be emphasized based on the sound acquired from the three sound-collecting sections on the other arm. Therefore, even when a total of three people (the wearer and the two interlocutors) are conversing, appropriate sound processing can be performed on all of their voices. Furthermore, by pre-providing three or more sound-collecting sections on each of the two arms, since the voice of an interlocutor can be acquired through the sound-collecting section on one arm, beamforming processing can be performed on the voice of the interlocutor even when the interlocutor is located to the left or right rear of the wearer. That is, according to the configuration of the present invention, the area behind the wearer where beamforming processing cannot be performed (the inaudible area) can be reduced.

[0017] Preferably, the sound input device of the present invention further includes a sound analysis unit. The sound analysis unit determines the spatial position or direction of the sound source emitting the sound based on the sound acquired by each sound collecting unit. The sound analysis unit may also calculate the time difference between the acquisition of the sounds acquired by each sound collecting unit and determine the position or direction of the sound source based on the time difference. Furthermore, the sound analysis unit may also determine the spatial position or direction of the sound source acquired by each sound collecting unit by referring to a learned model after machine learning. Therefore, beamforming processing, such as emphasis or suppression, can be applied to the sound based on the position or direction of the sound source acquired by each sound collecting unit.

[0018] Preferably, the sound input device of the present invention is a neckband-type device, and the wearer's mouth is used as the target sound source. A neckband-type device is preferred for recording the voices of the wearer and their interlocutors.

[0019] In the sound input device of the present invention, preferably, the sound analysis unit determines whether the sound source determined based on the sound obtained by three or more sound collectors provided on the first arm matches the mouth of the first interlocutor located on the first arm side of the wearer, and determines whether the sound source determined based on the sound obtained by three or more sound collectors provided on the second arm matches the mouth of the second interlocutor located on the second arm side of the wearer. Thus, the voice of the first interlocutor can be recorded and emphasized using the three or more sound collectors on the first arm, and the voice of the second interlocutor can be recorded and emphasized using the three or more microphones on the second arm. In this way, by utilizing the sound collectors on the first arm and the sound collectors on the second arm separately, the separation performance of the voice components of each speaker can be improved when the first and second interlocutors speak simultaneously.

[0020] Preferably, the sound input device of the present invention further includes a sound processing unit. The sound processing unit performs processing to emphasize or suppress sound components included in the sound data acquired by the sound collection unit based on the position or direction of the sound source determined by the sound analysis unit. Alternatively, the sound processing unit may also perform processing to emphasize or suppress sound components included in the sound data acquired by the sound collection unit simultaneously based on the position or direction of the sound source determined by the sound analysis unit. For example, for sound data acquired by the sound collection unit on the wearer's first arm, the sound component of the first interlocutor is emphasized based on the sound component acquired by the sound collection unit on the first arm, while components other than the sound component of the first interlocutor (mainly the sound component of the second interlocutor) are suppressed using the sound component acquired by the sound collection unit on the second arm. Similarly, for sound data acquired by the sound collection unit on the wearer's second arm, the sound component of the second interlocutor is emphasized based on the sound component acquired by the sound collection unit on the second arm, while components other than the sound component of the second interlocutor (mainly the sound component of the first interlocutor) are suppressed using the sound component acquired by the sound collection unit on the first arm. Thus, by using the sound-collecting parts on the first arm and the second arm separately, it is possible to emphasize or suppress the vocal components of the first and second speakers.

[0021] The sound input device of the present invention can be a neckband-type device, and may further include one or more sound-collecting parts at a position corresponding to the back of the wearer's neck. Thus, by pre-positioning the sound-collecting parts at a position corresponding to the back of the wearer's neck, sound from the wearer's back side can also be appropriately collected. In particular, by pre-positioning three sound-collecting parts on each of the left and right arms, and further pre-positioning the sound-collecting parts at the back of the wearer's neck, beamforming can be performed even for sound sources from the wearer's back side. The additional sound-collecting parts provided at the back of the wearer's neck can be one or more. Furthermore, in order to beamform sound sources from the wearer's back side solely through the additional sound-collecting parts provided at the back of the wearer's neck, three or more of these sound-collecting parts may be provided.

[0022] Invention Effects

[0023] According to the present invention, a sound input device is provided that can appropriately acquire the sound of the conversation between the wearer and two interlocutors. Attached Figure Description

[0024] [ Figure 1 ] Figure 1 This is a perspective view showing an example of a neckband-type audio input device;

[0025] [ Figure 2 ] Figure 2 It is a schematic side view showing the state of wearing a voice input device;

[0026] [ Figure 3 ] Figure 3 This is a block diagram illustrating an example of the functional configuration of a sound input device;

[0027] [ Figure 4 ] Figure 4 This schematically illustrates beamforming processing that captures the voices of the wearer and a conversation partner.

[0028] [ Figure 5 ] Figure 5 This schematically illustrates beamforming processing that captures the voices of the wearer and a conversation partner.

[0029] [ Figure 6 ] Figure 6 This schematically illustrates beamforming processing used to capture the voices of the wearer and two conversation partners.

[0030] [ Figure 7 ] Figure 7 This is a flowchart illustrating an example of beamforming processing that captures the voices of the wearer and two conversation partners. Detailed Implementation

[0031] The following description uses accompanying drawings to illustrate specific embodiments of the present invention. The present invention is not limited to the manner described below, but also includes modifications made thereto that are readily apparent to those skilled in the art.

[0032] Figure 1 This indicates that the neckband-type device 100 is one embodiment of the sound input device of the present invention. Furthermore, Figure 2 This indicates the state of wearing the neck-mounted device 100. For example... Figure 1 As shown, the housing constituting the neck-mounted device 100 includes a left arm portion 10, a right arm portion 20, and a main body portion 30. The left arm portion 10 and the right arm portion 20 extend forward from the left and right ends of the main body portion 30, respectively. Viewed from above, the neck-mounted device 100 has a generally U-shaped overall structure. When wearing the neck-mounted device 100, as... Figure 2 As shown, the entire device can be hung around the neck simply by placing the main body 30 against the back of the wearer's neck and allowing the left arm 10 and right arm 20 to hang down from the wearer's neck towards the chest. Various electronic components are housed within the casing of the neck-hanging device 100.

[0033] Multiple microphones 41-46 are provided on the left arm 10 and right arm 20 respectively. The microphones 41-46 are primarily configured to acquire the voices of the wearer and their interlocutors. Figure 1 As shown, a first sound-collecting section 41, a second sound-collecting section 42, and a third sound-collecting section 43 are provided on the left arm portion 10, while a fourth sound-collecting section 44, a fifth sound-collecting section 45, and a sixth sound-collecting section 46 are provided on the right arm portion 20. Furthermore, one or more sound-collecting sections may be added to the left arm portion 10 and the right arm portion 20 as arbitrary elements. Further, a seventh sound-collecting section 47 may be provided as an arbitrary additional element on the main body portion 30 located between the left arm portion 10 and the right arm portion 20. This seventh sound-collecting section 47 is positioned corresponding to the back of the wearer's neck and is configured to acquire sound from the wearer's back. The sound signals acquired by these sound-collecting sections 41 to 47 are transmitted to a control unit 80 (see reference 80) provided within the main body portion 30. Figure 3 The system performs predetermined analysis processing. Furthermore, the main body 30 contains a control system circuit, including electronic circuitry or a battery, such as the control unit 80.

[0034] The first to sixth sound-collecting parts 41 are respectively disposed in front of the left arm 10 and the right arm 20 (on the chest side of the wearer). Specifically, assuming that the neck-mounted device 100 is worn on the neck of a typical adult male (neck circumference 35cm to 37cm), it is preferable that at least the first to sixth sound-collecting parts 41 are located further forward (on the chest side) than the wearer's neck. The neck-mounted device 100 is a device that is assumed to collect the voices of both the wearer and the person speaking to it, and by arranging each of the sound-collecting parts 41 to 46 in front of the wearer's neck, not only the wearer's voice can be obtained, but also the voice of the person speaking to them can be appropriately obtained. Furthermore, preferably, the first to third sound-collecting parts 43 on the left arm 10 and the fourth to sixth sound-collecting parts 44 on the right arm 20 are arranged in a left-right symmetrical manner. Specifically, the first and fourth sound-collecting parts 41, the second and fifth sound-collecting parts 42, and the third and sixth sound-collecting parts 43 are respectively arranged in a linearly symmetrical position.

[0035] A camera unit 60 is further provided on the left arm portion 10. Specifically, the camera unit 60 is provided on the front end face 12 of the left arm portion 10, and can capture still or moving images of the wearer's front side. The images captured by the camera unit 60 are transmitted to the control unit 80 within the main body portion 30 and stored as image data. Furthermore, the images captured by the camera unit 60 can also be transmitted to a server device via the Internet. The position of the speaker's mouth can also be determined from the images captured by the camera unit 60, and processing (beamforming processing) can be performed to emphasize the sound emitted from the speaker's mouth.

[0036] A non-contact sensor 70 is further provided on the right arm portion 20. The sensor 70 is mainly used to sense the hand movements of the wearer on the front side of the neckband device 100 and is located on the front end surface 22 of the right arm portion 20. The sensing information from the sensor 70 is mainly used for activating the camera 60, starting and stopping photography, etc. For example, the sensor 70 can sense the wearer's hand or other object approaching its sensor 70 to control the camera 60, or it can sense the wearer making a predetermined gesture within the sensing range of the sensor 70 to control the camera 60. Furthermore, although the camera 60 is located on the front end surface 12 of the left arm portion 10 and the sensor 70 is located on the front end surface 22 of the right arm portion 20 in this embodiment, the positions of the camera 60 and the sensor 70 can be interchanged.

[0037] Furthermore, the sensing information in the sensor unit 70 can also be used to activate the camera unit 60, the sound collection units 41-47, and / or the control unit 80 (main CPU). For example, while the sensor unit 70, the sound collection units 41-47, and the control unit 80 are normally activated and the camera unit 60 is stopped, the camera unit 60 can be activated when the sensor unit 70 senses a specific gesture (condition 1). Furthermore, in this condition 1, the camera unit 60 can also be activated when the sound collection units 41-47 sense a specific sound. Alternatively, while the sensor unit 70 and the sound collection units 41-47 are normally activated and the control unit 80 and the camera unit 60 are stopped, either the control unit 80 or the camera unit 60 can be activated when the sensor unit 70 senses a specific gesture (condition 2). In this condition 2, the control unit 80 and the camera unit 60 can also be activated when the sound collection units 41-47 sense a specific sound. Alternatively, when only the sensing unit 70 is normally activated while the sound collection units 41-47, the control unit 80, and the camera unit 60 are stopped, any one of the sound collection units 41-47, the control unit 80, and the camera unit 60 can be activated when the sensing unit 70 senses a specific gesture (condition 3). It can be said that the power consumption reduction effect of conditions 1 to 3 is in the order of condition 3 > condition 2 > condition 1.

[0038] The aforementioned left arm portion 10 and right arm portion 20 can be positioned to clamp the neck. These left arm portions 10 and right arm portions 20 are connected by a main body portion 30 located at the back of the wearer's neck. This main body portion 30 houses electronic components (control system circuitry) such as a processor and battery. Figure 1 As shown, the housing constituting the main body 30 has a generally flat shape and can accommodate a planar (plate-like) circuit board and a battery. Furthermore, the main body 30 has a downward-hanging portion 31 that extends further downward than the left arm portion 10 and the right arm portion 20. By providing the downward-hanging portion 31 in the main body 30, space is ensured for the built-in control system circuitry. The control system circuitry is centrally mounted in the main body 30. This control system circuitry includes a battery 90 and a circuit board, which is equipped with various electronic components such as a processor that receives power from the battery 90 and is driven by it. Therefore, when the total weight of the neckband device 100 is set to 100%, the weight of the main body 30 accounts for 40% to 80% or 50% to 70%. By placing this heavy main body 30 behind the wearer's neck, stability during wear is improved. Furthermore, by placing the heavy main body 30 close to the wearer's torso, the burden on the wearer caused by the overall weight of the device is reduced.

[0039] Furthermore, a proximity sensor 83 is provided on the inner side (wearer side) of the main body 30. The proximity sensor 83 only needs to be pre-mounted to, for example, the inner surface of a circuit board. The proximity sensor 83 is used to sense the approach of an object; if the neckband-type device 100 is worn around the wearer's neck, it will sense the approach of the neck. Therefore, when the proximity sensor 83 senses the approach of an object, the devices such as the sound collectors 41-47, the camera 60, and the sensor 70 are turned on (driven state); when the proximity sensor 83 does not sense the approach of an object, these devices can be turned off (sleep state) or set to a state where they cannot be started. This can effectively suppress the power consumption of the battery 90. Furthermore, when the proximity sensor 83 does not sense the approach of an object, by preventing the camera 60 and the sound collectors 41-47 from starting, it is also expected to prevent intentional or unintentional data recording when not wearing the device. Furthermore, while a known sensor can be used as the proximity sensor 83, in the case of using an optical sensor, a penetrating part for allowing the sensing light of the proximity sensor 83 to pass through can be provided in the main body housing.

[0040] Furthermore, the first sound collectors 41 to 46 and the seventh sound collector 47, which can be added as an optional element, can be controlled separately. That is, the seventh sound collector 47 is a component designed to acquire sound from the wearer's back, and it is not necessary to acquire sound from the wearer's front using the seventh sound collector 47. Therefore, in normal scenarios, if the first sound collectors 41 to 46 are activated first, the seventh sound collector 47 does not need to be activated. On the other hand, the seventh sound collector 47 is activated when sound from the wearer's back is sensed by the first sound collectors 41 to 46, or when sound that cannot be properly beamformed is sensed by these sound collectors 41 to 46. In this way, by controlling the seventh sound collector 47 to be turned on / off independently from the first sound collectors 41 to 46, power consumption can be suppressed, and the seventh sound collector 47 can be used efficiently.

[0041] Furthermore, a sound-emitting section 84 (speaker) is provided on the outer side of the main body 30 (opposite to the wearer). The sound-emitting section 84 can be pre-assembled onto, for example, the outer surface of a circuit board. Figure 2As shown, in this embodiment, the sound output unit 84 is configured to output sound towards the outside of the main body 30. Thus, by outputting sound from behind the wearer's neck towards the rear, the sound output from the sound output unit 84 is less likely to be directly transmitted to a person speaking directly in front of the wearer. This prevents the person speaking from confusing the wearer's own voice with the sound output from the sound output unit 84 of the neckband device 100. Furthermore, in this embodiment, the left arm 10 and right arm 20 are provided with the first to sixth sound collection units 41 to 46, but by pre-positioning the sound output unit 84 at a position corresponding to the back of the wearer's neck, the physical distance between the sound output unit 84 and the sound collection units 41-46 can be maximized. That is, when the wearer's or person speaking is being collected by the sound collection units 41-46, if some sound is output from the sound output unit 84, the sound from the sound output unit 84 (self-output sound) may mix with the collected voice of the wearer, etc. If the self-output sound mixes with the recorded sound, it will hinder sound recognition, so it is necessary to remove this self-output sound through echo cancellation or other means. However, in reality, due to factors such as shell vibration, it is difficult to completely remove the self-output sound even after echo cancellation. Therefore, in order to minimize the volume of the self-output sound mixed with the wearer's voice, it is preferable to position the sound output unit 84 at a position corresponding to the back of the wearer's neck, as described above, and at a physical distance from the sound collection units 41 to 46.

[0042] Furthermore, the sound output section 84 is preferably positioned slightly to the left or right, rather than in a location corresponding to the center of the back of the wearer's neck. This is because, compared to being located in the center of the back of the neck, the sound output section 84 is closer to either ear. Thus, by positioning the sound output section 84 slightly to the left or right, rather than roughly in the center of the main body 30, even when the output volume is reduced, the wearer can clearly hear the output sound using either ear. Moreover, if the output volume is low, the output sound is less likely to reach the other person, thus preventing the other person from confusing the wearer's voice with the output sound of the sound output section 84.

[0043] Furthermore, as a structural feature of the neckband-type device 100, the left arm portion 10 and the right arm portion 20 have flexible portions 11 and 21 near their connection points with the main body portion 30. The flexible portions 11 and 21 are formed of flexible materials such as rubber or silicone resin. Therefore, when wearing the neckband-type device 100, the left arm portion 10 and the right arm portion 20 easily conform to the wearer's neck or shoulder. In addition, wiring connecting the first sound-collecting portion 41 to the sixth sound-collecting portion 46 and the operation portion 50 to the control portion 80 is also inserted into the flexible portions 11 and 21.

[0044] Figure 3 This is a block diagram illustrating the functional configuration of the neck-mounted device 100. For example... Figure 3As shown, the neck-mounted device 100 includes a first sound-collecting unit 41 to a seventh sound-collecting unit 47, an operation unit 50, a camera unit 60, a sensor unit 70, a control unit 80, a storage unit 81, a communication unit 82, a proximity sensor 83, a sound playback unit 84, and a battery 90. In this embodiment, the first sound-collecting unit 41, the second sound-collecting unit 42, the third sound-collecting unit 43, the operation unit 50, and the camera unit 60 are arranged on the left arm portion 10. Furthermore, the fourth sound-collecting unit 44, the fifth sound-collecting unit 45, the sixth sound-collecting unit 46, and the sensor unit 70 are arranged on the right arm portion 20. Furthermore, the control unit 80, the storage unit 81, the communication unit 82, the proximity sensor 83, the sound playback unit 84, the seventh sound-collecting unit 47, and the battery 90 are arranged on the main body portion 30. In addition, the neck-mounted device 100, besides... Figure 3 The functional configuration shown can also be appropriately equipped with modules such as gyroscope sensors, accelerometers, geomagnetic sensors, or GPS sensors, which are mounted on general portable information terminals.

[0045] For each sound collection unit 41-47, any known microphone such as a dynamic microphone, condenser microphone, or MEMS (Micro-Electrical-Mechanical Systems) microphone can be used. The sound collection units 41-47 convert sound into electrical signals, amplify these signals using an amplification circuit, convert them into digital information using an A / D conversion circuit, and output them to the control unit 80. One of the objectives of the neckband-type device 100 of the present invention is to acquire not only the wearer's voice but also the voices of one or more people in the surrounding area. Therefore, in order to widely collect the sound generated around the wearer, omnidirectional microphones are preferably used for each sound collection unit 41-47.

[0046] The operation unit 50 receives input from the wearer. The operation unit 50 can be a known switching circuit or touch panel. For example, the operation unit 50 receives operations such as indicating the start or stop of sound input, indicating the power on or off of the device, indicating the volume of the speaker to be increased or decreased, or other operations necessary to realize the functions of the neckband device 100. Information input through the operation unit 50 is transmitted to the control unit 80.

[0047] The camera unit 60 acquires image data of still or moving images. A typical digital camera can be used as the camera unit 60. The camera unit 60 is, for example, composed of: a camera lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor, a digital signal processor (DSP) that reads the charge from the photoelectric conversion element and generates image data, and an IC memory. Furthermore, the camera unit 60 preferably includes: an autofocus sensor (AF sensor) that measures the distance from the camera lens to the subject; and an adjustment mechanism for adjusting the focal distance of the camera lens based on the distance sensed by the AF sensor. The type of AF sensor is not particularly limited; any known passive sensor such as a phase difference sensor or a contrast sensor can be used. Alternatively, an active sensor can be used as the AF sensor, which directs infrared or ultrasonic light toward the subject and receives its reflected light or waves. The image data acquired by the camera unit 60 is supplied to the control unit 80 and stored in the storage unit 81 for predetermined image analysis processing or transmitted to a server device via the Internet through the communication unit 82.

[0048] Furthermore, the camera unit 60 preferably has a so-called wide-angle lens. Specifically, the vertical viewing angle of the camera unit 60 is preferably 100 to 180 degrees, and particularly preferably 110 to 160 degrees or 120 to 150 degrees. In this way, by setting the vertical viewing angle of the camera unit 60 to a wide angle, it is possible to photograph a wide area from at least the head to the chest of the person speaking, and, depending on the situation, the entire body of the person speaking. Furthermore, the horizontal viewing angle of the camera unit 60 is not particularly limited, but a wide angle of approximately 100 to 160 degrees is preferred.

[0049] Furthermore, since the camera unit 60 generally consumes a large amount of power, it is preferable to only activate it when needed, and otherwise put it into a sleep state. Specifically, although the activation of the camera unit 60 and the start or stop of photography are controlled based on the sensing information from the sensor unit 70 or the proximity sensor 83, the camera unit 60 can also be put back into a sleep state after a certain period of time has elapsed since photography has stopped.

[0050] The sensing unit 70 is a non-contact sensing device used to sense the movement of objects such as the wearer's fingers. Examples of the sensing unit 70 are proximity sensors or gesture sensors. A proximity sensor, for example, senses when the wearer's fingers approach a predetermined range. Known sensors such as optical, ultrasonic, magnetic, capacitive, or thermal sensors can be used as proximity sensors. A gesture sensor, for example, senses the movement or shape of the wearer's fingers. An example of a gesture sensor is an optical sensor, which senses the movement or shape of an object by illuminating light from an infrared LED towards the object and capturing changes in the reflected light using a light-receiving element. The sensing information obtained by the sensing unit 70 is transmitted to the control unit 80, primarily for controlling the camera unit 60. Furthermore, the sound collection units 41-47 can also be controlled based on the sensing information obtained by the sensing unit 70. Since the sensing unit 70 generally consumes little power, it is preferable to keep it constantly activated during the power-on period of the neckband device 100. Furthermore, the sensing unit 70 can also be activated when the wear of the neckband device 100 is detected by the proximity sensor 83.

[0051] The control unit 80 performs calculations to control other elements of the neckband-type device 100. The control unit 80 can utilize a processor such as a CPU. The control unit 80 essentially reads a program stored in the storage unit 81 and executes predetermined calculations according to this program. Furthermore, the control unit 80 can appropriately write or read the calculation results obtained by following the program into the storage unit 81. Details will be described later, but the control unit 80 mainly includes a sound analysis unit 80a, a sound processing unit 80b, an input analysis unit 80c, a camera control unit 80d, and an image analysis unit 80e for controlling or beamforming the camera unit 60. These elements 80a to 80e are basically implemented as software functions. However, these elements can also be implemented as hardware circuits.

[0052] The storage unit 81 is used to store information and the results of calculations used in the control unit 80. Specifically, the storage unit 81 stores a program that enables a general-purpose portable information communication terminal to function as the voice input device of this invention. If this program is initiated according to a user's instruction, the control unit 80 executes the program's processing. The storage function of the storage unit 81 can be implemented using non-volatile memory such as HDDs and SDDs. Furthermore, the storage unit 81 can also function as a memory used to write or read intermediate processes of the calculations performed by the control unit 80. The memory function of the storage unit 81 can be implemented using volatile memory such as RAM or DRAM. Furthermore, the storage unit 81 can also store the user's unique ID information. Additionally, the storage unit 81 can also store IP addresses, which are the identification information of the neckband device 100 on the network.

[0053] Furthermore, the learned model can also be used in the beamforming process performed by the storage control unit 80 in the storage unit 81. The learned model is, for example, an inference model obtained through machine learning such as deep learning or reinforcement learning performed on a server device in the cloud. Specifically, in beamforming, sound data acquired by multiple sound collectors is analyzed to determine the location or direction of the sound source. For example, at this time, a large number of data sets (teacher data) containing the sound source location information held by the server device and the sound data generated from the sound source acquired by multiple sound collectors are accumulated, and machine learning using this teacher data is performed to pre-build a learned model. Then, when an individual neckband device 100 acquires sound data through multiple sound collectors, the location or direction of the sound source can be effectively determined by referring to this learned model. Furthermore, the neckband device 100 can update this learned model at any time by communicating with the server device.

[0054] The communication unit 82 is an element used for wireless communication with server devices or other neckband devices in the cloud. To communicate with server devices or other neckband devices via the Internet, the communication unit 82 can, for example, employ a communication module for wireless communication using known mobile communication standards such as 3G (W-CDMA), 4G (LTE / LTE-Advanced), 5G, or wireless LAN methods such as Wi-Fi. Furthermore, for direct communication with other neckband devices, the communication unit 82 can also employ a communication module for short-range wireless communication such as Bluetooth or NFC.

[0055] The proximity sensor 83 is mainly used to sense the proximity of the neckband device 100 (especially the main body 30) to the wearer. As mentioned above, the proximity sensor 83 can be a known sensor such as an optical, ultrasonic, magnetic, electrostatic capacitive, or temperature-sensitive sensor. The proximity sensor 83 is disposed inside the main body 30 to sense that the wearer's neck has approached within a predetermined range. When the proximity sensor 83 detects that the wearer's neck is approaching, the sound collecting units 41-47, the camera unit 60, the sensing unit 70, and / or the sound playing unit 84 can be activated. In addition, as mentioned above, when the proximity sensor 83 detects that the wearer's neck is approaching, only the first sound collecting units 41 to the sixth sound collecting units 46 can be activated initially, while the seventh sound collecting unit 47 can be set to a closed state until sound from the wearer's back is detected.

[0056] The sound-emitting unit 84 is an audio device that converts electrical signals into physical vibrations (i.e., sound). An example of the sound-emitting unit 84 is a general loudspeaker that transmits sound to the wearer through air vibration. In this case, as described above, it is preferable to configure the sound-emitting unit 84 to be located on the outer side of the main body 30 (opposite to the wearer), emitting sound in a direction away from the back of the wearer's neck (horizontally backward) or along the back of the neck (vertically upward). Furthermore, the sound-emitting unit 84 can also be a bone conduction loudspeaker that transmits sound to the wearer by vibrating the wearer's bones. In this case, it is sufficient to configure the sound-emitting unit 84 to be located on the inner side of the main body 30 (wearer side), so that the bone conduction loudspeaker contacts the bones (cervical vertebrae) at the back of the wearer's neck.

[0057] The battery 90 supplies power to the various electronic components included in the neckband device 100. A rechargeable battery can be used as the battery 90. Known batteries such as lithium-ion batteries, lithium polymer batteries, alkaline batteries, nickel-cadmium batteries, nickel-metal hydride batteries, or lead-acid batteries are acceptable. The battery 90 is disposed within the housing of the main body 30, with the circuit board positioned between the battery 90 and the back of the wearer's neck.

[0058] Next, refer to Figures 4 to 6 This section explains the basic concepts of beamforming. If the user wears... Figure 1 The neck-hanging device 100 of the illustrated embodiment, such as Figure 4 (a) and Figure 4 As shown in (b), six microphones 41-46 are located on the chest side of the wearer's neck. The first to sixth microphones 46 are all omnidirectional microphones, primarily collecting sound from the wearer's mouth and also collecting ambient sounds from the wearer's surroundings. Ambient sounds include the voices of other people speaking to the wearer. If the wearer and / or other people speaking make a sound, sound data is acquired by each microphone 41-46. Each microphone 41-46 outputs its own sound data to the control unit 80.

[0059] The sound analysis unit 80a of the control unit 80 processes the sound data obtained from each of the sound collection units 41 to 46. Specifically, the sound analysis unit 80a determines the spatial position or direction of the sound source emitting the sound based on the sound data from each of the sound collection units 41 to 46. For example, when a learned model that has completed machine learning is installed on the neckband-type device 100, the sound analysis unit 80a can determine the position or direction of the sound source from the sound data of each of the sound collection units 41 to 46 by referring to the learned model. Alternatively, since the distance between each of the sound collection units 41 to 46 is known, the sound analysis unit 80a can also calculate the distance from each of the sound collection units 41 to 46 to the sound source based on the time difference of the sound arriving at each of the sound collection units 41 to 46, and determine the spatial position or direction of the sound source from the distance using triangulation. According to triangulation, as long as the distance from at least three sound collection units to the sound source can be determined, the spatial position or direction of the sound source can be determined. Therefore, the location of the sound source can be determined even from the sound data obtained only from the first sound collection section 41 to the third sound collection section 43 on the left arm 10. Similarly, the location of the sound source can be determined even from the sound data obtained only from the fourth sound collection section 44 to the sixth sound collection section 46 on the right arm 20.

[0060] Furthermore, the sound analysis unit 80a determines whether the position or direction of the sound source determined through the above processing matches the position or direction of the mouth presumed to be that of the wearer or the speaker. For example, since the positional relationship between the neckband device 100 and the wearer's mouth or the neckband device 100 and the speaker's mouth is predictable, if the sound source is within its predictable range, it is sufficient to determine that the sound source is the mouth of the wearer or the speaker. Furthermore, if the sound source is clearly located below, above, or behind the neckband device 100, it can be determined that the sound source is not the mouth of the wearer or the speaker.

[0061] Next, the sound processing unit 80b of the control unit 80 performs processing to emphasize or suppress sound components included in the sound data based on the position or direction of the sound source determined by the sound analysis unit 80a. Specifically, when the position or direction of the sound source coincides with the position or direction of the mouth presumed to be that of the wearer or the person speaking, the sound components emitted from the sound source are emphasized. On the other hand, when the position or direction of the sound source does not coincide with the mouth of the wearer or the person speaking, the sound components emitted from the sound source are simply considered as noise and suppressed. Thus, in this invention, multiple omnidirectional microphones are used to acquire omnidirectional sound data, and beamforming processing is performed. This beamforming processing emphasizes or suppresses specific sound components through sound processing in software by the control unit 80. As a result, the voice of the wearer and the voice of the person speaking can be acquired simultaneously, and the sound components of their voices can be emphasized as needed.

[0062] Next, refer to Figure 5 This describes a method that can appropriately beamform the speaker's voice. For example... Figure 5 As shown in (a), when the speaker is positioned in front of the wearer, the sound emitted by the speaker travels in a straight line to all six sound-collecting sections 41-46 mounted on the neckband-type device 100. In this case, as described above, beamforming processing can be performed to emphasize the sound emitted by the speaker. Furthermore, as... Figure 5 As shown in (a), for example, when the speaker is located on the wearer's right side, beamforming can be performed as long as the sound emitted by the speaker on the right side can be obtained using at least the three sound collectors 44-46 provided on the right arm portion 20 of the neckband device 100. The same applies when the speaker is located on the wearer's left side. Thus, in this invention, since three sound collectors 41-46 are provided on each of the left arm portion 10 and the right arm portion 20, the sound of a speaker located on the wearer's left or right side can be handled using only the sound collectors 41-43 on the left arm portion 10 or the sound collectors 44-46 on the right arm portion 20.

[0063] And, as Figure 5 As shown in (b), even when the speaker is located to the left or right rear of the wearer, beamforming can be performed as long as the speaker's voice can be obtained through three or more of the sound-collecting parts 41-43 on the left arm 10 or 44-46 on the right arm 20. Furthermore, in a preferred embodiment of the invention, a seventh sound-collecting part 47 is provided at a position corresponding to the back of the wearer's neck. Therefore, even when the speaker is located approximately directly behind the wearer, beamforming can be performed through at least two of the sound-collecting parts (e.g., the third sound-collecting part 43 and the sixth sound-collecting part 46) and the seventh sound-collecting part 47, as long as the speaker's voice reaches in a straight line to at least two of these three sound-collecting parts (e.g., the third sound-collecting part 43 and the sixth sound-collecting part 46) and the seventh sound-collecting part 47. Therefore, by providing three sound-collecting parts 41-46 on each of the left arm 10 and the right arm 20, and further providing a seventh sound-collecting part 47 on the main body 30, beamforming can be performed in all directions (approximately 360 degrees) around the wearer. In this case, there are no dead zones where beamforming cannot be performed.

[0064] Therefore, in order to properly perform beamforming, the speaker's voice needs to reach at least three sound-collecting points in a straight line. Considering this limitation, by configuring three sound-collecting points (a total of six) on each of the left and right arms 10 and 20 of the neckband device 100, the audible range is expanded to a wide area beyond the wearer's rear, meaning the range where the speaker's voice can be properly beamformed. Specifically, by providing three sound-collecting points 41-46 on each of the left and right arms 10 and 20, the beamforming audible range can be expanded from the wearer's field of vision to approximately 260 degrees or more, preferably 320 degrees or more. This allows the wearer's left and right sides to be almost entirely within the audible range. Therefore, not only can the voice of a speaker facing the wearer be appropriately acquired, but also the voice of a speaker positioned laterally to the wearer can be appropriately acquired. Furthermore, in addition to the aforementioned sound-collecting sections 41 to 46, a seventh sound-collecting section 47 is provided in the main body section 30, thereby enabling beamforming even in areas that cannot be addressed by the sound-collecting sections 41 to 46 on the left and right arms 10 and 20 alone (especially the wearer's back).

[0065] Next, refer to Figure 6 This describes a scenario where a conversation takes place between three people: the wearer of the neck-mounted device 100 and two other individuals not wearing the neck-mounted device. Figure 6 In the example shown, for convenience, the three people participating in the dialogue in the top view are positioned at the vertices of an equilateral triangle, and their relative positions remain unchanged. However, even if their relative positions change over time, it is sufficient to periodically determine their positions from each speaker's voice and track their changed positions. Furthermore, in Figure 6 In this context, the person on the left arm (side 10) will be designated as the "first person to speak" from the perspective of the "wearer," and the person on the right arm (side 20) will be designated as the "second person to speak."

[0066] Thus, suppose a dialogue occurs between three people, with the first and second speakers speaking simultaneously. An example of beamforming processing for this scenario is shown below. Figure 7That is, in the neck-mounted device 100 of the present invention, firstly, the sound analysis unit 80a uses first sound data obtained from the first sound collecting unit 41 to the third sound collecting unit 43 on the left arm 10 to determine the sound source position or direction of the first interlocutor's voice (S1). Then, the sound processing unit 80b processes the first sound data to emphasize the determined voice components of the first interlocutor (S2). Similarly, the sound analysis unit 80a uses second sound data obtained from the fourth sound collecting unit 44 to the sixth sound collecting unit 46 on the right arm 20 to determine the sound source position or direction of the second interlocutor's voice (S1). Then, the sound processing unit 80b processes the second sound data to emphasize the determined voice components of the second interlocutor (S2). Thus, in the case of dialogue among the three, preferably, the three sound collection sections 41-43 on the left arm 10 and the three sound collection sections 44-46 on the right arm 20 are used independently to separate the first sound data obtained by the sound collection sections 41-43 on the left arm 10 and the second sound data obtained by the sound collection sections 44-46 on the right arm 20, and then perform sound processing on each sound data.

[0067] Furthermore, in the neck-mounted device 100 of the present invention, the suppression processing of the respective intersecting sound components can be performed together with the aforementioned sound emphasis processing. That is, the first sound data obtained by the sound collection units 41-43 on the left arm 10 is as described above as the sound component emphasized for the first speaker, but it also includes the sound component for the second speaker, etc. On the other hand, the sound component for the second speaker has already undergone emphasis processing in the second sound data obtained by the sound collection units 4-43 on the right arm 20. Therefore, as Figure 7 As shown, the sound processing unit 80b uses the emphasized voice component of the second speaker in the second sound data to suppress the voice component of the second speaker in the first sound data (S3). Similarly, the sound processing unit 80b uses the emphasized voice component of the first speaker in the first sound data to suppress the voice component of the first speaker in the second sound data (S3). Thus, in the first sound data obtained by the sound collection units 41-43 on the left arm 10, the voice component of the first speaker is emphasized, and the voice component of the second speaker is suppressed. Furthermore, in the second sound data obtained by the sound collection units 44-46 on the right arm 20, the voice component of the second speaker is emphasized, and the voice component of the first speaker is suppressed. Therefore, by utilizing only the sound collection units 41-43 of the left arm 10 and the sound collection units 44-46 of the right arm 20, the separation performance of the voice components of each speaker can be improved even when the first and second speakers are speaking simultaneously.

[0068] Furthermore, signal processing to suppress acoustic components can utilize appropriate and well-known processing methods, but the simplest is spectral subtraction. For example, subtracting the emphasized voice of the second speaker from the emphasized voice of the first speaker in the time-frequency region and using the subtraction result as the final result. However, in the case of simple spectral subtraction, a side effect known as musical noise is known, and other processing methods that can reduce this side effect can be combined. Musical noise reduction processing generally falls into the field of noise suppression processing with reference signals, and known methods include, for example, MMSE (Minimum Mean-Square-Error) or MAP (Maximum a Posteriori). In addition, processing using "adaptive filtering" included in so-called echo cancellation processing can also be used. For example, when using the MMSE or MAP method, sound emphasis processing is performed on the first 3ch audio data obtained by the sound collection units 41-43 of the left arm 10 and the second 3ch audio data obtained by the sound collection units 44-46 of the right arm 30. After obtaining a 1ch reference signal from each audio data, suppression processing is performed between the reference signals to obtain the final result in which the voice components of the first and second speakers are separated. Furthermore, when using "adaptive filtering," sound emphasis processing is performed on the first 3ch audio data obtained by the sound collection units 41-43 of the left arm 10 and the second 3ch audio data obtained by the sound collection units 44-46 of the right arm 30, and a 1ch reference signal is obtained from each audio data. Then, adaptive filtering for each channel is applied to the original 3ch audio data to generate new 3ch audio data from the original 3ch audio data after removing the voice components from different speakers. Then, this new audio data is processed with audio emphasis to obtain the final result of 1ch, in which the audio components of the first and second speakers are separated.

[0069] Furthermore, it is preferable to activate the camera unit 60 and photograph the speaker when the speaker's voice is detected. Specifically, the wearer performs a predetermined gesture with their fingers within the sensing range of the non-contact sensor unit 70. The gesture includes performing a predetermined action or making a predetermined shape with the fingers. If the sensor unit 70 detects a finger movement, the input analysis unit 80c of the control unit 80 analyzes the sensing information from the sensor unit 70 and determines whether the wearer's finger gesture matches a preset gesture. For example, since preset gestures are provided for activating the camera unit 60, starting photography with the camera unit 60, stopping photography, and other predetermined gestures for controlling the camera unit 60, the input analysis unit 80c determines whether the wearer's gesture matches these predetermined gestures based on the sensing information from the sensor unit 70.

[0070] Next, the camera control unit 80d of the control unit 80 controls the camera unit 60 based on the analysis result of the input analysis unit 80c. For example, if the input analysis unit 80c determines that the wearer's gesture matches the gesture used to start the camera unit 60, the camera control unit 80d starts the camera unit 60. Furthermore, after starting the camera unit 60, if the input analysis unit 80c determines that the wearer's gesture matches the gesture used to start photography, the camera control unit 80d controls the camera unit 60 to start capturing an image. Moreover, after photography begins, if the input analysis unit 80c determines that the wearer's gesture matches the gesture used to stop photography, the camera control unit 80d controls the camera unit 60 to stop capturing an image. Additionally, the camera control unit 80d can also put the camera unit 60 back into a sleep state after a certain period of time following the stopping of photography.

[0071] The image analysis unit 80e of the control unit 80 analyzes image data from still or moving images acquired by the camera unit 60. For example, the image analysis unit 80e can determine the distance or positional relationship between the neckband device 100 and the speaker's mouth by analyzing the image data. Furthermore, the image analysis unit 80e can determine whether the speaker is speaking by analyzing whether the speaker's mouth is open or closed based on the image data. The analysis results obtained by the image analysis unit 80e are used in the beamforming process described above. Specifically, by utilizing the analysis results of the sound data collected by each of the sound collection units 41-47, in addition to the analysis results of the image data obtained by the camera unit 60, the accuracy of determining the spatial position or direction of the speaker's mouth can be improved. Furthermore, by analyzing the movement of the speaker's mouth included in the image data to determine whether the speaker is speaking, the accuracy of emphasizing the sound emitted from the speaker's mouth can be improved.

[0072] The audio data processed by the audio processing unit 80b and the image data acquired by the camera unit 60 are stored in the storage unit 81. Furthermore, the control unit 80 can transmit the processed audio data and image data to a server device or other neckband device 100 in the cloud via the communication unit 82. The server device can also perform text-based processing, translation processing, statistical processing, and other arbitrary language processing based on the audio data received from the neckband device 100. Furthermore, the accuracy of the aforementioned language processing can be improved using the image data acquired by the camera unit 60. Additionally, the server device can utilize teacher data for machine learning from the audio data and image data received from the neckband device 100 to improve the accuracy of the learned model. Furthermore, long-distance communication can be conducted between wearers by sending and receiving audio data between the neckband devices 100. In this case, audio data can be directly sent and received between the neckband devices 100 via short-range wireless communication, or via the Internet through the server device.

[0073] This application primarily describes an embodiment where the neckband device 100 is configured with a sound analysis unit 80a, a sound processing unit 80b, and an image analysis unit 80e, and performs beamforming processing locally. However, a server connected to the cloud via the Internet can also perform the functions of any one or all of the sound analysis unit 80a, the sound processing unit 80b, and the image analysis unit 80e. In this case, for example, the neckband device 100 can transmit sound data acquired by each of the sound collection units 41-47 to the server device, where the server device determines the location or direction of the sound source, or performs sound processing that emphasizes the voice of the wearer or the person speaking and suppresses other noise. Furthermore, image data acquired by the camera unit 60 can also be transmitted from the neckband device 100 to the server device, where the server device performs image data analysis processing. In this case, the neckband device 100 and the server device constitute a sound processing system.

[0074] In the above description of this application, in order to illustrate the content of the present invention, reference is made to the appendix. Figure 1 The embodiments of the present invention have been described. However, the present invention is not limited to the above embodiments, but includes modifications or improvements that are obvious to those skilled in the art based on the matters described in this application specification.

[0075] Furthermore, the imaging method performed by the camera unit 60 can be controlled based on the sensing information obtained by the sensing unit 70. Specifically, the imaging method of the camera unit 60 can include, for example, still image photography, video photography, slow motion photography, panoramic photography, time-lapse photography, and time-lapse photography. If the sensing unit 70 senses a finger movement, the input analysis unit 80c of the control unit 80 analyzes the sensing information of the sensing unit 70 and determines whether the wearer's finger gesture is consistent with a preset gesture. For example, if the imaging method of the camera unit 60 has a fixed gesture, the input analysis unit 80c will determine whether the wearer's gesture is consistent with the preset gesture based on the sensing information of the sensing unit 70. The camera control unit 80d controls the imaging method performed by the camera unit 60 based on the analysis result of the input analysis unit 80c. For example, if the input analysis unit 80c determines that the wearer's gesture is consistent with the gesture used for photographing a still image, the camera control unit 80d controls the camera unit 60 to perform still image photography. Alternatively, if the input analysis unit 80c determines that the wearer's gesture matches the gesture used for video recording, the camera control unit 80d controls the camera unit 60 to record video. In this way, the recording method performed by the camera unit 60 can be specified based on the wearer's gesture.

[0076] Furthermore, in the above-described embodiment, although the camera unit 60 is primarily controlled based on the sensing information obtained by the sensor unit 70, the sound collection units 41 to 47 can also be controlled based on the sensing information obtained by the sensor unit 70. For example, a pre-set inherent gesture related to starting or stopping sound collection by the sound collection units 41 to 47 can be used, and the input analysis unit 80c determines whether the wearer's gesture matches the pre-set gesture based on the sensing information from the sensor unit 70. Then, when a gesture related to starting or stopping sound collection is sensed, sound collection through each sound collection unit 41 to 47 can be started or stopped simply based on the sensing information of that gesture.

[0077] Furthermore, in the aforementioned embodiment, although the camera unit 60 is mainly controlled based on the sensing information obtained by the sensing unit 70, it can also be controlled based on the sound information input to each of the sound collection units 41-47. Specifically, the sound analysis unit 80a analyzes the sound obtained by the sound collection units 41-47. That is, it performs voice recognition of the wearer or the person speaking, and determines whether their voice is related to the control of the camera unit 60. Then, the camera control unit 80d controls the camera unit 60 based on the sound analysis results. For example, in the case of a predetermined sound input collection unit 41-47 related to the start of photography, the camera control unit 80d starts the camera unit 60 and begins photography. Furthermore, in the case of a predetermined sound input collection unit 41-47 specifying the photography method to be performed by the camera unit 60, the camera control unit 80d controls the camera unit 60 to execute the specified photography method. Furthermore, after the sound collection units 41 to 47 are activated based on the sensing information obtained by the sensing unit 70, the camera unit 60 can be controlled based on the sound information input to the sound collection units 41 to 47.

[0078] Furthermore, the content of control commands based on input information from the sensor unit 70 can be changed according to the images captured by the camera unit 60. Specifically, firstly, the image analysis unit 80e analyzes the images acquired by the camera unit 60. For example, based on feature points included in the image, the image analysis unit 80a determines whether an image of a person was captured, whether an image of a specific subject (artificial or natural object, etc.) was captured, or the conditions under which the image was captured (location, time of capture, weather, etc.). Furthermore, for individuals included in the image, they can be categorized by gender or age, or specific individuals can be identified.

[0079] Next, based on the type of image (person, subject, situation category), the pattern of the control command based on the gesture made by a person's fingers is stored in the storage unit 81. At this time, even the same gesture can result in different control commands depending on the type of image. Specifically, even the same gesture may become a control command to focus on the face of the person in the image, while it may become a control command to panoramically photograph the area around the natural object in the image. Furthermore, the definition of the gesture can be different by sensing the gender or age of the person in the image, whether the subject is an artificial or natural object, or the location, time, and weather of the image. Then, the input analysis unit 80c refers to the image analysis result of the image analysis unit 80e, determines the definition content corresponding to the image analysis result for the gesture sensed by the sensing unit 70, and generates a control command to be input into the neck-mounted device 100. In this way, by changing the definition content of the gesture according to the content of the image, various changing control commands can be input into the device through gestures according to the shooting situation or purpose of the image.

[0080] Explanation of reference numerals in the attached figures

[0081] 10: Left arm (first arm)

[0082] 11: Flexible part

[0083] 12: Front end

[0084] 13: Lower surface

[0085] 14: Upper surface

[0086] 20: Right arm (second arm)

[0087] 21: Flexible part

[0088] 22: Front end

[0089] 23: Lower surface

[0090] 24: Upper surface

[0091] 30: Main body

[0092] 31: Drooping part

[0093] 41: Episode 1 Sound Department

[0094] 42: Episode 2 Sound Department

[0095] 43: Episode 3 Sound Department

[0096] 44: Episode 4 Sound Department

[0097] 45: Episode 5 Sound Department

[0098] 46: Episode 6 Sound Department

[0099] 47: Episode 7 Music Department

[0100] 50: Operations Department

[0101] 60: Camera Department

[0102] 70: Sensor Department

[0103] 80: Control Department

[0104] 80a: Sound Analysis Section

[0105] 80b: Sound Processing Department

[0106] 80c: Input parsing unit

[0107] 80D: Camera Control Department

[0108] 80e: Image Analysis Unit

[0109] 81: Storage Department

[0110] 82: Ministry of Communications

[0111] 83: Proximity sensor

[0112] 84: Sound Reproduction Section

[0113] 90: Battery

[0114] 100: Neckband-type device (voice input device)

Claims

1. A sound input device, characterized in that, have: The first arm and the second arm can be positioned to hold the target sound source; Multiple sound-collecting parts, each having three or more locations on each of the first arm and the second arm; A sound analysis unit determines the spatial position or direction of the sound source emitting the sound based on the sounds acquired by each sound collecting unit; and The sound processing unit, based on the location or direction of the sound source determined by the sound analysis unit, performs processing to emphasize and suppress sound components included in the sound data obtained by the sound collection unit. The sound analysis unit determines the spatial position or direction of each sound source by using the sound collected by the sound collecting unit located on the first arm and the sound collected by the sound collecting unit located on the second arm. It determines whether the sound source determined solely based on the first sound data obtained from the plurality of sound collecting units disposed on the first arm matches the mouth of the first interlocutor located on the first arm side of the wearer, and further determines whether the sound source determined solely based on the second sound data obtained solely based on the plurality of sound collecting units disposed on the second arm matches the mouth of the second interlocutor located on the second arm side of the wearer. The sound processing unit performs the following processing: The first audio data is processed to emphasize the audio components of the first speaker determined by the audio analysis unit; The second audio data is processed to emphasize the audio components of the second speaker determined by the audio analysis unit; Processing that uses the voice components of the second speaker that have been emphasized in the second voice data to suppress the voice components of the second speaker in the first voice data; as well as Processing that uses the voice components of the first speaker that have been emphasized in the first voice data to suppress the voice components of the first speaker in the second voice data.

2. The sound input device according to claim 1, characterized in that, The sound input device is a neckband-type device. The sound source is the mouth of the wearer of the sound input device.

3. The sound input device according to claim 1, characterized in that, The sound input device is a neckband-type device. One or more sound-collecting parts are further provided at a position corresponding to the back of the wearer's neck.

4. The sound input device according to claim 1, characterized in that, The audio input device further includes a camera unit. The sound analysis unit determines whether the sound source determined solely based on the sound data obtained by the plurality of sound collecting units provided on the first arm matches the mouth of the first interlocutor located on the first arm side of the wearer as determined by the camera unit, and determines whether the sound source determined solely based on the sound data obtained solely based on the plurality of sound collecting units provided on the second arm matches the mouth of the second interlocutor located on the second arm side of the wearer as determined by the camera unit.

Images