Sound output device, sound output method, and sound output program

Abstract

To make it possible to output, from sound wave output means, a sound wave adjusted to be easily heard by a user.SOLUTION: A sound wave output device is configured to: generate sound wave characteristic information indicating characteristics of a sound wave reflected in the interior of the ears by sound wave output means for converting an acoustic signal into a sound wave and outputting it, among sound waves heard by a user; and control characteristics of the acoustic signal based on a voice of a telephone conversion partner so that a deviation becomes smaller between frequency characteristics indicated by the sound wave characteristic information and frequency characteristics indicated by reference sound characteristic information related to a reference sound having a frequency component that the user can hear, which are characteristics of a sound wave in which the sound wave of the reference sound is reflected in the interior of the ears of the user and is set to be easily heard by the user, before being input to the sound wave output means via amplification means.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a sound wave output device, a sound wave output method, and a sound wave output program.

Background Art

[0002] When listening to a speaker's voice without face-to-face interaction, or in an online meeting, since the volume and frequency of the voice vary for each speaker, the listener who listens to the speaker's voice may have difficulty in hearing the speaker's voice. In addition, there are individual differences in the internal functions of the listener's ears and the way of feeling sound, and the frequency components or volumes of sound waves that are easy for the listener to hear vary depending on the listener. Therefore, there is a problem that it is difficult for a listener who listens to a speaker's voice without face-to-face interaction to hear the speaker's voice in an easy-to-hear state.

[0003] Patent Document 1 describes a communication device that identifies a speaker and makes the voice uttered by the identified speaker easier to hear than that of other speakers. Specifically, the communication device described in Patent Document 1 changes the voice signal processing of the voice to the voice signal processing preset for each speaker, and outputs the voice subjected to the changed voice signal processing.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method described in Patent Document 1, there are cases where it is difficult for the listener to hear the voice processed by the voice signal processing set for each speaker. This is because, as described above, there are individual differences in the functions inside the listener's ear and the way of perceiving sound, and the frequency components or volumes of sound waves that the listener feels easy to hear are different depending on the listener. Therefore, when it is difficult for the listener to hear the voice, if the settings of the voice signal processing are not changed by the listener, the communication device cannot make it easier to hear the speaker's voice. However, in the method described in Patent Document 1, the communication device cannot output sound waves adjusted to be easy for the listener to hear.

[0006] An example of the object of the present invention is to provide a sound wave output device, a sound wave output method, and a sound wave output program that enable a sound wave adjusted to be easy for a user to hear to be output from a sound wave output means.

Means for Solving the Problems

[0007] In one aspect of the present invention, a sound wave output device includes: a generation means for generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves that a user hears, by a sound wave output means that converts an acoustic signal into a sound wave and outputs it; a frequency characteristic indicated by the sound wave characteristic information; and a frequency characteristic indicated by reference sound characteristic information regarding a reference sound having a frequency component that the user can hear, wherein the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear, and a control means for controlling the characteristics of the acoustic signal based on the voice of the communication partner so that the deviation from the characteristics of the sound wave is reduced, and inputting it to the sound wave output means via an amplification means.

[0008] Also, in another aspect of the present invention, the sound wave output method generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves that the user can hear by the sound wave output means that converts an acoustic signal into a sound wave and outputs it, and the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having a frequency component that the user can hear, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear. The characteristics of the acoustic signal based on the voice of the other party in the call are controlled so that the deviation from the characteristics of the sound wave is reduced, and the acoustic signal is input to the sound wave output means via the amplification means.

[0009] Also, in another aspect of the present invention, the sound wave output program causes a computer to generate a generation function for generating sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves that the user can hear by the sound wave output means that converts an acoustic signal into a sound wave and outputs it, and the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having a frequency component that the user can hear, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear. The control function controls the characteristics of the acoustic signal based on the voice of the other party in the call so that the deviation from the characteristics of the sound wave is reduced, and inputs the acoustic signal to the sound wave output means via the amplification means.

Advantages of the Invention

[0010] With the sound wave output device, sound wave output method, and sound wave output program of the present invention, it becomes possible to output from the sound wave output means a sound wave adjusted to be easy for the user to hear.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0012] [First Embodiment] The first embodiment of the present invention will be described.

[0013] FIG. 1 is a block diagram showing a configuration example of the sound wave output device 1 of the present embodiment.

[0014] The sound wave output device 1 of the present embodiment includes a generation unit 11 and a control unit 12.

[0015] For example, the sound wave output device 1 is connected to an earphone (not shown) having a sound wave output means (not shown) for converting an acoustic signal into a sound wave and outputting it, and a sound wave input means (not shown) for receiving an input of a sound wave reflected from inside the user's ear. Alternatively, the sound wave output device 1 is connected to a headphone having a sound wave output means for converting an acoustic signal into a sound wave and outputting it, and a sound wave input means for receiving an input of a sound wave reflected from inside the user's ear. The sound wave output means is, for example, a speaker. The sound wave output means converts an acoustic signal into a sound wave and outputs it. The acoustic signal input to the sound wave output means is, for example, a voice signal indicating the voice of a speaker other than the user. The voice of the speaker is the raw voice during a call or a meeting, the voice recorded in the memory, or the voice by the voice signal received through the network. The sound wave input means is, for example, a microphone.

[0016] The generation unit 11 generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the user's ear among the sound waves that the user hears by the sound wave output means that converts an acoustic signal into a sound wave and outputs it. Specifically, the generation unit 11 generates sound wave characteristic information from the output of the microphone to which the sound wave reflected inside the ear is input.

[0017] The control unit 12 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having frequency components that the user can hear becomes small. The frequency characteristics indicated by the reference sound characteristic information regarding the reference sound are the characteristics of the sound wave that is set to be easily heard by the user when the sound wave of the reference sound is reflected inside the user's ear. The control unit 12 inputs the acoustic signal with controlled characteristics to the sound wave output means via an amplification means (not shown).

[0018] In this way, the sound wave output device 1 generates sound wave characteristic information of the sound wave reflected inside the ear among the sound waves that the user hears by the sound wave output means that converts and outputs an acoustic signal into a sound wave. The sound wave output device 1 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear, becomes small. The sound wave output device 1 inputs the acoustic signal with controlled characteristics to the sound wave output means via the amplification means. As a result, the sound wave output means can output a sound wave by the acoustic signal whose characteristics are adjusted according to the reference sound characteristic information to the user.

[0019] Next, with reference to FIG. 2, an operation example of the sound wave output device 1 of the present embodiment will be described. FIG. 2 is a flowchart showing an operation example of the sound wave output device 1.

[0020] The generation unit 11 generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves that the user hears by the sound wave output means that converts and outputs an acoustic signal into a sound wave (step S101).

[0021] The control unit 12 controls the characteristics of the acoustic signal based on the voice of the call partner and inputs it to the sound wave output means via the amplification means (step S102). In step S102, the control unit 12 controls the characteristics of the acoustic signal so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information becomes small.

[0022] In this way, the sound wave output device 1 inputs the acoustic signal with controlled characteristics to the sound wave output means so that the deviation between the sound wave that is easy for the user to hear and the characteristics of the sound wave reflected inside the user's ear becomes small. Therefore, the sound wave output device 1 can cause the sound wave output means to output a sound wave adjusted to be easy for the user to hear.

[0023] [Second Embodiment] Next, the sound wave output device 3 in the second embodiment of the present invention will be specifically described.

[0024] FIG. 3 is a block diagram showing a configuration example of a sound wave output system according to a second embodiment of the present invention. In the second embodiment, the sound wave output system includes a sound wave output device 3. In the second embodiment, the generation unit 31 corresponds to, for example, the generation unit 11 of the first embodiment. In the second embodiment, the control unit 32, the determination unit 36, and the reference sound characteristic information storage unit 37 correspond to, for example, the control unit 12 of the first embodiment. The second embodiment is different from the sound wave output device 1 of the first embodiment in that the sound wave output device 3 further includes a sound wave output unit 33, a reception unit 34, and a sound wave input unit 35.

[0025] The sound wave output system of this embodiment is a conference communication system. Before the start of a conference, the sound wave output device 3 performs an initial setting for generating reference sound characteristic information indicating the characteristics of sound waves reflected inside the user's ear that are easy for the user to hear. When performing the initial setting described later, the sound wave output device 3 is connected to the external device 2 by wireless communication or wired communication as shown in FIG. 3. The external device 2 may be disconnected from the sound wave output device 3 after the initial setting of the sound wave output device 3 is completed. That is, the external device 2 may be separated from the sound wave output device 3.

[0026] In addition, the sound wave output device 3 is connected to a sound wave input device (in this example, the sound wave input device 4-1) that receives the input of the voice of the user (user A1) by wired communication or wireless communication. Also, as shown in FIG. 3, the sound wave output device 3 is connected to sound wave input devices (in this example, the sound wave input devices 4-2, ···, 4-n (n is an integer of 2 or more)) that receive the input of the voices of other users (users A2 to An) by wired communication or wireless communication. The sound wave input devices (in this example, the sound wave input devices 4-2 to 4-n) that receive the input of the voices of other users (users A2 to An) may be connected to the sound wave output device 3 after the above-described initial setting is completed.

[0027] The external device 2 performs initial settings for the sound wave output device 3. The external device 2 is, for example, a smartphone. Alternatively, the external device 2 may be a computer. Application software that realizes the functions of the input / output unit 21, the transmission unit 22, and the determination unit 23, which will be described later, is pre-installed in the external device 2. Details of the initial settings using the external device 2 will be described later.

[0028] The sound wave output device 3 is, for example, headphones. Alternatively, the sound wave output device 3 may be earphones. Thus, in this embodiment, the sound wave output device 3 is built into headphones or earphones.

[0029] The sound wave output device 3 is used, for example, in an online meeting or a meeting by a call between two or more people. For example, each participant in the online meeting uses the same sound wave output device as the sound wave output device 3 of user A1 and a sound wave input device that receives voice input. In this example, among the participants in the online meeting, user A1 of the sound wave output device 3 inputs voice using the sound wave input device 4-1. Also, each of the other users (user A2 to user An) inputs voice using any one of the sound wave input devices 4-2 to 4-n. The receiving unit 34 of the sound wave output device 3 receives a voice signal, which is a signal corresponding to the voice, from the sound wave input devices (each of the sound wave input devices 4-2 to 4-n in this example) that receive the voice input of the other users.

[0030] Note that the receiving unit 34 may receive a voice signal corresponding to the voice from the sound wave input device (the sound wave input device 4-1 in this example) that receives the voice input of user A1. The receiving unit 34 may transmit the voice signal of user A1 to the sound wave output devices of the other users (user A2 to user An). Note that in this embodiment, the sound wave input device 4-1 transmits a voice signal to the sound wave output devices of the other users (user A2 to user An).

[0031] The sound wave output device 3 of the present embodiment performs the above-described initial settings. In the initial settings, based on an output request notification for requesting the output of a reference sound for initial settings from the external device 2, the control unit 32 causes the sound wave output unit 33 to output a reference sound. The reference sound is a sound wave used for the user to determine whether it is easy to hear. Also, the reference sound is a plurality of types of sound waves having different frequencies and volumes from each other. The sound wave input unit 35 inputs, for each of the reference sounds, the sound wave output from the sound wave output unit 33 into the ear and reflected inside the ear, and outputs an input acoustic signal indicating the input sound wave to the generation unit 31. In the initial settings, the reception unit 34 receives, for each of the reference sounds, evaluation information indicating whether it is easy to hear from the external device 2. The evaluation information includes information indicating the result of the user's determination as to whether the reference sound is easy to hear. The generation unit 31, which will be described later, stores the evaluation information and the reference sound characteristic information in the reference sound characteristic information storage unit 37. The reference sound characteristic information may include the evaluation information. The reference sound characteristic information includes output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds. In the initial settings, the sound wave output device 3 stores, in the reference sound characteristic information storage unit 37, reference sound characteristic information including frequency information indicating the frequency characteristics of the sound wave reflected inside the user's ear for the reference sound. When used in an online meeting, the sound wave output device 3 controls the characteristics of the acoustic signal.

[0032] Also, when the initial settings are completed and the online meeting starts, the receiving unit 34 of the sound wave output device 3 receives a voice signal, which is an acoustic signal based on the voices of other users A2 to An who are the call partners, from the sound wave input devices 4-2 to 4-n. The acoustic signal is input to the sound wave output unit 33 via the control unit 32. Immediately after the start of the online meeting, the control unit 32 has not yet controlled the characteristics of the acoustic signal so that the divergence between the characteristics of the sound wave that user A1 is likely to hear and the characteristics of the sound wave reflected inside the user's ear is small. The sound wave output unit 33 converts the acoustic signal input from the control unit 32 into a sound wave and outputs it inside user A1's ear. The sound wave input unit 35 inputs the sound wave of the call partner that is output inside the ear from the sound wave output unit 33 and reflected inside the ear, and outputs it to the generation unit 31. The generation unit 31 generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves output inside user A1's ear by the sound wave output unit 33. Next, the sound wave output device 3 executes the following process using the determination unit 36, the reference sound characteristic information storage unit 37, and the control unit 32. The process is to control the characteristics of the acoustic signal so that the divergence between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are the characteristics of the sound wave that user A1 is likely to hear and is reflected inside the user's ear, is small. The details of the process will be described later. The control unit 32 inputs the acoustic signal with controlled characteristics to the sound wave output unit 33 via an amplification means (not shown).

[0033] [Configuration of Sound Wave Input Device] The sound wave input device 4-1 includes a sound wave input unit 41-1 and a transmission unit 42-1. The sound wave input device 4-2 includes a sound wave input unit 41-2 and a transmission unit 42-2. The sound wave input device 4-n includes a sound wave input unit 41-n and a transmission unit 42-n. Thus, each of the sound wave input devices 4-1 to 4-n includes a sound wave input unit and a transmission unit.

[0034] The sound wave input devices 4-1 to 4-n are, for example, smartphones. Alternatively, the sound wave input device may be built into an external device. For example, the sound wave input device 4-1 may be built into the external device 2 used by the user A1. Specifically, the external device 2 may further have the function of the sound wave input unit 41-1 described later, and the transmission unit 22 described later may realize the function of the transmission unit 42-1 of the sound wave input device 4-1 described later.

[0035] Each of the sound wave input devices 4-1 to 4-n receives the input of the voices of the participants (users A1 to An) in the online meeting and outputs a voice signal, which is a signal corresponding to the voice.

[0036] Here, regarding the configuration of the sound wave input devices 4-1 to 4-n of the present embodiment, the sound wave input device 4-1 will be described in detail as a representative.

[0037] The sound wave input unit 41-1 receives the input of the voice of the user A1 and generates a voice signal, which is a signal corresponding to the voice. The sound wave input unit 41-1 outputs the generated voice signal to the transmission unit 42-1.

[0038] The voice signal of the user A1 is input from the sound wave input unit 41-1 to the transmission unit 42-1. The transmission unit 42-1 transmits the input voice signal to a sound wave output device (in this example, a sound wave output device other than the sound wave output device 3 (not shown)) used by other users participating in the online meeting. The sound wave input device 4-1 that receives the input of the voice of the user A1 does not output the voice signal to the sound wave output device 3 used by the user A1.

[0039] [Configuration of External Device 2] Referring to FIG. 3, the configuration of the external device 2 of the present embodiment will be described in detail. The external device 2 includes an input / output unit 21, a transmission unit 22, and a determination unit 23.

[0040] The input / output unit 21 receives the input of an instruction to start the initial setting in response to the operation of the user. The input / output unit 21 outputs the instruction to start the initial setting to the transmission unit 22.

[0041] An instruction to start the initial setting is input from the input / output unit 21 to the transmission unit 22. When an instruction to start the initial setting is input, the transmission unit 22 transmits an output request notification to the sound wave output device 3, which requests the output of sound waves of a reference sound with an initial value for the initial setting. The reference sound is a plurality of types of sound waves with different frequencies and volumes from each other.

[0042] Frequency information about the sound waves of the reference sound and volume information of the reference sound are input from the transmission unit 22 to the determination unit 23. The frequency information is information indicating one or a plurality of frequencies. This frequency information and volume information are characteristic information of the reference sound for which evaluation information was transmitted, input by the transmission unit 22 when the evaluation information described later is transmitted to the sound wave output device 3. When the frequency information and the volume information are input, the determination unit 23 determines whether evaluation information has been generated for all the reference sounds. When it is determined that evaluation information has been generated for all the reference sounds, the determination unit 23 outputs an end notification of the initial setting to the transmission unit 22. Also, if there is a reference sound for which evaluation information has not been generated among the sound waves of all the reference sounds, the determination unit 23 outputs, to the transmission unit 22, the frequency information and the volume information of the reference sound for which evaluation information has not been generated, in association with each other.

[0043] Frequency information and volume information are input from the determination unit 23 to the transmission unit 22. The transmission unit 22 transmits an output request notification for the initial setting, including the input frequency information and volume information, to the sound wave output device 3. Thereby, the external device 2 causes the sound wave output unit 33 of the sound wave output device 3 to output the sound waves of the reference sound.

[0044] Also, an end notification of the initial setting is input from the determination unit 23 to the transmission unit 22. The transmission unit 22 transmits the end notification of the initial setting to the sound wave output device 3.

[0045] Note that when an instruction to start the initial setting is input, the input / output unit 21 may receive an input of reference sound characteristic information according to the operation of the user. When the reference sound characteristic information is input to the input / output unit 21, the transmission unit 22 may transmit the reference sound characteristic information to the sound wave output device 3.

[0046] By transmitting an output requirement notification, a sound wave of a reference sound is output from the sound wave output unit 33 of the sound wave output device 3 into the ear of user A1. The input / output unit 21 of the external device 2 receives, for each sound wave of a plurality of reference sounds, an operation input regarding an evaluation indicating whether the reference sound is easy to hear according to the operation of the user, and generates evaluation information corresponding to the operation input. Note that the evaluation information may include the following information. The evaluation information may include information indicating whether the user can easily hear the pitch of the reference sound output from the sound wave output unit 33. Further, when the evaluation information is information indicating that it is difficult to hear, the input / output unit 21 may include, in the evaluation information, information indicating that the volume of the sound wave is large or information indicating that the volume is small. The input / output unit 21 outputs the generated evaluation information to the transmission unit 22.

[0047] The transmission unit 22 transmits the generated evaluation information to the sound wave output device 3. Further, when transmitting the evaluation information to the sound wave output device 3, the transmission unit 22 outputs, to the determination unit 23, by associating the frequency information and the volume information included in the output requirement notification transmitted immediately before the evaluation information is generated.

[0048] [Configuration of Sound Wave Output Device 3] Next, with reference to FIG. 3, the configuration of the sound wave output device 3 of the present embodiment will be described in detail. The sound wave output device 3 includes a generation unit 31 and a control unit 32.

[0049] The sound wave output unit 33 is connected to the control unit 32. The reception unit 34 is connected to the generation unit 31 and the control unit 32. The sound wave input unit 35 is connected to the generation unit 31. The determination unit 36 is connected to the generation unit 31, the control unit 32, and the reference sound characteristic information storage unit 37. The reference sound characteristic information storage unit 37 is connected to the generation unit 31 and the determination unit 36.

[0050] The sound wave output device 3 may be implemented as two or more devices. For example, among the components of the sound wave output device 3, the sound wave output unit 33 and the sound wave input unit 35 may be earphones or headphones, and the other components may be implemented as an information processing device. Further, in the earphones or headphones, the left ear sound wave input unit 351 and the left ear sound wave output unit 331 may be integrally mounted, and the right ear sound wave input unit 352 and the right ear sound wave output unit 332 may be integrally mounted.

[0051] First, each component of the sound wave output device 3 of the present embodiment will be described in detail for the case of performing initial settings.

[0052] The receiving unit 34 receives an output request notification for initial settings including frequency information and volume information. The receiving unit 34 outputs the frequency information and the volume information to the control unit 32 in association with each other.

[0053] The control unit 32 inputs an acoustic signal for outputting a sound wave of a reference tone at the frequency indicated by the input frequency information and the volume indicated by the input volume information to the sound wave output unit 33 via the amplification means.

[0054] The sound wave output unit 33 includes a left ear sound wave output unit 331 and a right ear sound wave output unit 332.

[0055] An acoustic signal for outputting a sound wave of a reference tone is input to the left ear sound wave output unit 331 from the control unit 32. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the user's left ear. For example, the left ear sound wave output unit 331 is a left ear speaker that converts an electrical signal into a sound wave.

[0056] An acoustic signal for outputting a sound wave of a reference tone is input to the right ear sound wave output unit 332 from the control unit 32. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the user's right ear. For example, the right ear sound wave output unit 332 is a right ear speaker.

[0057] The sound wave input unit 35 includes a left ear sound wave input unit 351 and a right ear sound wave input unit 352.

[0058] The left-ear sound wave input unit 351 inputs sound waves reflected inside the left ear of the user and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. For example, the input acoustic signal is a signal indicating the waveform of the sound waves reflected inside the ear. For example, the left-ear sound wave input unit 351 is a microphone for the left ear. Specifically, the left-ear sound wave input unit 351 is a bone conduction microphone for the left ear.

[0059] The right-ear sound wave input unit 352 inputs sound waves reflected inside the right ear of the user and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. For example, the right-ear sound wave input unit 352 is a microphone for the right ear. Specifically, the right-ear sound wave input unit 352 is a bone conduction microphone for the right ear.

[0060] In the case of initial setting, an input acoustic signal regarding the acoustic signal of the reference sound, which is generated by the control unit 32 based on the frequency information and volume information transmitted from the transmission unit 22, is input to the generation unit 31 from the sound wave input unit 35. The generation unit 31 analyzes the strength of the signals of each frequency component constituting the input acoustic signal. The generation unit 31 performs frequency decomposition of the input acoustic signal and generates reflected sound wave characteristic information indicating a frequency characteristic including information indicating a frequency distribution or information indicating a frequency component according to the analysis result. The reflected sound wave characteristic information is frequency information indicating the frequency characteristic of the sound waves reflected inside the user's ear by the reference sound. The generation unit 31 outputs sound wave characteristic information including frequency information. For example, the sound wave characteristic information includes frequency information indicating at least one frequency. The generation unit 31 also detects the volume at that frequency based on the intensity of the amplitude of the input acoustic signal. Therefore, the reflected sound wave characteristic information of the present embodiment includes volume information indicating the volume of the sound waves indicated by the input acoustic signal.

[0061] Note that the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352 may generate reflected sound wave characteristic information. When the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352 generate reflected sound wave characteristic information, the generation unit 31 may obtain sound wave characteristic information from each of the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352 instead of generating the reflected sound wave characteristic information.

[0062] Also, the receiving unit 34 receives evaluation information. The receiving unit 34 associates the evaluation information with the frequency information and the volume information included in the output request notification received before receiving the evaluation information, and outputs the result to the generation unit 31.

[0063] The generation unit 31 receives the frequency information, the volume information, and the evaluation information from the receiving unit 34. The frequency information and the volume information input from the receiving unit 34 are output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output.

[0064] Also, the receiving unit 34 receives an end notification of the initial setting from the external device 2. When receiving the end notification of the initial setting, the receiving unit 34 outputs the end notification of the initial setting to the generation unit 31.

[0065] When the end notification of the initial setting is input, the generation unit 31 performs the following operation. The generation unit 31 stores, as reference sound characteristic information, the output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output and the reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds in the reference sound characteristic information storage unit 37. In the present embodiment, the generation unit 31 stores the reference sound characteristic information and the evaluation information in association with each other in the reference sound characteristic information storage unit 37.

[0066] In the reference sound characteristic information storage unit 37, evaluation information and reference sound characteristic information are associated and stored by the generation unit 31. In the reference sound characteristic information storage unit 37, at least the reference sound characteristic information associated with the evaluation information indicating easy-to-listen is stored. Specifically, in the reference sound characteristic information storage unit 37 of the present embodiment, the reference sound characteristic information associated with the evaluation information indicating easy-to-listen and the reference sound characteristic information associated with the evaluation information indicating difficult-to-listen are stored. The reference sound characteristic information includes output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output. Further, the reference sound characteristic information includes reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds.

[0067] Note that the reference sound characteristic information storage unit 37 may store the reference sound characteristic information for the left ear and the reference sound characteristic information for the right ear. When the reference sound characteristic information for the left ear and the reference sound characteristic information for the right ear are stored in the reference sound characteristic information storage unit 37, the control unit 32 may operate as follows. The control unit 32 may control the characteristics of the acoustic signal for the left ear sound wave output unit 331 based on the reference sound characteristic information for the left ear, and control the characteristics of the acoustic signal for the right ear sound wave output unit 332 based on the reference sound characteristic information for the right ear. In this way, the sound wave output device 3 may individually control each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 when performing the initial setting and when used in an online meeting.

[0068] Next, each configuration of the sound wave output device 3 of the present embodiment will be described in detail for the case of being used in an online meeting.

[0069] The receiving unit 34 receives an audio signal from a sound wave input device (in this example, sound wave input devices 4-2 to 4-n). The receiving unit 34 outputs the audio signal to the control unit 32.

[0070] An audio signal is input from the receiving unit 34 to the control unit 32. Further, a notification requesting control of the characteristics of the acoustic signal is input from the determination unit 36 to the control unit 32. When no notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 inputs, via the amplification means, an acoustic signal for outputting a sound wave based on the audio signal to the sound wave output unit 33. When a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the receiving unit 34. Specifically, the control unit 32 converts the audio signal into a signal or data in the frequency domain via a Fourier transform circuit. The control unit 32 changes the characteristics of the acoustic signal based on the voice of the call partner so that the frequency characteristics indicated by the reference sound characteristic information, which is the sound wave that the user can easily hear, has a small deviation from the characteristics of the sound wave reflected inside the user's ear. For example, the frequency and its magnitude, the volume, are changed. After that, the control unit 32 restores the signal or data in the frequency domain, whose characteristics are controlled by the change, to the audio signal on the time axis by means of an inverse Fourier transform circuit. Then, the control unit 32 inputs, via the amplification means, the acoustic signal, which is the audio signal with controlled characteristics, to the sound wave output unit 33.

[0071] For example, when a sound wave with frequency characteristics that are difficult for the user to hear is output from the sound wave output unit 33, a notification requesting control of the characteristics of the acoustic signal is input from the determination unit 36. The control unit 32 controls the characteristics of the acoustic signal with respect to the audio signal input from the receiving unit 34 so that the frequency characteristics indicated by the reference sound characteristic information, which is the sound wave that the user can easily hear, has a small deviation from the characteristics of the sound wave reflected inside the user's ear. Specifically, the control unit 32 controls the acoustic signal so that the deviation of the frequency characteristics of the sound wave characteristic information from the frequency characteristics of the reference sound characteristic information becomes small. For example, as the frequency characteristics of the sound wave characteristic information, when the intensity of the signal of a certain frequency component is weaker than the intensity of the signal of the frequency component of the sound wave that the user can easily hear and is reflected inside the user's ear, the control unit 32 controls the characteristics of the acoustic signal as follows. The control unit 32 controls the characteristics of the acoustic signal so as to increase the intensity of the signal of a certain frequency component.

[0072] The sound wave output unit 33 converts the acoustic signal input from the control unit 32 into a sound wave and outputs it.

[0073] The sound wave input unit 35 inputs the sound wave reflected inside the user's ear and outputs an input acoustic signal indicating the input sound wave to the generation unit 31.

[0074] The generation unit 31 generates sound wave characteristic information according to the analysis result of the strength of the signals of each frequency component constituting the input acoustic signal input from the sound wave input unit 35. The generation unit 31 outputs the generated sound wave characteristic information to the determination unit 36.

[0075] The determination unit 36 determines, based on the input sound wave characteristic information, whether a sound wave that is easy for the user to hear is output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. When the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which is considered easy for the user to hear, and the frequency characteristic indicated by the sound wave characteristic information deviate by a predetermined threshold or more, the determination unit 36 makes the following determination. The determination unit 36 determines that a sound wave that is difficult for the user to hear is output by the sound wave output unit 33. When it is determined that a sound wave that is difficult for the user to hear is output by the sound wave output unit 33, the determination unit 36 performs the following operation. The determination unit 36 identifies the output acoustic information associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 and associated with the evaluation information indicating easy-to-hear for the frequency characteristic indicated by the sound wave characteristic information determined to be difficult to hear. The determination unit 36 outputs a notification requesting control of the characteristics of the acoustic signal to the control unit 32. The notification requesting control of the characteristics of the acoustic signal includes the identified output acoustic information.

[0076] Further, when the deviation between the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information, which the user has determined to be easy to hear and stored in the reference sound characteristic information storage unit 37, and the frequency characteristics indicated by the sound wave characteristic information is less than a predetermined threshold, the determination unit 36 makes the following determination. The determination unit 36 determines that a sound wave that is easy for the user to hear is being output by the sound wave output unit 33. When it is determined that a sound wave that is easy for the user to hear is being output by the sound wave output unit 33, the process of outputting a notification requesting control of the characteristics of the acoustic signal to the control unit 32 is not performed.

[0077] As described above, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 controls the characteristics of the acoustic signal so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information becomes smaller, and inputs the controlled signal to the sound wave output unit 33 via the amplification means. Specifically, the control unit 32 controls the characteristics of the acoustic signal so that the deviation from the frequency characteristics indicated by the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal becomes smaller.

[0078] Note that the sound wave output device 3 may output a notification requesting control of the characteristics of an acoustic signal including two or more pieces of output acoustic information to the control unit 32 regarding the sound wave characteristic information of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. The control unit 32 may control the acoustic signal so that the frequency characteristics of the acoustic signal become the frequency characteristics corresponding to the frequency characteristics of the sound wave characteristic information and the frequency characteristics of the reference sound characteristic information based on the frequency characteristics of each of the plurality of pieces of output acoustic information.

[0079] As described above, the sound wave output device 3 generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the user's ear among the sound waves that the user can hear, by the sound wave output unit 33 that converts an acoustic signal into sound waves and outputs them. The sound wave output device 3 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear, becomes small. Also, the sound wave output device 3 inputs the acoustic signal with controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output to the user a sound wave generated by the acoustic signal whose characteristics are adjusted according to the reference sound characteristic information. Thereby, the sound wave output device 3 can cause the sound wave output unit 33 to output a sound wave adjusted to be easy for the user to hear.

[0080] [Operation] Next, with reference to FIGS. 4 to 8, an operation example of the sound wave output system of the present embodiment will be described.

[0081] FIGS. 4 and 5 are sequence diagrams showing an operation example of the sound wave output system. FIG. 6 is a flowchart showing an operation example of the external device 2. FIGS. 7 and 8 are flowcharts showing an operation example of the sound wave output device 3.

[0082] First, with reference to FIG. 4, the operation in which the external device 2 performs the initial setting of the sound wave output device 3 will be described together with the operation of the sound wave output device 3.

[0083] The input / output unit 21 of the external device 2 receives an instruction to start the initial setting in response to the user's operation (step S201).

[0084] An instruction to start the initial setting is input from the input / output unit 21 to the transmission unit 22. The transmission unit 22 transmits an output request notification for requesting the output of a reference sound for initial setting, which includes frequency information indicating the initial frequency and volume information indicating the initial volume, among sound waves having different characteristics to be evaluated, to the sound wave output device 3 (step S202).

[0085] The receiving unit 34 of the sound wave output device 3 receives an output request notification for initial setting, which includes frequency information and volume information. The receiving unit 34 outputs the frequency information and the volume information to the control unit 32 in an associated manner.

[0086] The control unit 32 receives the frequency information and the volume information from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting a sound wave having the frequency indicated by the input frequency information and the volume indicated by the input volume information to the left ear sound wave output unit 331 and the right ear sound wave output unit 332.

[0087] An acoustic signal for outputting a sound wave of a reference sound is input to the left ear sound wave output unit 331 from the control unit 32. Also, an acoustic signal for outputting a sound wave of a reference sound is input to the right ear sound wave output unit 332 from the control unit 32. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the user's right ear (step S203). Thereby, the user can hear the sound wave of the reference sound based on the output request notification.

[0088] The left ear sound wave input unit 351 inputs the sound wave reflected inside the left ear and outputs an input acoustic signal indicating the input sound wave to the generation unit 31. Also, the right ear sound wave input unit 352 inputs the sound wave reflected inside the right ear and outputs an input acoustic signal indicating the input sound wave to the generation unit 31 (step S204).

[0089] The generation unit 31 receives the input acoustic signal from the left ear sound wave input unit 351. Also, the generation unit 31 receives the input acoustic signal from the right ear sound wave input unit 352. The generation unit 31 analyzes the strength of the signal of each frequency component constituting the input acoustic signal (step S205). The generation unit 31 generates reflected sound wave characteristic information indicating the frequency characteristics according to the analysis result.

[0090] The input / output unit 21 of the external device 2 receives an operation input regarding an evaluation indicating whether it is easy to hear according to the operation of the user, and generates evaluation information corresponding to the operation input (step S206). The input / output unit 21 outputs the generated evaluation information to the transmission unit 22.

[0091] The transmission unit 22 transmits the evaluation information generated in step S206 to the sound wave output device 3 (step S207). The transmission unit 22 outputs, to the determination unit 23, by associating the frequency information and the volume information included in the output request notification transmitted immediately before the evaluation information is generated.

[0092] Note that the operations of step S206 and step S207 are performed in parallel with the operations of step S204 and step S205. The order in which the operations from step S204 to step S207 are performed is not limited to the order shown in FIG. 4.

[0093] The receiving unit 34 of the sound wave output device 3 receives the evaluation information. The receiving unit 34 outputs, to the generation unit 31, by associating the evaluation information with the frequency information and the volume information included in the output request notification received before receiving the evaluation information.

[0094] The frequency information, the volume information, and the evaluation information are input to the generation unit 31 from the receiving unit 34. The frequency information and the volume information input from the receiving unit 34 are output sound information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output.

[0095] The determination unit 23 of the external device 2 determines whether evaluation information has been generated by the input / output unit 21 for all the reference sounds (step S208). When evaluation information has been generated for all the reference sounds, the determination unit 23 outputs an end notification of the initial setting to the transmission unit 22. When the end notification of the initial setting is input, the transmission unit 22 transmits the end notification of the initial setting to the sound wave output device 3.

[0096] The receiving unit 34 of the sound wave output device 3 receives the end notification of the initial setting from the external device 2. When receiving the end notification of the initial setting, the receiving unit 34 outputs the end notification of the initial setting to the generation unit 31.

[0097] When an end notification of the initial setting is input, the generation unit 31 associates the evaluation information with the reference sound characteristic information for each of the reference sounds and stores them in the reference sound characteristic information storage unit 37. The reference sound characteristic information includes output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds.

[0098] In step S208, if there is a reference sound for which evaluation information is not generated among all the reference sounds, the determination unit 23 of the external device 2 associates the frequency information and volume information of the reference sound for which evaluation information is not generated and outputs them to the transmission unit 22. When the frequency information and volume information are input, the transmission unit 22 of the external device 2 performs the operation of step S202. That is, the transmission unit 22 transmits a reference sound output request notification for initial setting, including the input frequency information and volume information, to the sound wave output device 3. Then, the external device 2 performs the operations of step S206, step S207, and step S208. Also, the sound wave output device 3 performs the operations from step S203 to step S205 until it receives the end notification of the initial setting.

[0099] Next, with reference to FIG. 5, the operation of controlling the characteristics of the acoustic signal and inputting the sound wave output device 3 to the sound wave output unit 33 for the voice signal received from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) will be described.

[0100] The sound wave output device 3 of the present embodiment generates sound wave characteristic information of the sound wave reflected inside the ear among the previously output sound waves during an online meeting. Then, the sound wave output device 3 controls the characteristics of the acoustic signal to be output to the sound wave output unit 33 next.

[0101] In FIG. 5, an example will be described in the case where a notification requesting control of the characteristics of the acoustic signal is not input to the control unit 32 of the sound wave output device 3 before the sound wave output device 3 receives the voice signal transmitted from the sound wave input devices 4-2 to 4-n in step S302 described later.

[0102] Each of the sound wave input devices 4-2 to 4-n receives the input of the voice of other users (step S301). Each of the sound wave input devices 4-2 to 4-n that has received the voice input generates a voice signal, which is a signal corresponding to the voice. Each of the sound wave input devices 4-2 to 4-n that has generated the voice signal transmits the voice signal to a sound wave output device (in this example, the sound wave output device 3 is included) excluding the sound wave output device of the other user who input the voice to the sound wave input device (step S302).

[0103] The receiving unit 34 of the sound wave output device 3 receives the voice signal from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n). The receiving unit 34 outputs the voice signal to the control unit 32.

[0104] The voice signal is input to the control unit 32 from the receiving unit 34. The control unit 32 inputs, via the amplification means, an acoustic signal for outputting a sound wave based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332.

[0105] The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the right ear of the user (step S303).

[0106] The left ear sound wave input unit 351 inputs the sound wave reflected inside the left ear of the user and outputs an input acoustic signal indicating the input sound wave to the generation unit 31. The right ear sound wave input unit 352 inputs the sound wave reflected inside the right ear of the user and outputs an input acoustic signal indicating the input sound wave to the generation unit 31 (step S304).

[0107] The generation unit 31 receives an input acoustic signal from the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352. The generation unit 31 analyzes the strength of the signals of each frequency component that constitutes the input acoustic signal (step S305). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 outputs the sound wave characteristic information generated from the input acoustic signal input from the left-ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right-ear sound wave input unit 352 to the determination unit 36.

[0108] Based on the input sound wave characteristic information, the determination unit 36 determines whether a sound wave that is easy for the user to hear is output by each of the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 (step S306). Specifically, the determination unit 36 determines whether the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which the user is considered to be easy to hear, deviate from the frequency characteristics indicated by the sound wave characteristic information by a predetermined threshold or more. If the deviation is more than the predetermined threshold, the sound wave corresponding to the input sound wave characteristic information is a sound wave that is difficult for the user to hear, and if it is less than the predetermined threshold, it indicates that the sound wave is easy for the user to hear.

[0109] In step S306, when the determination unit 36 determines that a sound wave that is difficult for the user to hear is output by the sound wave output unit 33 based on the input sound wave characteristic information, the following operation is performed. The determination unit 36 identifies the output acoustic information associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 and associated with the evaluation information indicating easy-to-hear for the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear. Alternatively, the determination unit 36 may calculate the similarity between the frequency characteristics indicated by the input sound wave characteristic information and the frequency characteristics indicated by the reflected sound wave characteristic information associated with the evaluation information indicating easy-to-hear, and identify the reflected sound wave characteristic information with a similarity higher than the threshold. Further, the determination unit 36 may identify the output acoustic information associated with the identified reflected sound wave characteristic information and stored in the reference sound characteristic information storage unit 37.

[0110] The determination unit 36 outputs a notification requesting control of the characteristics of the acoustic signal, including the specified output acoustic information, to the control unit 32.

[0111] Also, in step S306, when the determination unit 36 determines that an easy-to-hear sound wave for the user is being output by the sound wave output unit 33 based on the input sound wave characteristic information, the determination unit 36 does not specify the output acoustic information. Further, the determination unit 36 does not perform the process of outputting a notification requesting control of the characteristics of the acoustic signal to the control unit 32.

[0112] A voice signal is input to the control unit 32 from the reception unit 34. When a notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 generates an acoustic signal for outputting a sound wave based on the voice signal. On the other hand, when a notification requesting control of the characteristics of the acoustic signal is input from the determination unit 36, the control unit 32 controls the characteristics of the acoustic signal to be output with respect to the voice signal from the reception unit 34. Specifically, the control unit 32 controls the characteristics of the acoustic signal input to the sound wave output unit 33 so that the deviation from the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal becomes small. Thereby, the control unit 32 generates an acoustic signal for causing the sound wave output unit 33 to output a sound wave based on the voice signal with the characteristics of the acoustic signal controlled (step S307). The control unit 32 inputs the acoustic signal generated in step S307 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplification means. In this way, the control unit 32 controls the characteristics of the acoustic signal so that the deviation from the characteristics of the sound wave reflected inside the user's ear, which is the reflected sound wave characteristic information that is the frequency characteristic indicated by the reference sound characteristic information and is easy for the user to hear, becomes small.

[0113] The output acoustic information included in the notification requesting control of the characteristics of the acoustic signal indicates the frequency characteristics of the acoustic signal when a reference sound that is easy for the user to hear is output. When controlling the characteristics of the acoustic signal so that the deviation from the frequency characteristics of the output acoustic information becomes small, the frequency characteristics of the sound wave characteristic information in the input acoustic signal of the acoustic signal whose characteristics are controlled are such that the deviation from the frequency characteristics of the reflected sound wave characteristic information is smaller than the deviation from the frequency characteristics of the sound wave characteristic information of the acoustic signal whose characteristics are not controlled.

[0114] The left ear sound wave output unit 331 and the right ear sound wave output unit 332 receive an acoustic signal from the control unit 32. The left ear sound wave output unit 331 and the right ear sound wave output unit 332 convert the acoustic signal into a sound wave and output it (step S308).

[0115] In addition, when the receiving unit 34 receives a voice signal after step S303, the acoustic signal generated in step S307 is an acoustic signal for outputting a sound wave based on the voice signal received after step S303.

[0116] Also, when a sound wave based on the voice signal transmitted in step S302 is being output at the start of step S307, the control unit 32 may perform the following operation in step S307. The control unit 32 may generate an acoustic signal for outputting a sound wave among the sound waves based on the voice signal transmitted in step S302 that will be output next.

[0117] Next, with reference to FIG. 6, an operation example of the external device 2 will be described. Also, the operation in FIG. 6 details the operations of steps S201, S202, S206, S207, and S208 in FIG. 4.

[0118] The input / output unit 21 receives an instruction to start initial settings in response to a user operation (step S401). The input / output unit 21 outputs an instruction to start initial settings to the transmission unit 22.

[0119] An instruction to start the initial setting is input from the input / output unit 21 to the transmission unit 22. When an instruction to start the initial setting is input, the transmission unit 22 performs the following operations. The transmission unit 22 transmits an output request notification for requesting the output of a reference sound for the initial setting, which includes frequency information and volume information, to the sound wave output device 3 (step S402). For example, the output request notification output first at the time of initial setting includes frequency information indicating the initial value frequency and volume information indicating the initial value volume among the reference sounds.

[0120] The input / output unit 21 receives an operation input regarding an evaluation indicating whether it is easy to hear according to the operation of the user. When an operation input regarding an evaluation indicating that it is easy to hear is received (step S403, YES), the input / output unit 21 generates evaluation information indicating that it is easy to hear and outputs the generated evaluation information to the transmission unit 22. Also, when an operation input regarding an evaluation indicating that it is difficult to hear is received (step S403, NO), the input / output unit 21 generates evaluation information indicating that it is difficult to hear and outputs the generated evaluation information to the transmission unit 22.

[0121] When evaluation information indicating that it is easy to hear is input, the transmission unit 22 transmits the evaluation information indicating that it is easy to hear to the sound wave output device 3 (step S404). When evaluation information indicating that it is difficult to hear is input, the transmission unit 22 does not perform the operation of step S404. The transmission unit 22 transmits the evaluation information indicating that it is difficult to hear to the sound wave output device 3 (step S405).

[0122] The transmission unit 22 that has performed the operation of step S404 or step S405 outputs the frequency information and the volume information included in the output request notification transmitted in step S402 to the determination unit 23 in association with each other.

[0123] Frequency information and volume information are input from the transmission unit 22 to the determination unit 23. The determination unit 23 determines whether evaluation information has been generated for all reference tones of the volume to be evaluated at the reference tone of the frequency indicated by the input frequency information (step S406). Here, the volume to be evaluated is a volume that is preset as the volume to be evaluated at the reference tone of a predetermined frequency. For example, the external device 2 causes the sound wave output device 3 to output a reference tone of a predetermined frequency at volumes with different predetermined intervals from the minimum volume for evaluation to the maximum volume for evaluation. Further, the external device may set the volume of the reference tone to be evaluated for each frequency based on the A characteristic, which is the frequency weighting characteristic.

[0124] If there is a reference tone of a volume for which evaluation information has not been generated at the reference tone of the frequency indicated by the input frequency information (step S406, NO), the determination unit 23 performs the following operation. The determination unit 23 then associates the frequency information and volume information of the reference tone to be evaluated next and outputs them to the transmission unit 22 (step S407). Specifically, in step S407, the determination unit 23 associates the frequency information input from the transmission unit 22 with the volume information indicating the volume for which evaluation information has not been generated and outputs them to the transmission unit 22.

[0125] Frequency information and volume information are input from the determination unit 23 to the transmission unit 22. In step S402, the transmission unit 22 transmits an output request notification including the frequency information and volume information output in step S407 to the sound wave output device 3. The external device 2 performs the operations from step S402 to step S407 until evaluation information is generated for all reference tones of the volume to be evaluated at the reference tone of the frequency indicated by the predetermined frequency information.

[0126] When evaluation information is generated for the reference tones of all volumes to be evaluated at the reference tone of the frequency indicated by the input frequency information (step S406, YES), the determination unit 23 performs the following operations. The determination unit 23 checks whether evaluation information has been generated for the reference tones of all frequencies to be evaluated (step S408). The frequency to be evaluated here is a frequency preset as the frequency to be evaluated. For example, the external device 2 causes the sound wave output device 3 to output reference tones of frequencies at predetermined intervals from the minimum frequency to be evaluated to the maximum frequency to be evaluated.

[0127] If there is a reference tone of a frequency for which evaluation information has not been generated (step S408, NO), the determination unit 23 performs the following operations. The determination unit 23 outputs the frequency information and volume information of the sound wave to be evaluated next in association with each other to the transmission unit 22 (step S409). Specifically, in step S409, the determination unit 23 outputs to the transmission unit 22 the frequency information indicating the frequency to be evaluated next for which evaluation information has not been generated and the volume information indicating the volume to be evaluated, in association with each other (step S409). The transmission unit 22 transmits an output request notification including the frequency information and volume information output in step S409 to the sound wave output device 3 in step S402. The external device 2 performs the operations from step S402 to step S409 until evaluation information is generated for the reference tones of all frequencies to be evaluated.

[0128] When evaluation information is generated for the reference tones of all frequencies to be evaluated (step S408, YES), the determination unit 23 inputs an end notification of the initial setting to the transmission unit 22.

[0129] An end notification of the initial setting is input to the transmission unit 22 from the determination unit 23. The transmission unit 22 transmits the end notification of the initial setting to the sound wave output device 3 (step S410). Also, since evaluation information has been generated for all reference tones, the external device 2 ends the operation in FIG. 6.

[0130] Next, with reference to FIG. 7, an operation example in which the sound wave output device 3 performs initial settings under the control of the external device 2 will be described. The operation of FIG. 7 details the operations from step S203 to step S205 of FIG. 4.

[0131] The receiving unit 34 receives an output request notification for initial settings, which includes frequency information and volume information (step S501). The receiving unit 34 outputs the frequency information and the volume information in association with each other to the control unit 32.

[0132] The control unit 32 receives the frequency information and the volume information from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting a sound wave having the frequency indicated by the input frequency information and the volume indicated by the input volume information to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplification means.

[0133] The left ear sound wave output unit 331 receives the acoustic signal from the control unit 32. Also, the right ear sound wave output unit 332 receives the acoustic signal from the control unit 32. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave inside the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave inside the user's right ear (step S502).

[0134] In step S502, the left ear sound wave output unit 331 may output the sound wave at a timing different from that of the right ear sound wave output unit 332, or may output the sound wave at the same timing as the right ear sound wave output unit 332. When the left ear sound wave output unit 331 outputs the sound wave at a timing different from that of the right ear sound wave output unit 332, the sound wave output device 3 may receive evaluation information for each of the left and right ears and store the reference sound characteristic information for each of the left and right ears in the reference sound characteristic information storage unit 37.

[0135] The left-ear sound wave input unit 351 inputs sound waves reflected inside the left ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. The right-ear sound wave input unit 352 inputs sound waves reflected inside the right ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31 (step S503). The right-ear sound wave input unit 352 and the left-ear sound wave input unit 351 output the input acoustic signal to the generation unit 31 at the same timing. Note that the right-ear sound wave input unit 352 may output the input acoustic signal to the generation unit 31 at a timing different from that of the left-ear sound wave input unit 351.

[0136] An input acoustic signal is input to the generation unit 31 from the left-ear sound wave input unit 351. Also, an input acoustic signal is input to the generation unit 31 from the right-ear sound wave input unit 352. The generation unit 31 analyzes the strength of the signals of each frequency component that constitutes the frequency of the input acoustic signal from the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352. The generation unit 31 generates reflected sound wave characteristic information according to the analysis result (step S504).

[0137] Also, the receiving unit 34 receives evaluation information (step S505). The receiving unit 34 associates the evaluation information with the frequency information and volume information included in the output request notification received before receiving the evaluation information and outputs the result to the generation unit 31. Note that the receiving unit 34 receives the evaluation information at an arbitrary timing. The frequency information and volume information included in the output request notification output to the generation unit 31 are output acoustic information.

[0138] The evaluation information and the output acoustic information are input to the generation unit 31 from the receiving unit 34. The generation unit 31 does not perform the operations after step S507 until an end notification of the initial setting is input (step S506, NO).

[0139] Note that each time the receiving unit 34 receives an output request notification (step S501), the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 perform the operation of step S502. Also, the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352 perform the operation of step S503. The generation unit 31 performs the operation of step S504. The receiving unit 34 performs the operation of step S505.

[0140] The receiving unit 34 receives an end notification of the initial setting from the external device 2. When receiving the end notification of the initial setting, the receiving unit 34 outputs the end notification of the initial setting to the generation unit 31.

[0141] An end notification of the initial setting is input to the generation unit 31 from the receiving unit 34 (step S506, YES). When the end notification of the initial setting is input from the receiving unit 34, the following operation is performed. The generation unit 31 associates the evaluation information with the reference sound characteristic information for each of the reference sounds and stores them in the reference sound characteristic information storage unit 37 (step S507). The reference sound characteristic information includes output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds.

[0142] Note that each time the output acoustic information and the evaluation information are input, the generation unit 31 may associate the evaluation information with the reference sound characteristic information and store them in the reference sound characteristic information storage unit 37.

[0143] Next, with reference to FIG. 8, an operation example in which the sound wave output device 3 inputs an acoustic signal to the sound wave output unit 33 for the voice signal received from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) will be described. The operation of FIG. 8 details the operations from step S303 to step S308 of FIG. 5.

[0144] Note that in FIG. 8, an example will be described in the case where a notification requesting control of the characteristics of the acoustic signal is not input to the control unit 32 of the sound wave output device 3 before the sound wave output device 3 receives the voice signal in step S601 to be described later.

[0145] The receiving unit 34 receives a voice signal from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) (step S601). The receiving unit 34 outputs the voice signal to the control unit 32.

[0146] The control unit 32 receives an audio signal from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means (step S602).

[0147] The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave inside the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave inside the user's right ear (step S603).

[0148] The left ear sound wave input unit 351 inputs the sound wave reflected inside the left ear and outputs an input acoustic signal indicating the input sound wave to the generation unit 31. The right ear sound wave input unit 352 inputs the sound wave reflected inside the right ear and outputs an input acoustic signal indicating the input sound wave to the generation unit 31 (step S604).

[0149] The generation unit 31 receives input acoustic signals from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 31 analyzes the strength of the signals of each frequency component constituting the input input acoustic signal (step S605). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 outputs the sound wave characteristic information generated from the input acoustic signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right ear sound wave input unit 352 to the determination unit 36.

[0150] Based on the input sound wave characteristic information, the determination unit 36 determines whether a sound wave that is easy for the user to hear is output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S606).

[0151] In step S606, the determination unit 36 makes the following determination. The determination unit 36 determines whether the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which the user finds easy to hear, and the frequency characteristics indicated by the sound wave characteristic information deviate from each other by a predetermined threshold or more. If the deviation is greater than or equal to the predetermined threshold, the determination unit 36 determines that a sound wave that is difficult for the user to hear is being output by the sound wave output unit 33. If the deviation is less than the predetermined threshold, the determination unit 36 determines that a sound wave that is easy for the user to hear is being output by the sound wave output unit 33.

[0152] When the determination unit 36 determines that a sound wave that is difficult for the user to hear is being output by the sound wave output unit 33 (step S606, NO), it performs the following operation. The determination unit 36 identifies the output acoustic information associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 and associated with the evaluation information indicating ease of hearing for the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear. For example, the determination unit 36 calculates the similarity between the frequency characteristics indicated by the reflected sound wave characteristic information and the frequency characteristics indicated by the input sound wave characteristic information among the reflected sound wave characteristic information associated with the evaluation information indicating ease of hearing, and identifies the reflected sound wave characteristic information with a similarity higher than the threshold.

[0153] The determination unit 36 outputs a notification requesting control of the characteristics of the acoustic signal, including the identified output acoustic information, to the control unit 32 (step S607).

[0154] When the determination unit 36 determines that a sound wave that is easy for the user to hear is being output by the sound wave output unit 33 based on the input sound wave characteristic information (step S606, YES), it does not perform the operation of step S607.

[0155] When a notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 generates an acoustic signal for outputting a sound wave based on the voice signal in step S602. When a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 performs the following operation in step S602. The control unit 32 controls the characteristics of the acoustic signal to be output with respect to the voice signal from the reception unit 34. Specifically, the control unit 32 controls the characteristics of the acoustic signal so that the deviation from the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal becomes small. Thereby, the control unit 32 generates an acoustic signal for causing the sound wave output unit 33 to output a sound wave based on the voice signal with the characteristics of the acoustic signal controlled.

[0156] Note that the determination unit 36 may make a determination as follows in step S606. The determination unit 36 may determine whether the frequency characteristics indicated by the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 associated with the evaluation information indicating that it is difficult to hear and the frequency characteristics indicated by the sound wave characteristic information deviate by a predetermined threshold or more. When the deviation is equal to or greater than a predetermined threshold, the determination unit 36 determines that a sound wave that is easy for the user to hear is being output by the sound wave output unit 33. When the deviation is less than the predetermined threshold, the determination unit 36 determines that a sound wave that is difficult for the user to hear is being output by the sound wave output unit 33. In addition to these methods, the determination unit 36 can use any method for determining whether a sound wave that is easy for the user to hear is being output by the sound wave output unit 33.

[0157] As described above, the sound wave output device 3 of the present embodiment generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves that the user can hear, by the sound wave output unit 33 that converts and outputs an acoustic signal into sound waves. The sound wave output device 3 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear, becomes small. Further, the sound wave output device 3 inputs the acoustic signal with controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output, to the user, sound waves by the acoustic signal whose characteristics are adjusted according to the reference sound characteristic information. Thereby, the sound wave output device 3 can cause the sound wave output unit 33 to output a sound wave adjusted to be easy for the user to hear.

[0158] Further, the sound wave output device 3 of the present embodiment generates an acoustic signal for outputting sound waves based on a voice signal, in which the characteristics of the acoustic signal are controlled so that the deviation from the output acoustic information included in a notification requesting control of the characteristics of the acoustic signal becomes small. The output acoustic information included in the notification requesting control of the characteristics of the acoustic signal indicates the frequency characteristics of the acoustic signal when each of the reference sounds that are easy for the user to hear is output. The sound wave characteristic information of the input acoustic signal of the acoustic signal with controlled characteristics has a smaller deviation from the reflected sound wave characteristic information of the reference sound characteristic information than the sound wave characteristic information of the input acoustic signal of the acoustic signal when the characteristics are not controlled. By operating in this way, the control unit 32 can cause the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves that are easy for the user to hear.

[0159] Further, since the sound wave output device 3 of the present embodiment continuously controls the characteristics of the acoustic signal input to the sound wave output unit 33, the sound wave output device 3 can automatically output sound waves that are easy for the user to hear during a call or an online meeting.

[0160] In addition, for the left-ear sound wave output unit 331 of the sound wave output device 3 of the present embodiment, the characteristics of the acoustic signal are controlled based on the reference sound characteristics information for the left ear, and for the right-ear sound wave output unit 332, the characteristics of the acoustic signal are controlled based on the reference sound characteristics information for the right ear. Thereby, even when the frequency characteristics of the sound waves that are easy to hear differ for each of the user's left and right ears, the sound wave output device 3 can output sound waves that are adjusted to be easy to hear according to the audibility of each of the left and right ears from the sound wave output unit 33.

[0161] [Third Embodiment] Next, the sound wave output device 5 in the third embodiment of the present invention will be specifically described.

[0162] In the third embodiment, the generation unit 51 corresponds to, for example, the generation unit 11 of the first embodiment. In the third embodiment, the control unit 52, the determination unit 53, the reference sound characteristics information storage unit 37, and the speaker information storage unit 54 correspond to, for example, the control unit 12 of the first embodiment. The third embodiment is different from the sound wave output device 1 of the first embodiment in that the sound wave output device 3 further includes a sound wave output unit 33, a reception unit 34, and a sound wave input unit 35.

[0163] Further, the third embodiment is different from the sound wave output device 3 of the second embodiment in that the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 of the sound wave output device 5 output sound waves based on the voice signal at different timings for each speaker. In the present embodiment, the sound wave output device 5 controls, for each speaker, the timing at which sound waves based on the voice signal are output from the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 as the characteristics of the acoustic signal. In the present embodiment, the description will focus on the differences from the second embodiment, but various examples described in the second embodiment can be applied.

[0164] With reference to FIG. 9, a configuration example of the sound wave output system of the present embodiment will be described. FIG. 9 is a block diagram showing a configuration example of the sound wave output system of the present embodiment.

[0165] The sound wave output system of the present embodiment includes a sound wave output device 5.

[0166] Since each configuration of the external device 2 in this embodiment is the same as each configuration of the external device 2 in the second embodiment shown in FIG. 3, the corresponding elements are denoted by the same reference numerals as in FIG. 3 and the description thereof is omitted.

[0167] Since each configuration of the sound wave input devices 4-1 to 4-n in this embodiment is the same as each configuration of the sound wave input devices 4-1 to 4-n in the second embodiment shown in FIG. 3, the corresponding elements are denoted by the same reference numerals as in FIG. 3 and the description thereof is omitted.

[0168] Next, with reference to FIG. 9, the configuration of the sound wave output device 5 of this embodiment will be described in detail.

[0169] The sound wave output device 5 includes a generation unit 51 and a control unit 52.

[0170] The sound wave output unit 33 is connected to the control unit 52. The reception unit 34 is connected to the generation unit 51 and the control unit 52. The sound wave input unit 35 is connected to the generation unit 51. The determination unit 53 is connected to the generation unit 51, the control unit 52, the reference sound characteristic information storage unit 37, and the speaker information storage unit 54. The reference sound characteristic information storage unit 37 is connected to the generation unit 51 and the determination unit 53. The speaker information storage unit 54 is connected to the determination unit 53.

[0171] Since the sound wave output unit 33, the reception unit 34, the sound wave input unit 35, and the reference sound characteristic information storage unit 37 in this embodiment are the same as each configuration of the sound wave output device 3 in the second embodiment shown in FIG. 3, the corresponding elements are denoted by the same reference numerals as in FIG. 3 and the common description thereof is omitted.

[0172] Also, the process in which each configuration of the sound wave output device 5 performs initial setting under the control of the external device 2 is the same as the process of each configuration of the sound wave output device 3 in the second embodiment. Therefore, the description of the process in which the sound wave output device 5 performs initial setting under the control of the external device 2 is omitted.

[0173] Hereinafter, each configuration of the sound wave output device 5 when used in an online meeting will be described in detail.

[0174] The receiving unit 34 receives an audio signal, which is a signal corresponding to voice from a sound wave input device (in this example, sound wave input devices 4-2 to 4-n). The receiving unit 34 outputs the audio signal to the control unit 52.

[0175] The sound wave output unit 33 includes a left ear sound wave output unit 331 and a right ear sound wave output unit 332.

[0176] An acoustic signal is input to the left ear sound wave output unit 331 from the control unit 52, which will be described later. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the user's left ear.

[0177] An acoustic signal is input to the right ear sound wave output unit 332 from the control unit 52. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the user's left ear. Each of the right ear sound wave output unit 332 and the left ear sound wave output unit 331 outputs a sound wave at a predetermined timing based on the acoustic signal. Also, the predetermined timing is the timing indicated by time difference information indicating the timing at which voice is output from the right ear sound wave output unit 332 and the timing at which voice is output from the left ear sound wave output unit 331, which are set for each speaker using the sound wave output system. Note that, similar to the timing, the sound wave output device 5 may output sound waves at different volumes to the right ear sound wave output unit 332 and the left ear sound wave output unit 331, respectively, so that voice can be heard from a specific direction for each speaker.

[0178] The generation unit 51 in the present embodiment is the same as the generation unit 31 of the sound wave output device 3 in the second embodiment shown in FIG. 3, and thus the description thereof is omitted.

[0179] The determination unit 53 receives the sound wave characteristic information generated from the input acoustic signal of the sound wave input unit 35 from the generation unit 51. Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. The process of determining whether sound waves that are easy for the user to hear are output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 is the same as that in the second embodiment, so the description is omitted.

[0180] When the determination unit 53 determines, based on the input sound wave characteristic information, that sound waves that are difficult for the user to hear are output by the sound wave output unit 33, it specifies the output acoustic information. The operation of specifying the output acoustic information is the same as that in the second embodiment, so the description is omitted.

[0181] Also, when the determination unit 53 determines, based on the input sound wave characteristic information, that sound waves that are easy for the user to hear are output by the sound wave output unit 33, it does not perform the process of specifying the output acoustic information.

[0182] The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. The speaker information indicates the frequency characteristics of each speaker's voice. In the speaker information storage unit 54, the speaker information is associated and stored with time difference information indicating the timing at which the voice is output from the right ear sound wave output unit 332 and the timing at which the voice is output from the left ear sound wave output unit 331, which are set for each speaker. The speaker information stored in the speaker information storage unit 54 is the sound wave characteristic information generated by the generation unit 51. The determination unit 53 may calculate the similarity between the information indicating the frequency distribution included in the sound wave characteristic information and the information indicating the frequency distribution included in the speaker information. The determination unit 53 identifies the speaker information with a similarity higher than the threshold value, and specifies the time difference information associated with the identified speaker information and stored in the speaker information storage unit 54.

[0183] Further, the determination unit 53 may utilize machine learning for identifying the speaker. For example, the determination unit 53 uses learning data that includes input acoustic signals of voices of multiple persons and information indicating which person's voice each of the input acoustic signals of the voices of multiple persons is. The determination unit 53 may generate an extraction model for extracting unique feature amounts of each speaker from the input acoustic signal based on the learning data, and extract information indicating the feature amounts from the input acoustic signal using the extraction model. The determination unit 53 may calculate the similarity between the feature amounts extracted from the input acoustic signal and the feature amounts stored in the speaker information storage unit 54 as speaker information. The determination unit 53 identifies the speaker information with a similarity higher than the threshold value, and identifies the time difference information stored in the speaker information storage unit 54 associated with the identified speaker information. Also, when the time difference information cannot be identified, the information indicating the feature amounts extracted from the input acoustic signal is stored in the speaker information storage unit 54 as the speaker information.

[0184] When the time difference information cannot be identified, the determination unit 53 performs the following operation. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54 to the speaker for whom the time difference information cannot be identified. For example, the determination unit 53 assigns the time difference information not yet assigned from among a plurality of preset time difference information in the order of the speakers' utterances. In this way, the determination unit 53 identifies the timing at which the sound is output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 for each speaker who made a statement in the online meeting so as not to overlap with other speakers. Thereby, for the voice of a user who newly participates in the online meeting, the control unit 52 can output the sound wave so that the voice of the user who newly participates in the online meeting can be heard from a direction different from that of other users. The determination unit 53 stores the assigned time difference information and the sound wave characteristic information as the speaker information in the speaker information storage unit 54.

[0185] When the output acoustic information is identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the characteristics of the acoustic signal including the identified output acoustic information and the time difference information. When the output acoustic information is not identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the output of the sound wave including the time difference information.

[0186] In the speaker information storage unit 54, speaker information and time difference information are stored in an associated manner. The speaker information stored in the speaker information storage unit 54 is sound wave characteristic information generated by the generation unit 51 that indicates the frequency characteristics of each speaker's voice. Also, the speaker information may be information indicating a feature amount extracted from the input acoustic signal. Further, in the speaker information storage unit 54, different time difference information for each speaker is stored in association with the speaker information. The speaker information and the time difference information may be stored in the speaker information storage unit 54 in advance by the user.

[0187] An audio signal is input to the control unit 52 from the reception unit 34. A notification requesting control of the characteristics of the acoustic signal is input to the control unit 52 from the determination unit 53. Or, a notification requesting control of the output of the sound wave is input to the control unit 52 from the determination unit 53. When a notification requesting control of the output of the sound wave is input from the determination unit 53, the control unit 52 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the reception unit 34 as follows. The control unit 52 generates an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information included in the notification requesting control of the output of the sound wave. The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via amplification means.

[0188] When a notification requesting control of the characteristics of an acoustic signal is input, the control unit 52 controls the characteristics of the acoustic signal to be output with respect to the voice signal from the reception unit 34 as follows. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the reference sound characteristics information and the characteristics of the sound wave reflected inside the user's ear, which the user can easily hear, is reduced. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for outputting a sound wave based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. The control unit 52 inputs the acoustic signal for outputting the sound wave based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means. In this way, the control unit 52 controls the acoustic signal so that sound waves are output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the specified time difference information.

[0189] Note that when neither a notification requesting control of the characteristics of the acoustic signal nor a notification requesting control of the output of the sound wave is input, the control unit 52 may perform the following operation. The control unit 52 may input an acoustic signal for simultaneously outputting a sound wave based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332. When neither a notification requesting control of the characteristics of the acoustic signal nor a notification requesting control of the output of the sound wave is input, for example, the start time of an online meeting is assumed.

[0190] As described above, the sound wave output device 5 generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves that the user can hear, by the sound wave output unit 33 that converts and outputs an acoustic signal into sound waves. The sound wave output device 5 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, where the sound waves of the reference sound are reflected inside the user's ear and are set to be easy for the user to hear, becomes small. The sound wave output device 5 inputs the acoustic signal with controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output, to the user, sound waves generated by the acoustic signal whose characteristics are adjusted according to the reference sound characteristic information. Thereby, the sound wave output device 5 can cause the sound wave output unit 33 to output sound waves that are adjusted to be easy for the user to hear.

[0191] Next, with reference to FIGS. 10 to 11, an operation example of the sound wave output system of the present embodiment will be described.

[0192] FIG. 10 is a sequence diagram showing an operation example of the sound wave output system. FIG. 11 is a flowchart showing an operation example of the sound wave output device 5.

[0193] First, with reference to FIG. 10, an operation in which the sound wave output device 5 controls the characteristics of the acoustic signal and inputs it to the sound wave output unit 33 for the voice signal received from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) will be described.

[0194] The sound wave output device 5 of the present embodiment generates sound wave characteristic information of the sound waves reflected inside the ear among the previously output sound waves during an online meeting. Then, the sound wave output device 5 controls at least the timings at which the left ear sound wave output unit 331 and the right ear sound wave output unit 332 output, as the characteristics of the acoustic signal of the sound waves to be output by the sound wave output unit 33 next.

[0195] Note that in FIG. 10, a case where a notification requesting control of the output of sound waves is input to the control unit 52 of the sound wave output device 5 before step S703, which will be described later, will be described as an example.

[0196] Each of the sound wave input devices 4-2 to 4-n receives the input of the voices of other users (step S701). Each of the sound wave input devices 4-2 to 4-n that has received the input of the voice generates a voice signal, which is a signal corresponding to the voice. Each of the sound wave input devices 4-2 to 4-n that has generated the voice signal transmits the voice signal to a sound wave output device (in this example, the sound wave output device 5 is included) excluding the sound wave output device of the other user who input the voice to the sound wave input device (step S702).

[0197] The receiving unit 34 of the sound wave output device 5 receives the voice signal from the sound wave input device (in this example, the sound wave input devices 4-2 to 4-n). The receiving unit 34 outputs the voice signal to the control unit 52.

[0198] The voice signal is input to the control unit 52 from the receiving unit 34. Further, a notification requesting control of the output of the sound wave is input to the control unit 52. The control unit 52 generates an acoustic signal for outputting a sound wave based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information included in the notification requesting control of the output of the sound wave. The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

[0199] The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave inside the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave inside the right ear of the user (step S703). In step S703, the left ear sound wave output unit 331 and the right ear sound wave output unit 332 output the sound wave at a predetermined timing based on the acoustic signal. Note that the sound wave output in step S703 may be a sound wave whose characteristics are controlled by the sound wave output device 5.

[0200] The left-ear sound wave input unit 351 inputs sound waves reflected inside the left ear of the user, and outputs an input acoustic signal indicating the input sound waves to the generation unit 51. The right-ear sound wave input unit 352 inputs sound waves reflected inside the right ear of the user, and outputs an input acoustic signal indicating the input sound waves to the generation unit 51 (step S704).

[0201] Input acoustic signals are input to the generation unit 51 from the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352. The generation unit 51 analyzes the strength of the signals of each frequency component constituting the input acoustic signal (step S705). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 outputs the sound wave characteristic information generated from the input acoustic signal input from the left-ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right-ear sound wave input unit 352 to the determination unit 53.

[0202] Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are output by each of the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 (step S706). Specifically, the determination unit 53 determines whether the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which is considered easy for the user to hear, deviate from the frequency characteristics indicated by the sound wave characteristic information by a predetermined threshold or more. If the deviation is more than the predetermined threshold, the sound wave corresponding to the input sound wave characteristic information is a sound wave that is difficult for the user to hear, and if it is less than the predetermined threshold, it indicates that it is a sound wave that is easy for the user to hear.

[0203] In step S706, when the determination unit 53 determines that a sound wave that is difficult for the user to hear is output by the sound wave output unit 33 based on the input sound wave characteristic information, the determination unit 53 performs the following operation. The determination unit 53 identifies output acoustic information associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 and associated with evaluation information indicating easy-to-hear for the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear.

[0204] When, in step S706, the determination unit 53 determines that an easy-to-hear sound wave for the user is being output by the sound wave output unit 33 based on the input sound wave characteristic information, the determination unit 53 does not specify the output acoustic information.

[0205] The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. The determination unit 53 identifies the time difference information stored in the speaker information storage unit 54 in association with the identified speaker information.

[0206] If the time difference information cannot be identified, the determination unit 53 performs the following operations. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54. The determination unit 53 stores the time difference information and the frequency information included in the sound wave characteristic information as the speaker information in the speaker information storage unit 54. In this way, the determination unit 53 identifies or assigns the time difference information (step S707).

[0207] When the output acoustic information is identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the characteristics of the acoustic signal including the identified output acoustic information and the time difference information. When the output acoustic information is not specified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the output of the sound wave including the time difference information.

[0208] The control unit 52 receives an audio signal from the receiving unit 34. The control unit 52 receives a notification requesting control of the characteristics of the acoustic signal from the determination unit 53. Or, the control unit 52 receives a notification requesting control of the output of the sound wave from the determination unit 53. When a notification requesting control of the output of the sound wave is input from the determination unit 53, the control unit 52 performs the following operations on the audio signal from the receiving unit 34 with respect to the acoustic signal to be output. The control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information included in the notification requesting control of the output of the sound wave. The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means. Also, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 performs the following operations on the audio signal from the receiving unit 34 with respect to the acoustic signal to be output. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the reference sound characteristics information, which is the sound wave that the user can easily hear, and the characteristics of the sound wave reflected inside the user's ear is reduced. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information. Thus, the control unit 52 generates an acoustic signal for outputting a sound wave based on the audio signal (step S708). The control unit 52 inputs the acoustic signal generated in step S708 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means.

[0209] The left ear sound wave output unit 331 and the right ear sound wave output unit 332 receive an acoustic signal from the control unit 52. The right ear sound wave output unit 332 and the left ear sound wave output unit 331 convert the acoustic signal into a sound wave and output the sound wave inside the user's ear (step S709).

[0210] Note that the determination unit 53 may perform the operations of step S706 and step S707 in parallel. Or, the determination unit 53 may perform the operation of step S706 after performing the operation of step S707.

[0211] Next, with reference to FIG. 11, an operation example in which the sound wave output device 5 inputs an acoustic signal to the sound wave output unit 33 for a voice signal received from a sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) will be described. The operation in FIG. 11 details the operations from step S703 to step S709 in FIG. 10. In FIG. 11, a case where a notification requesting control of the output of sound waves is input to the control unit 52 of the sound wave output device 5 before step S802 described later will be described as an example.

[0212] The receiving unit 34 receives a voice signal from a sound wave input device (in this example, the sound wave input devices 4-2 to 4-n) (step S801). The receiving unit 34 outputs the voice signal to the control unit 52.

[0213] The voice signal is input to the control unit 52 from the receiving unit 34. Also, a notification requesting control of the output of sound waves is input to the control unit 52. The control unit 52 inputs an acoustic signal for outputting sound waves based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at a predetermined timing via an amplification means (step S802). The predetermined timing is the timing indicated by the time difference information.

[0214] In step S802, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 performs the following operations on the acoustic signal to be output with respect to the voice signal from the receiving unit 34. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the calling party so that the deviation between the frequency characteristics indicated by the reference sound characteristics information, which is the sound wave that the user can easily hear, and the characteristics of the sound wave reflected inside the user's ear is small. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for outputting sound waves based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. The control unit 52 inputs an acoustic signal for outputting sound waves based on the voice signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means.

[0215] The left-ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave inside the user's left ear. The right-ear sound wave output unit 332 converts an acoustic signal into a sound wave and outputs the sound wave inside the user's right ear (step S803).

[0216] The left-ear sound wave input unit 351 inputs the sound wave reflected inside the left ear, and outputs an input acoustic signal indicating the input sound wave to the generation unit 51. The right-ear sound wave input unit 352 inputs the sound wave reflected inside the right ear, and outputs an input acoustic signal indicating the input sound wave to the generation unit 51 (step S804).

[0217] Input acoustic signals are input to the generation unit 51 from the left-ear sound wave input unit 351 and the right-ear sound wave input unit 352. The generation unit 51 analyzes the strength of the signals of each frequency component constituting the input acoustic signal (step S805). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 outputs the sound wave characteristic information generated from the input acoustic signal input from the left-ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right-ear sound wave input unit 352 to the determination unit 53.

[0218] Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are output by each of the left-ear sound wave output unit 331 and the right-ear sound wave output unit 332 (step S806). Since the specific processing in step S806 is the same as the processing in step S706 in FIG. 10, the description is omitted.

[0219] When the determination unit 53 determines based on the input sound wave characteristic information that sound waves that are difficult for the user to hear are output by the sound wave output unit 33 (step S806, NO), the following operation is performed. The determination unit 53 specifies output acoustic information associated with reflection sound wave characteristic information stored in the reference sound characteristic information storage unit 37 and associated with evaluation information indicating easy-to-hear for the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear (step S807).

[0220] When the determination unit 53 determines that an easy-to-hear sound wave for the user is being output by the sound wave output unit 33 based on the input sound wave characteristic information (step S806, YES), the operation in step S807 is not performed.

[0221] The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. When the speaker can be identified (step S808, YES), the determination unit 53 performs the following operation. The determination unit 53 identifies the time difference information associated with the speaker information of the identified speaker and stored in the speaker information storage unit 54 (step S809).

[0222] When the speaker cannot be identified (step S808, NO), the determination unit 53 does not perform the operation in step S809. For a speaker for whom the speaker cannot be identified and thus the time difference information cannot be identified, the determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54. The determination unit 53 stores the assigned time difference information and the sound wave characteristic information as the speaker information in the speaker information storage unit 54 (step S810).

[0223] The determination unit 53 that has performed the operation in step S809 or step S810 performs the following operation. When the output acoustic information is identified, that is, the determination unit 53 that has performed the operation in step S807 outputs a notification requesting control of the characteristics of the acoustic signal to the control unit 52. The notification requesting control of the characteristics of the acoustic signal includes the identified output acoustic information and the time difference information. When the output acoustic information is not identified, that is, the determination unit 53 that does not perform the operation in step S807 performs the following operation. The determination unit 53 outputs a notification requesting control of the output of the sound wave including the time difference information to the control unit 52. In this way, the determination unit 53 outputs a notification requesting control of the characteristics of the acoustic signal or a notification requesting control of the output of the sound wave to the control unit 52 (step S811).

[0224] Note that the determination unit 53 may perform the operations from step S806 to step 807 and the operations from step S808 to step S810 in parallel. Alternatively, the determination unit 53 may perform the operations from step S806 to step 807 after performing the operations from step S808 to step S810.

[0225] As described above, the sound wave output device 5 of the present embodiment generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves that the user can hear, by the sound wave output unit 33 that converts and outputs an acoustic signal into a sound wave. The sound wave output device 5 controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, where the sound wave of the reference sound is reflected inside the user's ear and is set to be easy for the user to hear, becomes small. The sound wave output device 5 inputs the acoustic signal with controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output, to the user, a sound wave by the acoustic signal whose characteristics are adjusted according to the reference sound characteristic information. Thereby, the sound wave output device 5 can cause the sound wave output unit 33 to output a sound wave that is adjusted to be easy for the user to hear.

[0226] Also, the sound wave output device 5 of the present embodiment outputs sound waves to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. Specifically, the sound wave output device 5 outputs sound waves to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at different timings for each speaker. Thereby, since the sound waves are output so that the voice can be heard from a specific direction for each speaker who is speaking, it becomes easier for the user of the sound wave output device 5 to determine which of the speakers is speaking.

[0227] Also, since the sound wave output device 5 of the present embodiment continuously controls the characteristics of the acoustic signal output from the sound wave output unit 33, the sound wave output device 5 can automatically output, during a call or an online meeting, a sound wave that is easy for the user to hear.

[0228] [Modification of the Third Embodiment] In addition to controlling the characteristics of the acoustic signal in the third embodiment, the control unit of the sound wave output device according to the modification of the third embodiment controls each of the left ear sound wave output unit and the right ear sound wave output unit so that the volume of the sound wave corresponding to the consonant is output within a predetermined threshold value. Since the method of controlling the characteristics of the acoustic signal in the third embodiment is the same as that in the third embodiment, the description thereof is omitted. Further, in addition to controlling the characteristics of the acoustic signal in the second embodiment, the sound wave output device 3 in the second embodiment may control each of the left ear sound wave output unit and the right ear sound wave output unit so that the volume of the sound wave corresponding to the consonant is output within a predetermined threshold value.

[0229] The generation unit of this modification analyzes the strength of the signals of each frequency component constituting the input input acoustic signal. The generation unit outputs to the determination unit acoustic wave characteristic information further including volume information indicating the volume of the sound wave corresponding to the consonant according to the analysis result.

[0230] Acoustic wave characteristic information further including volume information of the sound wave corresponding to the consonant is input to the determination unit of this modification. The determination unit outputs a notification requesting control of the volume of the consonant to the control unit.

[0231] Note that the determination unit may determine whether or not the volume of the sound wave corresponding to the consonant is within a predetermined threshold value based on the volume information of the sound wave corresponding to the consonant. When it is within the predetermined threshold value, the determination unit may operate in the same manner as the sound wave output device 5 of the third embodiment. Further, when it is not within the predetermined threshold value, the determination unit may output a notification requesting control of the volume of the consonant to the control unit.

[0232] A notification requesting control of the volume of the consonant is input to the control unit of this modification. When a notification requesting control of the volume of the consonant is input, the control unit controls the characteristics of the acoustic signal to be output as follows for the voice signal from the receiving unit. The control unit generates an acoustic signal such that the volume of the sound wave corresponding to the consonant is within a predetermined threshold value. The control unit inputs an acoustic signal for outputting a sound wave based on the voice signal to the left ear sound wave output unit and the right ear sound wave output unit.

[0233] The sound wave output unit outputs sound waves based on an acoustic signal into the inside of the user's ear.

[0234] The control unit of the sound wave output device of this modified example generates an acoustic signal so that the volume of the sound wave corresponding to the consonant is within a predetermined threshold value. Thereby, the control unit of the sound wave output device of this modified example can cause the sound wave output unit to output a sound wave adjusted so that the volumes of the consonants are about the same. Since the control unit controls the characteristics of the acoustic signal input to the sound wave output unit so that the magnitude of the volume of the sound wave corresponding to the consonant disappears, the user can clearly hear the conversation.

[0235] [Hardware Configuration Example] A configuration example of hardware resources for realizing each of the sound wave output devices (1, 3, 5) in each of the above-described embodiments of the present invention using one information processing device (computer) will be described. Note that the sound wave output device may be realized using at least two information processing devices physically or functionally. Further, the sound wave output device may be realized as a dedicated device. Further, only some functions of the sound wave output device may be realized using an information processing device.

[0236] FIG. 12 is a diagram schematically showing a hardware configuration example of an information processing device capable of realizing the sound wave output device of each embodiment in the present invention. The information processing device 6 includes a communication interface 61, an input / output interface 62, an arithmetic device 63, a storage device 64, a non-volatile storage device 65, and a drive device 66.

[0237] For example, the generation unit 11 of the sound wave output device 1 in FIG. 1 can be realized by the communication interface 61 and the arithmetic device 63. Further, the control unit 12 of the sound wave output device 1 in FIG. 1 can be realized by the arithmetic device 63.

[0238] The communication interface 61 is a communication means for the acoustic wave output device of each embodiment to communicate with an external device by wire or / and wirelessly. When the acoustic wave output device is realized using at least two information processing devices, they may be connected so as to be able to communicate with each other via the communication interface 61.

[0239] The input / output interface 62 is a man-machine interface such as a keyboard which is an example of an input device and a display as an output device.

[0240] The arithmetic unit 63 is realized by an arithmetic processing device such as a general-purpose CPU (Central Processing Unit) or a microprocessor, or a plurality of electric circuits. The arithmetic unit 63 can, for example, read out various programs stored in the non-volatile storage device 65 to the storage device 64 and execute processing according to the read programs.

[0241] The storage device 64 is a memory device such as a RAM (Random Access Memory) that can be referred to by the arithmetic unit 63, and stores programs, various data, etc. The storage device 64 may be a volatile memory device.

[0242] The non-volatile storage device 65 is a non-volatile storage device such as a ROM (Read Only Memory), a flash memory, etc., and can store various programs and data, etc.

[0243] The drive device 66 is a device that processes, for example, reading and writing of data to a recording medium 67 described later.

[0244] The recording medium 67 is an arbitrary recording medium capable of recording data, such as an optical disk, a magneto-optical disk, a semiconductor flash memory, etc.

[0245] Each embodiment of the present invention may configure a sound wave output device by, for example, the information processing apparatus 6 illustrated in FIG. 12. And each embodiment of the present invention may be realized by supplying a program capable of realizing the functions described in each of the above embodiments to this sound wave output device.

[0246] In this case, by the arithmetic unit 63 executing the program supplied to the sound wave output device, it is possible to realize the embodiment. Also, instead of all of the sound wave output device, it is also possible to configure some of the functions by the information processing apparatus 6.

[0247] Furthermore, the above program may be recorded on the recording medium 67, and the sound wave output device may be configured such that the above program is appropriately stored in the nonvolatile storage device 65 at the shipping stage or the operation stage, etc. of the sound wave output device. Note that, in this case, as the method for supplying the above program, a method of installing it inside the sound wave output device using an appropriate jig may be adopted at the manufacturing stage before shipping or the operation stage, etc. Also, as the method for supplying the above program, a general procedure such as a method of downloading from the outside via a communication line such as the Internet may be adopted.

[0248] Note that each of the above-described embodiments is a preferred embodiment of the present invention, and various modifications can be made without departing from the gist of the present invention.

Description of Reference Numerals

[0249] 1, 3, 5 Sound wave output device 11, 31, 51 Generation unit 12, 32, 52 Control unit 33 Sound wave output unit 331 Left ear sound wave output unit 332 Right ear sound wave output unit 34 Reception unit 35 Sound wave input unit 351 Left ear sound wave input unit 352 Right ear sound wave input unit 36, 53 Judgment unit 37 Reference sound characteristic information storage unit 54 Speaker information storage unit 2 External device 21 Input / output unit 22 Transmission unit 23 Judgment unit 4-1 to 4-n Sound wave input device 41-1 to 41-n Sound wave input section 42-1 to 42-n Transmission section 6 Information processing device 61 Communication interface 62 Input / output interface 63 Arithmetic unit 64 Storage device 65 Non-volatile storage device 66 Drive device 67 Recording medium

Claims

1. Generating means for generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by a user by sound wave output means that converts an acoustic signal into sound waves and outputs them; The control means controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having frequency components that the user can hear is small, and inputs the signal to the sound wave output means via an amplification means, where the sound wave of the reference sound is reflected inside the user's ear and the characteristics of the sound wave are set to be easy to hear by the user; A sound wave output device comprising the above.

2. The sound wave output device further comprises sound wave input means for inputting the sound waves of the call partner output from the sound wave output means into the ear and reflected inside the ear, and outputting an input acoustic signal indicating the input sound waves; The reference sound characteristic information includes the result of the user's determination of whether the reference sound is easy to hear and frequency information indicating the frequency characteristics of the sound waves reflected inside the user's ear by the reference sound; The generating means generates the sound wave characteristic information according to the analysis result of the strength of the signals of each frequency component constituting the input acoustic signal The sound wave output device according to claim 1, characterized in that.

3. The control means controls the acoustic signal so that the deviation of the frequency characteristics of the sound wave characteristic information from the frequency characteristics of the reference sound characteristic information is small The sound wave output device according to claim 2, characterized in that.

4. The reference sound is a plurality of types of sound waves output by the sound wave output means having different frequencies and volumes from each other, The sound wave input means outputs the input acoustic signal for each of the reference sounds, The sound wave input means further comprises a receiving unit for receiving evaluation information indicating whether it is easy to hear for each of the reference sounds from an external device, The generating means stores, in the reference sound characteristic information storage means, as the reference sound characteristic information, output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds; When the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information, which the user has determined to be easy to hear and is stored in the reference sound characteristic information storage means, deviate from the frequency characteristics indicated by the sound wave characteristic information, the control means controls the characteristics of the acoustic signal so that the deviation from the frequency characteristics indicated by the output acoustic information associated with the reflected sound wave characteristic information becomes smaller. The sound wave output device according to claim 2 or claim 3, characterized by the above.

5. The generation means further generates the sound wave characteristic information including volume information indicating the volume of the sound wave corresponding to the consonant according to the analysis result of the strength of the signals of the respective frequency components constituting the input acoustic signal. The control means controls the characteristics of the acoustic signal so that the volume of the sound wave corresponding to the consonant is within a predetermined threshold value. The sound wave output device according to any one of claims 2 to 4, characterized by the above.

6. The sound wave output means includes a right ear sound wave output means for inputting sound to the right ear of the user and a left ear sound wave output means for inputting sound to the left ear of the user. Speaker information storage means for storing speaker information indicating the frequency characteristics of the voices of the respective speakers and time difference information indicating the timing at which the sound is output from the right ear sound wave output means and the timing at which the sound is output from the left ear sound wave output means, which are set for each of the speakers. The sound wave output device further includes a determination means for identifying the speaker of the sound based on the speaker information stored in the speaker information storage means and identifying the timing at which the sound is output from the right ear sound wave output means and the left ear sound wave output means according to the identified speaker and the time difference information. The control means controls the acoustic signal so that sound waves are output from the left ear sound wave output means and the right ear sound wave output means at the timing indicated by the identified time difference information. The sound wave output device according to any one of claims 1 to 5, characterized by the above.

7. Among the sound waves that the user hears through the sound wave output means that converts the acoustic signal into a sound wave and outputs it, sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear is generated. The characteristics of the acoustic signal based on the voice of the call partner are controlled so that the deviation between the frequency characteristics indicated by the acoustic characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having frequency components that can be heard by the user, where the sound wave of the reference sound is reflected inside the user's ear and the characteristics of the sound wave are set to be easy to hear for the user, becomes small, and the acoustic signal is input to the sound wave output means via the amplification means. Sound wave output method.

8. On a computer, A generation function that generates acoustic characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves that the user can hear by the sound wave output means that converts the acoustic signal into a sound wave and outputs it; A control function that controls the characteristics of the acoustic signal based on the voice of the call partner so that the deviation between the frequency characteristics indicated by the acoustic characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having frequency components that can be heard by the user, where the sound wave of the reference sound is reflected inside the user's ear and the characteristics of the sound wave are set to be easy to hear for the user, becomes small, and the acoustic signal is input to the sound wave output means via the amplification means; A sound wave output program for realizing the above.

Images