Information processing device, information processing method, information processing program, and information processing system
By performing phase inversion processing on the specific frequency band of the target sound in the information processing device, the problem of unnatural audibility in the BMLD processing is solved, and a more natural auditory experience in the noisy environment is achieved.
Patent Information
- Application Number
- CN202380029230.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-31
- Filing Date
- 2023-03-20
- Publication Date
- 2024-11-08
AI Technical Summary
When performing binaural masking difference (BMLD) signal processing, the possible phase reversal processing sound causes the listener to feel unnatural audibility problems.
By performing phase inversion processing on only the specific frequency band of the target sound in the information processing device, the frequency bands are divided into inversion bands and non-inversion bands, and the additive signal is sent synchronously with the original sound signal, providing information of natural audibility Processing method.
It effectively solves the problem of unnatural audibility in BMLD processing, improves the listener's auditory experience, and makes the target sound easier to listen in a noisy environment.
Smart Images

Figure CN118923134A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information processing device, an information processing method, an information processing program, and an information processing system. Background Art
[0002] Various techniques have been proposed for enhancing speech desired to be listened to using signal processing applying binaural masking level difference (BMLD) which is one of human auditory psychological phenomena.
[0003] For example, Patent Document 1 proposes a hearing aid system that increases the perceived sound pressure level by estimating a target sound from external sounds and separating the target sound from ambient noise to set opposite phases between both ears.
[0004] Patent Document 2 proposes a system that reproduces ambient noise having a sound pressure level corresponding to a listener's position to prevent a portion of the listeners from listening to speech received in a vehicle.
[0005] Reference List
[0006] Patent Literature
[0007] Patent Document 1: JP 2015-39208A
[0008] Patent Document 2: JP 2014-176052 A Summary of the invention
[0009] Technical issues
[0010] However, in the related art, as a result of performing signal processing to which the binaural masking level difference is applied, there may occur a problem that the sound subjected to the phase inversion processing is heard floating and gives unnatural audibility to the listener.
[0011] Therefore, the present disclosure proposes an information processing apparatus, an information processing method, an information processing program, and an information processing system that can give natural audibility to a listener in signal processing applying a binaural masking level difference.
[0012] Solution to the problem
[0013] In order to solve the above-mentioned problems, according to an embodiment of the present disclosure, an information processing device that provides services requiring identity authentication processing includes: an information processing device, including: a signal copying unit, which copies a sound signal of a target sound set as a processing target; a frequency band division unit, which divides the frequency band of the target sound into an inversion frequency band set as the target of the phase inversion processing and a non-inversion frequency band not set as the target of the phase inversion processing; a signal inversion unit, which generates an inversion signal obtained by inverting the phase of a first sound signal corresponding to the inversion frequency band; a signal addition unit, which generates an addition signal obtained by adding the inversion signal and a second sound signal corresponding to the non-inversion frequency band; a buffer unit, which temporarily stores the original sound signal of the target sound before processing; and a signal transmission unit, which sends the addition signal and the original sound signal stored in the buffer unit to an external device while synchronizing the addition signal and the original sound signal. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 is a diagram for explaining the outline of BMLD.
[0015] Figure 2 is a diagram showing an example of frequency characteristics of a BMLD.
[0016] Figure 3 is a diagram illustrating an example of a signal processing method according to a comparative example.
[0017] Figure 4 is a diagram illustrating an example of a signal processing method according to an embodiment of the present disclosure.
[0018] Figure 5 1 is a schematic diagram showing a configuration example of an information processing system according to a first embodiment of the present disclosure.
[0019] Figure 6 It is a block diagram showing an example of the configuration of devices included in the information processing system according to the first embodiment of the present disclosure.
[0020] Figure 7 is a diagram illustrating an example of a user interface according to the first embodiment of the present disclosure.
[0021] Figure 8 It is a diagram for explaining a specific example of the units of the reproduction device according to the first embodiment of the present disclosure.
[0022] Fig. 9 It is a diagram for explaining a specific example of the units of the reproduction device according to the first embodiment of the present disclosure.
[0023] Fig.10It is a diagram for explaining a modified example of the unit of the reproduction device according to the first embodiment of the present disclosure.
[0024] Fig.11 : is a flowchart showing an example of a processing procedure (Part 1) of the signal processing method according to the first embodiment of the present disclosure.
[0025] Fig.12 : is a flowchart showing an example of a processing procedure (Part 2) of the signal processing method according to the first embodiment of the present disclosure.
[0026] Fig.13 is a schematic diagram showing a configuration example of an information processing system according to a second embodiment of the present disclosure.
[0027] Fig.14 is a block diagram showing an example of a configuration of devices included in an information processing system according to a second embodiment of the present disclosure.
[0028] Fig.15 : is a diagram for explaining a specific example of the units of the information processing device according to the second embodiment of the present disclosure.
[0029] Fig.16 2 is a diagram for explaining a specific example of units of the information processing device according to the second embodiment of the present disclosure.
[0030] Fig.17 : is a flowchart showing an example of a processing procedure (Part 1) of a reproduction device according to a modification example of the present disclosure.
[0031] Fig.18 : is a flowchart showing an example of a processing procedure (Part 2) of a reproduction device according to a modification example of the present disclosure.
[0032] Fig.19 : is a block diagram showing a hardware configuration example of a computer corresponding to a device according to the embodiment and modification examples of the present disclosure. DETAILED DESCRIPTION
[0033] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. In addition, in the embodiments described below, redundant descriptions of structural elements having substantially the same functional structure are sometimes omitted by attaching the same reference numerals and symbols to the structural elements. In this specification and the accompanying drawings, multiple components having substantially the same functional configuration are sometimes distinguished and explained by adding different numbers or symbols after the same numbers or symbols.
[0034] The explanation of the present disclosure is made according to the order of items described below.
[0035] 1. Introduction
[0036] 2. First Implementation
[0037] 2-1. Overview of Signal Processing Method According to Comparative Example
[0038] 2-2. Overview of Signal Processing Method According to First Embodiment
[0039] 2-3. System Configuration Example
[0040] 2-4. Device configuration example
[0041] 2-4-1. Configuration example of sound output device
[0042] 2-4-2. Configuration Example of Reproduction Device
[0043] 2-4-3. Specific example of the unit of the reproduction device
[0044] 2-4-4. Modification of the unit of the reproduction device
[0045] 2-5. Processing example
[0046] 2-5-1. Processing in the frequency domain
[0047] 2-5-2. Time domain processing
[0048] 3. Second Implementation
[0049] 3-1. System Configuration Example
[0050] 3-2. Device configuration example
[0051] 3-2-1. Example of configuration of communication terminal
[0052] 3-2-2. Headphone configuration example
[0053] 3-2-3. Configuration example of information processing device
[0054] 3-2-4. Specific examples of units of information processing device
[0055] 4. Modifications
[0056] 4-1. Processing procedure when target sound is a stereo signal (Part 1)
[0057] 4-2. Processing procedure when target sound is a stereo signal (Part 2)
[0058] 5. Others
[0059] 6. Hardware Configuration Example
[0060] 7. Conclusion
[0061] <<1. Introduction>>
[0062] The following reference Figure 1 An overview of binaural masking level difference (hereinafter referred to as “BMLD”), which is one of the human auditory psychological phenomena, will be described. Figure 1 is a diagram for explaining the outline of BMLD. Figure 1 In the figure, "S" represents the sound signal of the target sound, which is the sound you want to listen to, and "N" represents the sound signal of the masker, which is the interference sound that blocks the target sound. "○(S or N) 0" means that there is no phase difference between the sounds in the two ears (left and right). "○(S or N) π "Indicates that the sounds in both ears (left and right) are in opposite phases to each other."○(S or N) u ” means there is no correlation between the sounds in the two ears (left and right).
[0063] The presence of a masker that makes it difficult to detect a target sound is called masking. The sound pressure level of the target sound when the masker can hardly detect the target sound when the masker sound pressure is constant is called the masking threshold. Figure 1 As shown in Modes A and B in , the difference between the masking threshold when listening to the target sound of the same phase (e.g., white noise) under the masker of the same phase and the masking threshold when listening to the target sound of the opposite phase (white noise) between the two ears under the masker of the same phase is BMLD. BMLD occurs even if the phase difference of the target sound between the two ears changes to any value other than 180 degrees (π). However, when the phase difference of the target sound between the two ears is set to 180 degrees (π), BMLD is maximized and it is easier to listen to the target sound. For example, a verification result has been reported, in which, when comparing the case of listening to the target sound of the opposite phase between the two ears in the same white noise environment with the case of listening to the target sound of the same phase between the two ears in the same white noise environment, when the target sound of the opposite phase is set between the two ears, a psychological volume increase equal to 15dB (decibel) is given to the listener. (Hirsh, IJ (1948). "The influence of interaural phase on interaural summation and inhibition." Journal of the Acoustical Society of America, 20, 536-544. Internet URL: https: / / doi.org / 10.1121 / 1.1906407).
[0064] It should be noted that Figure 1As shown in Modes A and C in FIG. 1 , BMLD also occurs in a method in which the target sound between the two ears is kept in the same phase and the masker (e.g., white noise) is made uncorrelated between the two ears. For example, in this case, a verification result has been reported in which a psychological volume increase equal to 13 dB (decibel) is given to the listener. As described above, in BMLD, the effect of making it easy to listen to the target sound in a masker environment is recognized.
[0065] By the way, when the processing of BMLD has been performed, the sound subjected to phase inversion processing sometimes has "spherical" audibility and is heard floating to give unnatural audibility to the listener. The factor of this audibility is believed to be in the human auditory peripheral organs. The sound from the ear is decomposed into frequencies by the cochlear duct in the inner ear. The phase difference of the sound between the two ears is calculated in a processing mechanism called the brain trunk (which is the entrance to the brain). At this time, the phase difference of the sound between the two ears is easily perceived, especially in the low frequency band. However, in the high frequency band, it is sometimes less easy to perceive the phase difference of the sound between the two ears.
[0066] An information processing device according to one aspect of the present disclosure (hereinafter, referred to as "the information processing device of the present disclosure") aims to solve the problem of listener's audibility that may occur when performing BMLD processing. The information processing device of the present disclosure performs signal processing for inverting the phase of only the sound components in a specific frequency band in the target sound, and thereby solves the problem of listener's audibility that may occur when performing BMLD processing.
[0067] In the following description, a reproduction device is described as an example of an information processing device of the present disclosure. Examples of the reproduction device include an audio reproduction device, a communication terminal (e.g., a smart phone), and a personal computer. If the reproduction device is a device that reproduces stereo sound and can be a new reproduction device, the reproduction device is not limited to the existing reproduction device. It is assumed that a sound output device (e.g., in-ear headphones or earphones for stereo reproduction) is used to listen to a sound signal processed by a signal processing method according to the present embodiment. A listener wearing the sound output device is referred to as a "user" for short.
[0068] In the following description, the frequency band of the target sound that is set as the target of the phase inversion process is referred to as the "inversion frequency band". The frequency band of the target sound that is not set as the target of the phase inversion process is referred to as the "non-inversion frequency band". Frequency division of the sound in the process of signal processing is called band division. As described above, a unique value corresponding to the user can be determined by, for example, pre-analyzing the user's voice. The above-mentioned inversion frequency band can be changed at any time according to the frequency distribution of the target sound. The boundary line of the inversion frequency can be changed sequentially according to the noise level. The inversion frequency band can have a bandpass characteristic. The target sound is not limited to speech, and can be music. In addition, in the inversion frequency band, the frequency distribution of the noise can be analyzed at any time, and the boundary value of the inversion frequency band of the target sound can be determined according to the frequency distribution of the noise.
[0069] It is known that the magnitude of BMLD has frequency dependence. Figure 2 is a diagram showing an example of the frequency characteristics of the BMLD; Figure 2 As shown, when the target sound is a sine wave, the BMLD is maximized when the frequency of the target sound is 200 Hz (Hertz). As described above, considering the frequency dependency of the amplitude of the BMLD, the inversion frequency band can be determined.
[0070] As a usage scenario of the signal processing performed by the information processing device of the present disclosure, an environment in which noise in a train, a crowd, etc. is assumed (hereinafter, referred to as "in a noisy environment"). Therefore, in a noisy environment, when a sound source (music content, voice content, etc.) pre-stored in a reproduction device is listened to via headphones or when a call is made through the reproduction device, an effect of making it easy to listen to a target sound (such as music, voice, or call voice) without causing discomfort can be expected.
[0071] As a usage scenario of the signal processing performed by the information processing device of the present disclosure, online communication is assumed. Therefore, when the voice (target sound) of a specific person among the participants in an online conference or the like using an online communication tool overlaps with the voices or noise of other participants, an effect of making it possible to listen to the voice of the specific person without feeling uncomfortable can be expected.
[0072] <<2. First embodiment>>
[0073] <2-1. Overview of Signal Processing Method According to Comparative Example>
[0074] In the following description, an overview of a signal processing method according to a comparative example is explained. Figure 3 is a diagram illustrating an example of a signal processing method according to a comparative example.
[0075] like Figure 3As shown, the reproducing apparatus 100EX according to the comparative example reproduces a target sound (monaural signal) reproduced in a noisy environment (step S1). The reproduced target sound is regarded as a sound signal for two channels, left and right.
[0076] The reproducing apparatus 100EX according to the comparative example inverts the phase of one of the sound signals for two channels (step S2 ). Note that the reproducing apparatus 100EX according to the comparative example does not invert the phase of the other sound signal.
[0077] The reproduction device 100EX according to the comparative example outputs a sound signal with an inverted phase and a sound signal with a non-inverted phase to the sound output device 10EX while synchronizing the sound signal. For example, in the sound signals of the two channels, the reproduction device 100EX outputs a sound signal with an inverted phase through a functional channel, and outputs a sound signal with a non-inverted phase through a non-functional channel. For example, the reproduction device 100EX outputs a sound signal with an inverted phase to a unit for the left ear corresponding to a functional channel (Lch) in the sound output device 10EX. The reproduction device 100EX outputs a sound signal with a non-inverted phase to a unit for the right ear corresponding to a non-functional channel (Rch) in the sound output device 10EX (step S3). Therefore, the sound output device 10EX can provide a target sound given a BMLD effect in a noisy environment to a user wearing the sound output device 10EX.
[0078] <2-2. Overview of Signal Processing Method According to First Embodiment>
[0079] In the following description, an overview of a signal processing method according to an embodiment of the present disclosure is described. The signal processing method according to an embodiment of the present disclosure differs from the signal processing method according to the comparative example in that only a specific frequency band of the target sound is phase-inverted. For example, the signal processing method according to an embodiment of the present disclosure phase-inverts only the sound components of the frequency band that are less likely to affect the perception of the phase difference of the sound between the two ears (between the left ear and the right ear). Therefore, the signal processing method according to an embodiment of the present disclosure can make it easy to listen to the high-frequency side of the frequency band of, for example, the target sound, where the phase difference of the sound between the two ears is less likely to be perceived. Figure 4 is a diagram illustrating an example of a signal processing method according to an embodiment of the present disclosure.
[0080] like Figure 4 As shown in FIG. 1 , the reproduction device 100 according to the present embodiment reproduces the target sound (monaural signal) reproduced in a noisy environment (step S11). Here, the reproduction device 100 temporarily stores the target sound (original sound signal) or the reproduced sound (reproduced signal) obtained by reproducing the target sound. Note that any sound such as music or speech is assumed to be the target sound.
[0081] Subsequently, the reproduction device 100 divides the frequency band of the target sound (original sound signal) or the reproduced sound (reproduced signal) obtained by replicating the target sound into an inversion frequency band set as a phase inversion target and a non-inversion frequency band not set as a phase inversion target (step S12). For example, the reproduction device 100 performs frequency analysis on the original sound signal or the reproduced signal (hereinafter collectively referred to as "sound signal") and divides the sound signal in the frequency domain. Specifically, the reproduction device 100 divides the sound signal into an inversion frequency band and a non-inversion frequency based on the frequency characteristics of the sound signal obtained by the frequency analysis.
[0082] In the case of a specific person's voice or a specific instrument's voice, as the inversion frequency band, a unique value can be determined for the person or instrument by, for example, analyzing the power distribution of the frequency in advance. The inversion frequency band can be changed at any time according to the frequency distribution. It should be noted that although Figure 4 , the inversion frequency band having a high-pass characteristic is exemplified, but the inversion frequency band may be optionally adjusted according to the frequency characteristics of the target sound. The inversion frequency band may have a low-pass characteristic or a band-pass characteristic. For example, as described above, the reproduction apparatus 100 may determine the inversion frequency band of the target sound using the frequency correlation of the BMLD. It should be noted that although Figure 4 An example of the frequency characteristics of the target sound is shown, but the frequency components included in the target sound are not limited to Figure 4 When the target sound includes any frequency component, the inverse frequency band of the target sound can be determined in the same manner using the frequency dependency of the BMLD.
[0083] Subsequently, the reproducing apparatus 100 inverts the phase of the first sound signal belonging to the inversion frequency band in the frequency band of the sound signal (step S13 ), and generates an inversion signal.
[0084] Subsequently, the reproduction apparatus 100 adds the inversion signal and the second sound signal belonging to the non-inversion frequency band in the frequency band of the sound signal (step S14 ), and generates an addition signal Thus, the reproduction apparatus 100 partially imparts the effect of BMLD to the target sound.
[0085] Then, while synchronizing the added signal and the sound signal or the copy signal, the reproducing device 100 outputs the added signal generated in step S14 and the temporarily stored original sound signal or the copy signal to the sound output device 10 (step S15).
[0086] As described above, the reproducing device 100 according to the embodiment of the present disclosure can impart the effect of BMLD to a specific frequency band by phase-inverting only the specific frequency band, and can impart natural audibility to the listener in signal processing applying the binaural masking level difference.
[0087] <2-3. System Configuration Example>
[0088] In the following instructions, refer to Figure 5 The configuration of an information processing system 1A according to a first embodiment of the present invention is described. Figure 5 1 is a schematic diagram showing a configuration example of an information processing system according to a first embodiment of the present disclosure.
[0089] like Figure 5 As shown, the information processing system 1A according to the first embodiment includes a sound output device 10 and a reproduction device 100. The sound output device 10 and the reproduction device 100 are connected to a network N. The sound output device 10 and the reproduction device 100 can communicate with each other via the network N. It should be noted that the sound output device 10 and the reproduction device 100 are not limited to wireless connection, and can be wired via a predetermined interface.
[0090] The network N may include a public line network such as the Internet, a telephone line network, or a satellite communication network, a different local area network (LAN) including Ethernet (registered trademark), or a wide area network (WAN). The network N may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network). The network N may include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).
[0091] The sound output device 10 is a device that outputs sound corresponding to the sound signal transmitted from the reproduction device 100. The sound output device 10 is, for example, earphones, in-ear headphones, headphones, etc. for stereo reproduction.
[0092] The reproduction device 100 is an information processing device that transmits a sound signal corresponding to a sound source (music content or voice content), a call voice, etc. to the sound output device 10. The reproduction device 100 can be implemented by a PC (personal computer), a notebook PC, a tablet terminal, a smart phone, a PDA (personal digital assistant), etc.
[0093] It should be noted that the information processing system 1A is not limited to the example in which the information processing system 1A is configured to include a sound output device 10 and a reproduction device 100 that are physically independent of each other, and may be a physically integrated information processing terminal such as a wearable device such as an HMD (head mounted display).
[0094] <2-4. Device Configuration Example>
[0095] In the following instructions, refer to Figure 6 A device configuration included in the information processing system 1A according to the first embodiment of the present disclosure is explained. Figure 6It is a block diagram showing an example of the configuration of devices included in the information processing system according to the first embodiment of the present disclosure.
[0096] (2-4-1. Configuration Example of Sound Output Device)
[0097] like Figure 6 As shown, the sound output device 10 included in the information processing system 1A includes an input unit 11, an output unit 12, a communication unit 13, a storage unit 14, and a control unit 15. It should be noted that Figure 6 An example of the functional configuration of the sound output device 10 according to the first embodiment is shown. The sound output device 10 is not limited to Figure 6 The example shown in and may have another configuration.
[0098] The input unit 11 receives inputs of various operations. The input unit 11 may include switches, buttons, etc. for receiving inputs of operations such as for changing the volume of a sound source (music content, voice content, call voice, etc.) being output. When the sound output device 10 is a headset, the input unit 11 includes a voice input device such as a microphone for inputting the user's voice, etc. For example, the input unit 11 can acquire the sound (ambient sound) around the sound output device 10. The input unit 11 transmits the sound signal of the acquired sound to the control unit 15 described below. The input unit 11 may include a shooting device such as a digital camera for shooting the user and the user's surroundings.
[0099] The output unit 12 outputs sounds corresponding to the sound signals of the two channels received from the reproduction device 100. The output unit 12 is implemented by an output device such as a speaker. For example, when the sound output device 10 is a dynamic type headphone, the output unit 12 includes a driver unit that reproduces the sound signal received from the reproduction device 100. The output unit 12 includes a unit for the right ear that outputs the sound to the right ear of the user and a unit for the left ear that outputs the sound to the left ear of the user.
[0100] The communication unit 13 sends and receives various information. The communication unit 13 is implemented by a communication module or the like, and is used to transmit data to another device (e.g., the reproduction device 100) by wire or radio and receive data from another device. For example, the communication unit 13 may include a communication module for communicating with another device such as the reproduction device 100 in a scheme such as a wired LAN (local area network), a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), or short-range or contactless communication.
[0101] For example, the communication unit 13 may transmit and receive control information for wirelessly connecting to the reproduction device 100, information on compression of a sound signal, etc. to and from the reproduction device 100. For example, the communication unit 13 receives a sound signal transmitted from the reproduction device 100. For example, the communication unit 13 may send a change request for changing the volume of a sound source (music content, voice content, call voice, etc.) being output to the reproduction device 100.
[0102] The storage unit 14 is implemented by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 14 may store programs, data, and the like for realizing various processing functions to be performed by the control unit 15. The programs stored by the storage unit 14 include an OS (Operating System) and various application programs. For example, the storage unit 14 may store a program and control information for performing pairing with the reproduction device 100, a program for performing processing related to a sound signal received from the reproduction device 100, a program and data for performing information processing according to the first embodiment, and the like.
[0103] The control unit 15 is implemented by a control circuit including a processor and a memory. For example, by executing the processor using the internal memory as a work area, various processes performed by the control unit 15 are implemented by commands described in the program read from the internal memory. The program read by the processor from the internal memory includes an OS (operating system) and an application program. The control unit 15 can be implemented by, for example, an integrated circuit such as an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), or an SoC (system on chip).
[0104] The main storage device and the auxiliary storage device used as the above-mentioned internal memory are realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.
[0105] like Figure 6 As shown, the control unit 15 includes a noise detection unit 15a, a signal receiving unit 15b, a first signal output unit 15c and a second signal output unit 15d.
[0106] The noise detection unit 15a detects noise (e.g., white noise) around the sound output device 10 in real time at predetermined short time intervals. For example, the noise detection unit 15a determines whether the sound pressure level of the sound signal of the ambient sound acquired by the input unit 11 is equal to or higher than a predetermined threshold value. The noise detection unit 15a, when it is determined that the sound pressure level of the sound signal of the ambient sound is above the prescribed threshold value, transmits a command signal (ON) for requesting execution of the signal processing method according to the first embodiment to the reproduction device 100 via the communication unit 13. After transmitting the command signal (ON) for requesting the start of execution of the signal processing method according to the first embodiment, when it is determined that the sound pressure level of the sound signal of the ambient sound is lower than the predetermined threshold value, the noise detection unit 15a transmits a command signal (OFF) for requesting the end of execution of the signal processing method according to the first embodiment to the reproduction device 100 via the communication unit 13.
[0107] The signal receiving unit 15b receives the sound signals of the two channels transmitted from the reproduction device 100 through the communication unit 13. The signal receiving unit 15b sends the received sound signals of the two channels to the first signal output unit 15c and the second signal output unit 15d of the corresponding channels, respectively. For example, when the first signal output unit 15c corresponds to a functional channel (e.g., "Lch"), the signal receiving unit 15b sends the sound signal corresponding to the functional channel to the first signal output unit 15c. When the second signal output unit 15d corresponds to a non-functional channel (e.g., "Rch"), the signal receiving unit 15b transmits the sound signal corresponding to the non-functional channel to the second signal output unit 15d.
[0108] The first signal output unit 15c outputs the sound signal acquired from the signal receiving unit 15b to a unit corresponding to the functional channel (for example, a unit for the left ear) through a path corresponding to the functional channel (for example, "Lch").
[0109] The second signal output unit 15d outputs the sound signal acquired from the signal receiving unit 15b to a unit corresponding to the non-functional channel (e.g., a unit for the right ear) through a path (e.g., “Rch”) corresponding to the non-functional channel.
[0110] (2-4-2. Configuration Example of Reproduction Device)
[0111] In the following description, a configuration example of the reproduction device 100 according to the first embodiment of the present disclosure is described. Figure 6 As shown, the reproduction device 100 included in the information processing system 1A includes an input unit 110, an output unit 120, a communication unit 130, a storage unit 140, and a control unit 150.
[0112] The input unit 110 receives inputs of various operations. The input unit 110 may include switches, buttons, etc., for receiving inputs such as operations for changing the volume of a sound source being output (music content, voice content, call voice, etc.). The input unit 110 may include a shooting device such as a digital camera for shooting the user and the user's surroundings.
[0113] For example, the input unit 110 receives an operation input from a user through a user interface outputted to the output unit 120 by the control unit 150 described below. The input unit 110 transmits information related to the operation input to the control unit 150 described below.
[0114] The output unit 120 outputs various information. The output unit 120 is implemented by an output device such as a display or a speaker. For example, the output unit 120 displays a user interface for receiving an operation input from a user in response to a request from a control unit 150 described below.
[0115] The communication unit 130 sends and receives various information. The communication unit 130 is implemented by a communication module or the like for transmitting data to and receiving data from other devices such as the sound output device 10 by wire or radio. For example, the communication unit 130 may include a communication module for communicating with other devices such as the sound output device 10 in a scheme such as a wired LAN (Local Area Network), a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), or short-range or contactless communication.
[0116] For example, the communication unit 130 transmits a sound signal generated by the control unit 150 described below, etc. to the sound output device 10. The communication unit 130 receives a command signal for requesting execution of the signal processing method according to the first embodiment from the sound output device 10. The communication unit 130 transmits control information for wireless connection to the sound output device 10, information on compression of the sound signal, etc. to the sound output device 10.
[0117] The storage unit 140 is implemented by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 14 can store programs, data, etc. for realizing various processing functions to be performed by the control unit 15. The programs stored by the storage unit 14 include an OS (Operating System) and various application programs.
[0118] like Figure 6 As shown, the storage unit 140 includes an environment information storage unit 141 , a parameter information storage unit 142 , and a content storage unit 143 .
[0119] The environment information storage unit 141 stores information about environment settings set by the user. For example, the information about environment settings stored in the environment information storage unit 141 includes information about a function channel selected by the user.
[0120] The parameter information storage unit 142 stores information about parameters for signal processing set by the user. For example, the parameters for signal processing stored in the parameter information storage unit 142 include information indicating the frequency band for dividing the inversion frequency band set as the target of the phase inversion processing and the non-inversion frequency band not set as the target of the phase inversion processing in the sound signal.
[0121] The content storage unit 143 stores information on sound sources, such as music content and voice content. The information on these sound sources may be target sounds processed by the signal processing method according to the first embodiment.
[0122] The control unit 150 is implemented by a control circuit including a processor and a memory. For example, by executing the processor using the internal memory as a work area, various processes performed by the control unit 150 are implemented by commands described in the program read from the internal memory. The program read by the processor from the internal memory includes an OS (operating system) and an application program. The control unit 150 can be implemented by an integrated circuit such as an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), or a SoC (system on chip).
[0123] like Figure 6 As shown, the control unit 150 includes an execution command unit 151, a signal copying unit 152, a frequency band division unit 153, a signal inversion unit 154, a signal addition unit 155, a buffer unit 156, a signal transmission unit 157, and a setting unit 158. Among these units included in the control unit 150, the signal copying unit 152, the frequency band division unit 153, the signal inversion unit 154, the signal addition unit 155, the buffer unit 156, and the setting unit 158 configure a signal processing block that provides a function for performing the signal processing method according to the first embodiment.
[0124] The execution command unit 151 controls the signal processing block according to the command signal transmitted from the sound output device 10 to execute the processing related to the signal processing method according to the first embodiment.
[0125] The signal copy unit 152 generates a copy signal obtained by copying a sound signal corresponding to a sound source or the like stored in the content storage unit 143 .
[0126] The frequency band dividing unit 153 divides one frequency band among the sound signal corresponding to the original sound source and the replica signal generated by the signal replicating unit 152 into an inversion frequency band set as a phase inversion target and a non-inversion frequency band not set as a phase inversion target.
[0127] The signal inversion unit 154 generates an inversion signal obtained by inverting the phase of the first sound signal belonging to the inversion frequency band.
[0128] The signal adding unit 155 generates a summed signal obtained by adding the inversion signal and the second sound signal belonging to the non-inversion frequency band.
[0129] The buffer unit 156 temporarily stores a sound signal corresponding to an original sound source or a copy signal generated by the signal copy unit 152 .
[0130] The signal transmission unit 157 outputs the addition signal generated by the signal addition unit 155 and the sound signal or the reproduction signal stored in the buffer unit 156 to the sound output device 10 via the communication unit 130 while synchronizing the addition signal with the sound signal or the reproduction signal.
[0131] The setting unit 158 receives various settings through a user interface provided to the user. Figure 6 As shown, the setting unit 158 includes an environment setting unit 158a and a parameter setting unit 158b. Figure 7 is a diagram illustrating an example of a user interface according to the first embodiment of the present disclosure.
[0132] The environment setting unit 158a is configured by Figure 7 The initial setting area 7 - 1 of the setting screen (user interface) illustrated in FIG. 7 receives a selection of a function channel from the user. Figure 7 The illustrated initial setting area 7-1 includes an operation unit 7-1_P1 that receives a selection operation of a function channel from a user. Figure 7 A state in which a channel (Lch) corresponding to the left unit of the sound output device 10 is selected as a functional channel is shown.
[0133] The parameter setting unit 158b sets a frequency band for dividing the inversion frequency band and the non-inversion frequency band. The parameter setting unit 158b can automatically change the inversion frequency band at any time according to the characteristics of the target sound and the characteristics of the user. For example, in the case of a specific person's voice or a specific musical instrument's sound, the parameter setting unit 158b can determine the inherent values of the person and the musical instrument as the frequency band dividing the inversion frequency band and the non-inversion frequency band, for example, by analyzing the power distribution of the frequency in advance.
[0134] In addition, the parameter setting unit 158b can also change the inversion frequency band at any time according to the frequency distribution of the target sound. In addition, the parameter setting unit 158b can arbitrarily adjust the inversion frequency band according to the frequency characteristics of the target sound. The frequency characteristics can be high-pass characteristics, low-pass characteristics, or band-pass characteristics. For example, as described above, the parameter setting unit 158b can use the frequency dependence of the BMLD to determine the inversion frequency band of the target sound. In addition, the parameter setting unit 158b can use the frequency dependence of the BMLD to determine the inversion frequency band of the target sound regardless of what frequency components the target sound contains.
[0135] For example, as an example of the characteristics of the above-mentioned user, the parameter setting unit 158b can obtain data of the user's auditory characteristics by measuring the user's auditory characteristics in advance, and change the inversion frequency band at any time according to the data. Here, the user's auditory characteristics can be general characteristics or characteristics unique to individual users (personal characteristics). The parameter setting unit 158b can manually receive the setting of the inversion frequency band from the user. In this case, the parameter setting unit 158b can present the power distribution of the frequency analyzed by the band division unit 153 and the optimal value of the inversion frequency band, and enable the user to select the power distribution of the frequency and the optimal value of the inversion frequency band. When the user uses, for example, a hearing aid or a sound collector, the parameter setting unit 158b can obtain data from the user's hearing test results, audiogram, etc.
[0136] For example, the parameter setting unit 158b can Figure 7 The frequency band setting area 7-2 of the illustrated setting screen receives settings of a boundary value for separating the inversion frequency band and the non-inversion frequency band from the user. Figure 7 The frequency band setting area 7-2 illustrated in FIG. 7 includes a frequency distribution display area 7-2_P1, an operation unit 7-2_P2, a display area 7-2_P3, and a determination button 7-2_P4.
[0137] The frequency distribution display area 7 - 2_P1 displays the power distribution of the frequency of the target sound to be reproduced in an area including a horizontal axis indicating the frequency and a vertical axis indicating the power (sound pressure level). Figure 7 An example of displaying the frequency power distribution of current noise on the power distribution of the frequency of the target sound in the frequency distribution display area 7-2_P1 is shown. The frequency distribution display area 7-2_P1 displays an image 7-2_G1 and an image 7-2_G2 in association with the operation on the operation unit 7-2_P2, the image 7-2_G1 indicating the frequency band being selected and the image 7-2_G2 indicating the recommended value of the frequency band displayed in the display area 7-2_P3.
[0138] The operation unit 7 - 2_P2 receives an operation for designating a boundary value (band) for separating an inversion frequency band and a non-inversion frequency band from a user. Figure 7 The example in which the operation unit 7-2_P2 includes a slide bar (also referred to as a slider) that enables the user to specify a band dividing the inversion frequency band and the non-inversion frequency band with intuitive operation is shown. The operation unit 7-2_P2 is configured to be able to select an arbitrary band such as a high pass, low pass, or band pass band.
[0139] The display area 7-2_P3 displays the selected frequency band selected in association with the operation on the operation unit 7-2_P2. The determination button 7-2_P4 receives an operation from the user to determine the setting of the boundary value for dividing the inversion frequency band and the non-inversion frequency band. Note that the boundary value may be set in advance or may be set sequentially.
[0140] The parameter setting unit 158b is Figure 7 The recommended value display area 7-3 of the setting screen (user interface) shown in the figure prompts the user with the recommended value for dividing the inversion frequency band and the non-inversion frequency band. For example, while maintaining the BMLD effect, the parameter setting unit 158b estimates the inversion frequency band that is not likely to cause auditory discomfort to the user based on the power distribution of the frequency of the target sound.
[0141] The parameter setting unit 158b is configured by Figure 7 The auditory characteristic measurement receiving area 7-4 of the setting screen (user interface) illustrated in FIG. 1 receives an auditory characteristic measurement instruction from the user. The auditory characteristic measurement receiving area 7-4 includes a start button 7-4_P1 for starting the auditory characteristic measurement and a switch button 7-4_P2 for switching the enable and disable of the function of the auditory characteristic mode. The user can perform the measurement of the auditory characteristic by switching the switch button 7-4_P2 to "on" and operating the start button 7-4_P1.
[0142] For example, the parameter setting unit 158b may perform the user's hearing measurement based on a processing module for hearing measurement pre-stored in the reproduction device 100. When the hearing measurement is performed, the parameter setting unit 158b may store data of the hearing characteristics of each user. Figure 7 The setting screen (user interface) illustrated in the figure is only an example. The configuration example of the screen and the button is not limited to this example. For example, the setting screen (user interface) may have a voice recognition function, and the selected value or recommended value of the frequency band is determined by receiving the user's voice instead of the user using a slide bar or input operation to determine the value.
[0143] (2-4-3. Specific example of the unit of the playback device)
[0144] In the following description, specific examples of the units of the reproduction device 100 are described with reference to the drawings. Figure 8 and Fig. 9It is a diagram for explaining a specific example of the units of the reproduction device according to the first embodiment of the present disclosure. Note that in the following explanation, the operation of the units is explained in the case of executing a signal processing method of performing frequency band division on a sound signal of a target sound in a frequency domain.
[0145] like Figure 8 As shown, for example, after receiving a command signal from the sound output device 10 (noise detection unit 15a), the execution command unit 151 instructs the signal processing block to start processing regarding the signal processing method according to the first embodiment. The execution command unit 151 reads a sound signal (monaural signal) corresponding to the target sound being reproduced from the content storage unit 143, and sends the sound signal to the signal processing block.
[0146] The signal duplication unit 152 duplicates the sound signal (mono signal) read from the content storage unit 143 to generate a duplicated signal, and prepares sound signals for two channels, one for a functional channel and one for a non-functional channel. The signal duplication unit 152 sends one sound signal to the frequency band division unit 153 and sends the other sound signal to the buffer unit 156.
[0147] The frequency band division unit 153 performs Fourier transform on the sound signal acquired from the signal copying unit 152 to analyze the frequency characteristics of the sound signal. The frequency band division unit 153 refers to the parameters stored in the parameter information storage unit 142, determines the inversion frequency band corresponding to the analysis result of the frequency characteristics, and performs band division for dividing the sound signal into a component of the inversion frequency band and a component of the non-inversion frequency band. The frequency band division unit 153 generates a first sound signal obtained by performing inverse Fourier transform on the components of the inversion frequency band, and sends the first sound signal to the signal inversion unit 154, and generates a second sound signal obtained by performing inverse Fourier transform on the components of the non-inversion frequency band, and sends the second sound signal to the signal addition unit 155.
[0148] The signal inversion unit 154 performs a phase inversion process of inverting the first sound signal corresponding to the component of the inversion frequency band, and sends the inversion signal after the phase inversion to the signal addition unit 155 .
[0149] The signal adding unit 155 generates a sum signal obtained by adding the inversion signal acquired from the signal inversion unit 154 and the second sound signal acquired from the frequency band dividing unit 153. The signal adding unit 155 transmits the generated sum signal to the signal transmitting unit 157.
[0150] The buffer unit 156 temporarily stores the sound signal obtained from the signal copying unit 152, and puts the sound signal on standby until the addition signal is sent from the signal adding unit 155 to the signal transmission unit 157. In order to perform the process of dividing the frequency band of the sound signal in real time in the frequency domain, the frequency band dividing unit 153 requires sufficient sampling for the analysis of the frequency characteristics of the target sound. For this reason, when dividing the frequency band of the sound signal of the target sound in the frequency domain, first, time for accumulating sufficient sampling is required, and time for analyzing the frequency characteristics in real time is also required. Therefore, for example, the buffer unit 156 monitors the processing situation in the signal adding unit 155, and when the addition signal is sent from the signal adding unit 155 to the signal transmission unit 157, the temporarily stored sound signal is sent to the signal transmission unit 157.
[0151] When the addition signal has been acquired from the signal addition unit 155 and the sound signal has been acquired from the buffer unit 156, the signal transmission unit 157 transmits the acquired signal to the sound output device 10 through the corresponding functional channel while synchronizing the signal. For example, the signal transmission unit 157 determines the functional channel with reference to the environment setting information received by the environment setting unit 158a or the environment setting information stored in the environment information storage unit 141. Then, the signal transmission unit 157 transmits the addition signal acquired from the signal addition unit 155 to the first signal output unit 15c of the sound output device 10 corresponding to the functional channel through the functional channel, and outputs the sound signal acquired from the buffer unit 156 to the second signal output unit 15d of the sound output device 10 corresponding to the functional channel through the non-functional channel.
[0152] like Fig. 9 As shown, for example, when a command signal is not received from the sound output device 10 (noise detection unit 15a), the execution command unit 151 reads a sound signal (mono signal) corresponding to the target sound being reproduced from the content storage unit 143, and sends the read sound signal directly to the signal transmission unit 157 without passing through the signal processing block.
[0153] When the sound signal has been directly acquired from the execution command unit 151, the signal transmission unit 157 copies the acquired sound signal to generate a copied signal and prepares sound signals for two channels of the functional channel and the non-functional channel. Then, the signal transmission unit 157 transmits the sound signal to the sound output device 10 through the functional channel while synchronizing the sound signal.
[0154] (2-4-4. Modification of the unit of the reproduction device)
[0155] refer to Fig.10 Variations of the units of the playback device 100 will be described. Fig.101 is a diagram for explaining a modified example of the unit of the reproduction device according to the first embodiment of the present disclosure. Fig.10 The reproduction device 100 of the modified example shown in Figure 8 and Fig. 9 The difference between the reproduction device 100 shown in FIG. 1 and FIG. 2 is that the reproduction device 100 according to the modified example does not include the execution command unit 151. In addition, the sound output device 10 may not have the noise detection unit 15a. That is, the reproduction device 100 according to the modified example executes the signal processing method according to the first embodiment while reproducing the target sound, regardless of whether there is a command signal from the sound output device 10. It should be noted that the processing of the signal processing block is the same as that of the reference Figure 8 Therefore, detailed description of the processing is omitted.
[0156] <2-5. Processing example>
[0157] (2-5-1. Processing in the frequency domain)
[0158] In the following instructions, refer to Fig.11 The processing procedure by the reproduction device 100 according to the first embodiment of the present disclosure is described. Fig.11 : is a flowchart showing an example of a processing procedure (Part 1) of the signal processing method according to the first embodiment of the present disclosure. Fig.11 An example of a processing procedure in the case where the frequency band of a sound signal is divided in the frequency domain is shown. Fig.11 The processing shown in FIG. 1 is started according to the reproduction of a target sound (such as music content or voice content). During the reproduction of the target sound, the processing is repeatedly performed for each predetermined processing unit (short time period) for dividing the target sound. Fig.11 The processing process shown in . Fig.11 The processing procedure shown in is performed by the control unit 150 included in the reproducing device 100.
[0159] like Fig.11 As shown, the execution command unit 151 determines whether a command signal (ON) for requesting the start of execution of the signal processing method according to the first embodiment has been received (step S101).
[0160] When determining that the instruction signal has been received (step S101 ; Yes), the execution command unit 151 reads the sound signal (monaural signal) corresponding to the target sound being reproduced from the content storage unit 143 (step S102 ).
[0161] The signal copying unit 152 copies the read sound signal (monaural signal) (step S103) and generates a copied signal. The signal copying unit 152 sends one sound signal to the frequency band dividing unit 153 and sends the other sound signal to the buffer unit 156.
[0162] The frequency band dividing unit 153 analyzes the frequency characteristics of the sound signal by performing Fourier transform on the sound signal acquired from the signal copying unit 152 (step S104-1). The buffering unit 156 temporarily stores the sound signal acquired from the signal copying unit 152 and puts the sound signal on standby (step S104-2).
[0163] The frequency band division unit 153 divides the sound signal into a component of the inverted frequency band and a component of the non-inverted frequency band based on the analysis result of the frequency characteristics (step S105). The frequency band division unit 153 generates a first sound signal obtained by performing inverse Fourier transform on the component of the inverted frequency band, and sends the first sound signal to the signal inversion unit 154, and generates a second sound signal obtained by performing inverse Fourier transform on the component of the non-inverted frequency band, and sends the second sound signal to the signal addition unit 155.
[0164] The signal inversion unit 154 performs a phase inversion process of inverting the phase of the first sound signal corresponding to the component of the inversion frequency band (step S106 ). The phase inversion unit 104 sends the phase-inverted phase-inverted signal to the signal addition unit 155 .
[0165] The signal adding unit 155 adds the inversion signal acquired from the signal inversion unit 154 and the second sound signal acquired from the frequency band dividing unit 153 (step S107 ) to generate an addition signal. The signal adding unit 155 transmits the generated addition signal to the signal transmitting unit 157 .
[0166] When acquiring the added signal from the signal adding unit 155 and the sound signal from the buffer unit 156 , the signal transmission unit 157 transmits the acquired signal to the sound output device 10 through the corresponding function channel while synchronizing the signals (step S108 ).
[0167] The execution command unit 151 determines whether the reproduction of the content has been stopped (step S110).
[0168] When it is determined that the reproduction of the content has been stopped (step S110; Yes), the execution command unit 151 ends Fig.11 The processing process shown in .
[0169] On the other hand, when it is determined that the reproduction of the content has not been stopped (step S110; No), the execution command unit 151 determines whether a command signal (OFF) for requesting the end of execution of the signal processing method according to the first embodiment has been received (step S111).
[0170] When determining that the command signal (OFF) for requesting the end of execution of the signal processing method according to the first embodiment has been received (step S111; Yes), the execution command unit 151 ends the execution of the signal processing method according to the first embodiment. Fig.11 The processing process shown in .
[0171] When it is determined that a command signal (OFF) for requesting the end of execution of the signal processing method according to the first embodiment is not received (step S111; No), the execution command unit 151 returns to step S102 described above and reads a sound signal (mono signal) corresponding to the target sound being reproduced from the content storage unit 143.
[0172] (2-5-2. Processing in the time domain)
[0173] In the first embodiment described above, the reproduction device 100 may perform frequency band division on the sound signal of the target sound not in the frequency domain but in the time domain. Fig.12 An example of a processing procedure in the case where the frequency band of a sound signal is divided in the time domain is explained. Fig.12 is a flowchart showing an example of a processing procedure (part 2) of the signal processing method according to the first embodiment of the present disclosure. During the reproduction of the target sound, the steps in Fig.12 The processing procedure shown in . Fig.12 The processing shown in Fig.11 The difference between the processing shown in FIG. 204 and FIG. 204 is the processing in step S204-1 and step S204-2. Fig.12 In the processing procedure shown, Fig.11 The processing procedures in step S104 - 1 , step S106 - 1 , and step S106 - 2 included in the processing procedure shown in FIG. 1 are unnecessary.
[0174] That is, the processing from step S201 to step S203 is the same as Fig.11 The processing procedure from step S101 to step S103 shown in is the same.
[0175] Then, the band dividing unit 153 divides the band of the sound signal using the band dividing filter (step S204-1). The band dividing unit 153 divides the band of the sound signal by performing a convolution operation of the sound signal using a pre-generated band dividing filter to divide the sound signal in the time domain.
[0176] The buffer unit 156 temporarily stores the sound signal acquired from the signal copy unit 152 and puts the sound signal on standby (step S204-2). At this time, the buffer unit 156 calculates a sampling shift (time shift) that can be caused by calculation using a bandpass filter, and puts the sound signal on standby until a time corresponding to the calculated sampling shift has passed. Note that the sampling shift (time shift) is determined by the filter size of the band division filter.
[0177] Thereafter, the processing from step S205 to step S209 is the same as Fig.11 The processing from step S107 to step S111 shown in is the same.
[0178] As described above, when the frequency band of the sound signal is divided in the frequency domain, there is an advantage that it is possible to perform processing according to the characteristics of the sound source while analyzing the characteristics. On the other hand, in the frequency band division of the sound signal in the frequency domain, it is necessary to accumulate the samples required for analyzing the frequency characteristics. The response speed is not always high. When the frequency band of the sound signal is divided in the time domain, there is an advantage of high response speed because the processing can be performed directly on the samples of the target sound. On the other hand, in the frequency band division of the sound signal in the time domain, the characteristics of the target sound are not considered, and any sound source is processed uniformly. As described above, the frequency band division for the sound signal in the frequency domain and the frequency band division for the sound signal in the time domain have different advantages. Therefore, the reproduction device 100 can appropriately use the frequency band division of the sound signal in the frequency domain and appropriately use the frequency band division of the sound signal in the time domain according to the situation. For example, each type of reproduction device may have a processing form of frequency band division in the time domain or frequency band division in the frequency domain. One type of reproducing apparatus may have two forms of band division processing in the time domain and band division processing in the frequency domain combined in a control unit, and may freely change the processing form of band division even while reproducing sound.
[0179] <<3. Second Embodiment>>
[0180] <3-1. System Configuration Example>
[0181] In the above-described first embodiment, an example is described in which signal processing of inverting the phase of only the sound component of a specific frequency band is performed on a sound source (e.g., music content, voice content, etc.) pre-stored in the reproduction device 100. For example, in online communication such as an online conference, when the speech of another participant, noise, etc. overlaps with the speech of a certain participant, the signal processing method according to the first embodiment can be applied in the same manner by processing the voice signal in the speech of the previous speaker as noise.
[0182] Therefore, in the second embodiment described below, an example of information processing in the case where the signal processing method according to the first embodiment is applied to a voice signal exchanged using a communication tool for online communication is described. Fig.13 The configuration of an information processing system 1B according to a second embodiment of the present invention is explained. Fig.13 is a schematic diagram showing a configuration example of an information processing system according to a second embodiment of the present disclosure.
[0183] It should be noted that in the following description, when there is no need to particularly distinguish between the communication terminal 30a, the communication terminal 30b, and the communication terminal 30c, the communication terminal is collectively referred to as "communication terminal 30" and explained. In addition, in the following description, when there is no need to particularly distinguish between the earphone 50a, the earphone 50b, and the earphone 50c, the earphone is collectively referred to as "earphone 50".
[0184] like Fig.13 As shown, the information processing system 1B according to the second embodiment includes a plurality of pairs of headphones 50, a plurality of communication terminals 30, and an information processing device 200. The communication terminals 30 and the information processing device 200 are connected to a network N by wire or radio. The communication terminal 30 can communicate with other communication terminals 30 and information processing devices 200 through the network N. The information processing device 200 can communicate with the communication terminal 30 through the network N. The headphones 50 are respectively connected to the corresponding communication terminals 30 by wire or radio.
[0185] The network N may include a public line network such as the Internet, a telephone line network, or a satellite communication network, a different local area network (LAN) including Ethernet (registered trademark), or a wide area network (WAN). The network N may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network). The network N may include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).
[0186] The communication terminal 30 is an information processing terminal used as a communication tool for online communication. Users of the communication terminal 30 can communicate with other users who are participants in an event such as an online conference through a platform provided by the information processing device 200 by respectively operating the online communication tool.
[0187] The communication terminal 30 has various functions for realizing online communication. For example, the communication terminal 30 includes: a communication device including a modem and an antenna for communicating with other communication terminals 30 or information processing device 200 through network N; and a display device including a liquid crystal display and a drive circuit for displaying images including still images or moving images. The communication terminal 30 includes: a voice output device, such as a speaker for outputting the voice of other users in online communication, and a voice input device, such as a microphone for inputting the voice of the user in online communication. The communication terminal 30 may include a shooting device, such as a digital camera for shooting the user and the user's surroundings.
[0188] The communication terminal 30 is realized by, for example, a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a smartphone, a PDA (Personal Digital Assistant), or a wearable device such as an HMD (Head Mounted Display).
[0189] The communication terminal 30 can output the voice of other users in the online communication, etc. to the headset 50 connected thereto. It should be noted that the headset 50 may be an in-ear headset, a hearing aid, a sound collector, etc., and the type of the headset 50 is not limited. That is, the in-ear headset may be an open-ear type or a duct type, and the hearing aid may be a CIC (completely in-ear) type, a BTE (behind-the-ear) type, a RIC (receiver in-ear) type, etc. The communication terminal 30 and the headset 50 may be configured as an information processing terminal physically and functionally integrated by a wearable device such as an HMD.
[0190] The information processing device 200 is an information processing device that provides a platform for realizing online communication to users. The information processing device 200 is implemented by a server device. The information processing device 200 may be implemented by a single server device or by a cloud system in which a plurality of server devices and a plurality of storage devices connected to a network N operate in cooperation with each other.
[0191] <3-2. Device Configuration Example>
[0192] In the following instructions, refer to Fig.14 A device configuration included in an information processing system 1B according to a second embodiment of the present disclosure is described. Fig.14 is a block diagram showing an example of a configuration of devices included in an information processing system according to a second embodiment of the present disclosure.
[0193] (3-2-1. Configuration Example of Communication Terminal)
[0194] like Fig.14 As shown, the communication terminal 30 included in the information processing system 1B includes an input unit 31, an output unit 32, a communication unit 33, a connection unit 34, a storage unit 35, and a control unit 36. It should be noted that Fig.14 An example of the functional configuration of the communication terminal 30 according to the second embodiment is shown. The communication terminal 30 is not limited to Fig.14 The example shown in and may have another configuration.
[0195] The input unit 31 receives various operations. The input unit 31 is implemented by a mouse and an input device such as a keyboard or a touch panel. The input unit 31 includes a voice input device such as a microphone for inputting the voice of the user U in online communication. The input unit 31 may include a shooting device such as a digital camera that shoots the user and the user's surroundings.
[0196] For example, the input unit 31 receives input of initial setting information about online communication. The input unit 31 receives a user's voice input uttered during execution of online communication.
[0197] The output unit 32 outputs various information. The output unit 32 is implemented by an output device such as a display or a speaker. The output unit 32 may be integrally configured to include the earphone 50 connected via the connection unit 34 or the like.
[0198] For example, the output unit 32 displays a setting window for initial setting related to online communication. The output unit 32 outputs a voice or the like corresponding to a voice signal of the other party user received by the communication unit 33 during execution of online communication.
[0199] The communication unit 33 sends and receives various information. The communication unit 33 is implemented by a communication module or the like for transmitting and receiving data to and from other devices such as other communication terminals 30 and the information processing device 200 by wire or radio. The communication unit 33 communicates with other devices in a scheme such as a wired LAN (Local Area Network), a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), or short-range or contactless communication.
[0200] For example, the communication unit 33 receives a voice signal of a communication partner from the information processing device 200 during online communication. The communication unit 33 transmits a voice signal of a user input through the input unit 31 to the information processing device 200 during online communication.
[0201] When the headset 50 includes a communication unit for wirelessly connecting to the communication terminal 30, the communication unit 33 may use a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), or wireless USB (WUSB) to establish a wireless connection with the headset 50. When the headset 50 includes a receiver for infrared communication, the communication unit 33 may transmit a voice signal using infrared rays.
[0202] The connection unit 34 is connected to other devices. For example, the connection unit 34 can establish a wired connection such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface) or MHL (Mobile High-Definition Link) with the headset 50 via a connection terminal (and, if necessary, a cable).
[0203] The storage unit 35 is implemented by, for example, a semiconductor storage element such as a RAM (random access memory) or a flash memory, a storage device such as a hard disk or an optical disk. The storage unit 35 can store, for example, programs and data for implementing various processing functions to be performed by the control unit 36. The programs stored by the storage unit 35 include an operating system (OS) and various application programs. For example, the storage unit 35 may store an application program for performing online communications such as online conferences through a platform provided from the information processing device 200. The storage unit 35 can store information indicating which of the functional channels and non-functional channels each of the first signal output unit 51 and the second signal output unit 52 included in the headset 50 corresponds to.
[0204] The control unit 36 is implemented by a control circuit including a processor and a memory. For example, by executing the processor using the internal memory as a work area, various processes to be performed by the control unit 36 are implemented by commands described in the program read from the internal memory. The program read by the processor from the internal memory includes an OS (operating system) and an application program. The control unit 36 can be implemented by an integrated circuit such as an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), or a SoC (system on chip).
[0205] For example, the main storage device and the auxiliary storage device used as the above-mentioned internal memory are realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.
[0206] like Fig.14 As shown, the control unit 36 includes a signal receiving unit 36a. The signal receiving unit 36a receives a sound signal of online communication sent from the information processing device 200 through the communication unit 33. When the first signal output unit 51 of the earphone 50 corresponds to a non-functional channel (for example, when "Rch" is used as a non-functional channel), the signal receiving unit 36a sends a voice signal for the right ear received from the information processing device 200 to the earphone 50 through the non-functional channel. When the second signal output unit 52 corresponds to a functional channel (for example, when "Lch" is used as a functional channel), the signal receiving unit 36a sends a voice signal for the left ear received from the information processing device 200 to the earphone 50 through the functional channel. Note that when the communication terminal 30 and the earphone 50 are wirelessly connected, the signal receiving unit 36a can send the voice signal to the earphone through the communication unit 33.
[0207] (3-2-2. Headphone Configuration Example)
[0208] like Fig.14 As shown, the earphone 50 included in the information processing system 1B includes a first signal output unit 51, a second signal output unit 52, a unit for the right ear 53, and a unit for the left ear 54.
[0209] For example, when "Rch" is used as a non-functional channel, the first signal output unit 51 transmits the voice signal acquired from the communication terminal 30 through a path corresponding to the non-functional channel ("Rch") to the unit for the right ear 53. The unit for the right ear 53 reproduces the voice signal received from the first signal output unit 51 as sound by changing it into the movement of the vibration membrane, and outputs the sound to the outside.
[0210] For example, when "Lch" is used as a functional channel, the second signal output unit 52 transmits the voice signal acquired from the communication terminal 30 to the unit for the left ear 54 through a path corresponding to the functional channel ("Lch"). The unit 54 for the left ear reproduces the voice signal received from the second signal output unit 52 as sound as the movement of the vibrating membrane and outputs the sound to the outside.
[0211] (3-2-3. Configuration Example of Information Processing Device)
[0212] like Fig.14 As shown, the information processing apparatus 200 included in the information processing system 1B includes a communication unit 210 , a storage unit 220 , and a control unit 230 .
[0213] The communication unit 210 sends and receives various information. The communication unit 210 is implemented by a communication module or the like for transmitting data to other devices (such as the communication terminal 30) by wire or radio and receiving data from other devices. The communication unit 210 communicates with other devices in a scheme such as a wired LAN (Local Area Network), a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), or short-range or contactless communication.
[0214] For example, the communication unit 210 receives a voice signal transmitted from the communication terminal 30. The communication unit 210 transmits the received voice signal to the control unit 230. For example, the communication unit 210 transmits a voice signal generated by the control unit 230 described below to the communication terminal 30.
[0215] The storage unit 220 is implemented by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 can store, for example, programs and data for implementing various processing functions performed by the control unit 230. The programs stored in the storage unit 220 include an operating system (OS) and various application programs.
[0216] like Fig.14 As shown, the storage unit 220 includes an environment information storage unit 221 and a parameter information storage unit 222 .
[0217] The environment information storage unit 221 stores information about environment settings set by the user. For example, the information about environment settings stored in the environment information storage unit 221 includes information about a function channel selected by the user.
[0218] The parameter information storage unit 222 stores information about parameters for signal processing set by a user. For example, the parameters for signal processing stored in the parameter information storage unit 222 include information indicating a band for dividing an inversion frequency band set as a target for phase inversion processing and a non-inversion frequency band not set as a target for phase inversion processing in a sound signal.
[0219] The control unit 230 is implemented by a control circuit including a processor and a memory. For example, by executing the processor using the internal memory as a work area, various processes to be performed by the control unit 230 are implemented by commands described in the program read from the internal memory. The program read by the processor from the internal memory includes an OS (operating system) and an application program. The control unit 230 can be implemented by an integrated circuit such as an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), or a SoC (system on chip).
[0220] like Fig.14As shown, the control unit 230 includes a signal identification unit 231, a signal copying unit 232, a frequency band division unit 233, a signal inversion unit 234, a signal addition unit 235, a buffer unit 236, a signal transmission unit 237, and a setting unit 238. Among these units included in the control unit 230, the signal copying unit 232, the frequency band division unit 233, the signal inversion unit 234, the signal addition unit 235, the buffer unit 236, and the setting unit 238 configure a signal processing block that provides a function for performing the signal processing method according to the second embodiment. It should be noted that the signal copying unit 232, the frequency band division unit 233, the signal inversion unit 234, the signal adding unit 235, the buffer unit 236, the signal transmission unit 237, and the setting unit 238 (the environment setting unit 238a and the parameter setting unit 238b) respectively correspond to the signal copying unit 152, the frequency band division unit 153, the signal inversion unit 154, the signal adding unit 155, the buffer unit 156, the signal transmission unit 157, and the setting unit 158 (the environment setting unit 158a and the parameter setting unit 158b) included in the control unit 150 of the reproduction device 100 according to the first embodiment.
[0221] When the signal strength of the first voice signal corresponding to the voice of the preceding speaker and the signal strength of the second voice signal corresponding to the voice of the middle speaker exceed a predetermined threshold, the signal recognition unit 231 detects an overlapping portion in which the first voice signal and the second voice signal are redundantly input. Then, the signal recognition unit 231 recognizes the first voice signal or the second voice signal as a phase reversal target in the overlapping portion.
[0222] For example, the signal recognition unit 231 refers to the information about the environment setting stored in the environment information storage unit 221, and recognizes the voice signal set as the phase reversal target based on the enhancement scheme corresponding to the information. The signal recognition unit 231 marks the user linked to the recognized voice signal. Thus, the signal recognition unit 231 recognizes the voice signal of the user who can be the target of the phase reversal operation among the multiple users who are participants in an event such as an online conference during the execution of online communication.
[0223] For example, when "Front" for enhancing the voice of the previous speaker is set as the corresponding enhancement scheme, after starting online communication, the signal recognition unit 231 immediately marks the user of the voice after voice input sufficient for the call starts from silence (signal equal to or less than a certain very small threshold or signal equal to or less than the sound pressure that can be recognized as voice). The signal recognition unit 231 continues to mark the voice of the target user until the voice of the target user becomes silent (signal equal to or less than a certain very small threshold or signal equal to or less than the sound pressure that can be recognized as sound).
[0224] The signal recognition unit 231 performs overlap detection to detect a voice (intervention sound) equal to or greater than a threshold value input from at least one or more other participants during the speech of the marked user (during the marking period). That is, when "previous" for emphasizing the voice of the previous speaker is set, the signal recognition unit 231 recognizes an overlapping portion where the voice signal of the previous speaker overlaps with the voice signal (intervention sound) of the intervening speaker.
[0225] When an overlap of intervening sounds is detected while marking of the target user's voice signal continues, the signal recognition unit 231 sends the voice signal acquired from the marked user as a command voice signal, and sends the voice signals acquired from other users as non-command voice signals to the signal processing block in a later stage in both paths.
[0226] (3-2-4. Specific Examples of Units of Information Processing Device)
[0227] In the following description, specific examples of each unit of the information processing device according to Embodiment 2 are described with reference to the drawings. Specific examples of each unit of the information processing system are described. Fig.15 and Fig.16 1 is a diagram for explaining a specific example of a unit of an information processing device according to a second embodiment of the present disclosure. Note that in the following explanation, a specific example of a unit in the case of setting a target sound in which the effect of BMLD is given to the voice of the previous speaker is explained. In the following explanation, it is assumed that the target sound is a monaural signal.
[0228] like Fig.15 As shown, when the front voice overlaps with the interference sound, the signal recognition unit 231 detects the overlap (hereinafter referred to as overlap detection), sends the front voice to the signal copying unit a included in the signal copying unit 232, and sends the interference sound to the signal copying unit b included in the signal copying unit 232. The overlap detection by the signal recognition unit 231 is performed according to the processing unit of the signal processing block executed in real time.
[0229] Since the front sound (mono signal) received from the signal recognition unit 231 is output as a stereo signal, the signal duplication unit a duplicates the front sound. Subsequently, the signal duplication unit a sends one of the duplicated voice signals to the frequency band division unit 233 and the remaining other to the buffer unit a included in the buffer unit 236.
[0230] When the intervening sound (mono signal) received from the signal recognition unit 231 is output as a stereo signal, the signal copying unit b copies the intervening sound. Subsequently, the signal copying unit b sends one of the copied voice signals to the buffer unit b included in the buffer unit 236, and sends the remaining other to the buffer unit c included in the buffer unit 236.
[0231] The frequency band dividing unit 233 divides the voice signal received from the signal copying unit a into a voice signal of an inverted frequency band and a voice signal of a non-inverted frequency band. Then, the band dividing unit 233 sends the voice signal of the inverted frequency band to the signal inverting unit 234, and sends the voice signal of the non-inverted frequency band to the signal adding unit a included in the signal adding unit 235.
[0232] The signal inversion unit 234 performs a phase inversion process of inverting the phase of the voice signal in the inversion frequency band received from the band division unit 233 , and transmits the generated inversion signal to the signal addition unit a included in the signal addition unit 235 .
[0233] The buffer unit 236 temporarily stores the voice signal received by each of the buffer unit a, the buffer unit b, and the buffer unit c, and waits to send the voice signal to the signal adding unit 235 until the signal processing (phase inversion processing) in the signal inversion unit 234 is completed.
[0234] Specifically, the buffer unit a temporarily stores the voice signal received from the signal copying unit a and sets the voice signal as a standby. Then, when it is detected that the signal processing in the signal inversion unit 234 is completed, the buffer unit a sends the temporarily stored voice signal to the signal addition unit b included in the signal addition unit 235. The buffer unit b caches the voice signal received from the signal copying unit b and sets the voice signal as a standby. Then, when it is detected that the signal processing in the signal inversion unit 234 is completed, the buffer unit b sends the temporarily stored voice signal to the signal addition unit a. The buffer unit c temporarily stores the sound signal received from the signal copying unit b and sets the sound signal to a standby state. Then, when it is detected that the signal processing in the signal inversion unit 234 is completed, the buffer unit c sends the voice signal to the signal addition unit b.
[0235] The signal adding unit a included in the signal adding unit 235 adds the inversion signal received from the signal inversion unit 234, the voice signal received from the buffer unit b, and the voice signal in the non-inversion frequency band received from the frequency band dividing unit 233, and sends the voice signal after addition to the signal transmission unit 237.
[0236] The signal adding unit b included in the signal adding unit 235 adds the voice signal received from the buffer unit a and the voice signal received from the buffer unit c, and transmits the voice signal after the addition to the signal transmission unit 237 .
[0237] The signal transmission unit 237 transmits the voice signals for two channels received from the signal adding unit 235 to the communication terminal 30 .
[0238] like Fig.16 As shown in , when the overlap of the previous voice and the interfering sound is not detected, the signal recognition unit 231 sends the acquired voice signal to the signal transmission unit 237 as it is.
[0239] <<4. Modification>>
[0240] <4-1. Processing procedure in case the target sound is a stereo signal (Part 1)>
[0241] When the target sound is a stereo signal, the effect of the target sound is affected by whether the sound components desired to be listened to are distributed to the left and right channels to the same extent. The signal processing method according to the first embodiment can apply the effect of BMLD to sounds including similar signal components in each of the left and right channels (for example, voices including lyrics in music content).
[0242] When the target sound is a stereo signal, the signal processing block (see FIG. 1 ) included in the reproduction apparatus 100 according to the first embodiment is omitted. Figure 8 , Fig.10 The signal processing method according to the first embodiment is performed on the sound signal of the channel side designated as the functional channel by the environment setting unit 158a.
[0243] In the following instructions, refer to Fig.17 The processing procedure performed by the reproduction device 100 when the target sound is a stereo signal will be described (Part 1). Fig.17 : is a flowchart showing an example of a processing procedure (Part 1) of a reproduction device according to a modification example of the present disclosure. Fig.17 An example of a processing procedure in the case where the frequency band of a sound signal is divided in the frequency domain is shown. Fig.17 The processing shown in FIG. 1 is the same as the processing of the reproduction device according to the first embodiment (see FIG. 1 ). Fig.11 ) is that a processing procedure in step S303 corresponding to the case where the target sound is a stereo signal is added. In the following description, the differences from the processing order of the first embodiment are described.
[0244] The execution command unit 151 determines whether the target sound is a monaural signal (step S303). When the execution command unit 151 determines that the target sound is a monaural signal (step S303; Yes), the signal copy unit 152 copies the sound signal (step S304).
[0245] On the other hand, when it is determined that the target sound is not a monaural signal (step S303; No), the execution command unit 151 sends the sound signal on the functional channel side of the sound signal to the frequency band division unit 153 and sends the sound signal on the non-functional channel side to the buffer unit 156.
[0246] <4-2. Processing procedure in case the target sound is a stereo signal (Part 2)>
[0247] In the following instructions, refer to Fig.18 The processing procedure performed by the reproduction device 100 when the target sound is a stereo signal will be described (Part 2). Fig.18 : is a flowchart showing an example of a processing procedure (Part 2) of a reproduction device according to a modification example of the present disclosure. Fig.18 The processing shown in FIG. 1 is the same as that of the reproduction apparatus according to the first embodiment (see Fig.12 ) is different from the processing procedure of the first embodiment in that a processing procedure in step S403 corresponding to the case where the target sound is a stereo signal is added. In the following description, the differences from the processing order of the first embodiment are described.
[0248] The execution command unit 151 determines whether the target sound is a monaural signal (step S403). When the execution command unit 151 determines that the target sound is a monaural signal (step S403; Yes), the signal copy unit 152 copies the sound signal (step S404).
[0249] On the other hand, when it is determined that the target sound is not a monaural signal (step S403; No), the execution command unit 151 sends the sound signal on the functional channel side of the sound signal to the frequency band division unit 153 and sends the sound signal on the non-functional channel side to the buffer unit 156.
[0250] <<5. Others>>
[0251] The signal processing method for implementing the signal processing method performed by the reproducing apparatus 100 according to the first embodiment described above (for example, see Fig.11 and Fig.12). At this time, the reproduction device 100 according to the first embodiment can realize the signal processing method (information processing method) according to the first embodiment of the present disclosure by installing various programs in the computer and executing the various programs. For realizing the signal processing method performed by the information processing device 200 according to the second embodiment (for example, see Fig.15 and Fig.16 ) can be distributed by being stored in a computer-readable recording medium such as an optical disk, a semiconductor memory, a magnetic tape, or a floppy disk. At this time, the information processing device 200 according to the second embodiment can implement the signal processing method (information processing method) according to the second embodiment of the present disclosure by installing various programs in a computer and executing the various programs.
[0252] For implementing the signal processing method performed by the reproduction device 100 according to the first embodiment (for example, see Fig.11 and Fig.12 ) can be stored in a disk device included in a server on a network (e.g., the Internet) and can be used to download to a computer. Fig.15 and Fig.16 ) can be stored in a disk device included in a server on a network such as the Internet and can be used to download to a computer. The functions provided by the various programs for implementing the signal processing method (information processing method) according to the above-described embodiments can be implemented by the cooperation of the OS and the application program. In this case, the part other than the OS can be stored in a medium and distributed, or the part other than the OS can be stored in an application server and can be used to download to a computer.
[0253] In the various processes described in the above-described embodiments and modifications, all or part of the processes described as automatically performed may be performed manually, or all or part of the processes described as manually performed may be performed automatically by a known method. In addition, unless otherwise specified, the above-described processing procedures, specific names, and information including various data and parameters described in the above-described documents and shown in the drawings may be optionally changed. For example, the various information shown in the drawings is not limited to the information shown.
[0254] The components of the reproduction device 100 according to the first embodiment described above are conceptual in function and do not always need to be configured as shown in the figure. For example, the units of the control unit 150 included in the reproduction device 100 may be functionally integrated in any unit or may be distributed. The components of the information processing device 200 according to the second embodiment described above are conceptual in function and do not always need to be configured as shown in the figure. For example, the units of the control unit 230 included in the information processing device 200 may be functionally integrated in any unit or may be distributed.
[0255] The embodiment and modification of the present disclosure can be appropriately combined within the range in which the processing contents are not contradictory. The order of the steps shown in the flowcharts according to the embodiment and modification of the present disclosure can be appropriately changed.
[0256] Although the embodiments and modifications of the present disclosure are described above, the technical scope of the present disclosure is not limited to the embodiments and modifications described above, and various changes can be made without departing from the gist of the present disclosure. Components in different embodiments and modifications can be appropriately combined.
[0257] <<7. Conclusion>>
[0258] For example, the reproduction device 100 according to the first embodiment of the present disclosure includes a signal copying unit 152, a frequency band division unit 153, a signal inversion unit 154, a signal addition unit 155, a buffer unit 156, and a signal transmission unit 157. The signal copying unit 152 copies a target sound set as a processing target. The frequency band division unit 153 divides the frequency band of the target sound into an inversion frequency band set as a target of phase inversion processing and a non-inversion frequency band not set as a target of phase inversion processing. The signal inversion unit 154 generates an inversion signal obtained by inverting the phase of a first sound signal corresponding to the inversion frequency band. The signal addition unit 155 generates an addition signal obtained by adding the inversion signal to a second sound signal corresponding to the non-inversion frequency band. The buffer unit 156 temporarily stores an original sound signal before processing. The signal transmission unit 157 transmits the addition signal and the original sound signal stored in the buffer unit 156 to an external device (e.g., the sound output device 10) while synchronizing the addition signal and the original sound signal. In this way, the reproduction device 100 according to the first embodiment of the present disclosure performs signal processing for inverting the phase of only the sound components in a specific frequency band in the target sound, thereby being able to solve the problem of listener's audibility that may occur when performing BMLD processing and give natural audibility to the listener.
[0259] In addition, the frequency band dividing unit 153 divides the frequency band of the target sound according to a boundary value set in order to divide the inversion frequency band and the non-inversion frequency band.
[0260] The frequency band dividing unit 153 divides the frequency band of the target sound according to a boundary value set based on the characteristics of the target sound or the characteristics of the environmental noise.
[0261] The frequency band dividing unit divides the frequency band of the target sound according to a boundary value set based on the auditory characteristics of the user.
[0262] In this manner, the reproducing apparatus 100 can appropriately divide the frequency band of the target sound.
[0263] The reproduction apparatus 100 further includes a parameter setting unit 158b that receives the setting of the boundary value from the user. Then, the frequency band division unit 153 divides the frequency band of the target sound according to the boundary value set by the user. In this way, the reproduction apparatus 100 can divide the frequency band of the target sound according to the user's request.
[0264] The parameter setting unit 158b presents information of a recommended value recommended as a boundary value to the user based on the analysis result of the frequency characteristics of the target sound. In this way, the reproduction apparatus 100 can assist the user's setting operation. The boundary value can take any value.
[0265] The frequency band dividing unit 153 divides the frequency band of the target sound in the frequency domain based on the analysis result of the frequency characteristics of the target sound. In this way, the reproducing apparatus 100 can perform signal processing based on the inverse frequency band matching characteristics of the target sound.
[0266] The frequency band division unit 153 divides the frequency band of the target sound in the frequency domain or the time domain. In this way, the reproducing apparatus 100 can perform responsive signal processing of the prioritized processing according to the situation.
[0267] The reproduction device 100 further includes an execution command unit 151, which receives an execution command of signal processing transmitted from an external device (e.g., the sound output device 10) under the condition that the sound pressure level of the noise exceeds a predetermined threshold. The execution command unit 151 starts the signal processing of partially reversing the phase of the target sound at the timing of receiving the execution command. In this way, the reproduction device 100 can make it easy to listen to the target sound even in a noisy environment.
[0268] The information processing device 200 according to the second embodiment can also provide natural audibility to the listener in online communication, and like the reproduction device 100, can support the listener so as to achieve smooth communication.
[0269] It should be noted that the effects described in this specification are only illustrative or exemplary and not restrictive. That is, the technology disclosed in this disclosure can achieve, together with or instead of the effects described above, other effects that are obvious to those skilled in the art from the description of this specification.
[0270] It should be noted that the technology of the present disclosure can also adopt the following configurations as configurations belonging to the technical scope of the present disclosure. (1)
[0272] An information processing device, comprising:
[0273] a signal duplication unit that duplicates a sound signal of a target sound set as a processing target;
[0274] a frequency band dividing unit that divides a frequency band of a target sound into an inversion frequency band set as a target of a phase inversion process and a non-inversion frequency band not set as a target of a phase inversion process;
[0275] a signal inversion unit that generates an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band;
[0276] a signal adding unit that generates a summed signal obtained by adding the inverted signal and a second sound signal corresponding to a non-inverted frequency band;
[0277] a buffer unit that temporarily stores an original sound signal of the target sound before processing; and
[0278] The signal transmission unit transmits the added signal and the original sound signal stored in the buffer unit to an external device while synchronizing the added signal and the original sound signal. (2)
[0280] The information processing device according to (1), wherein
[0281] Frequency band division unit
[0282] The frequency band of the target sound is divided according to a boundary value set in order to divide the inversion frequency band and the non-inversion frequency band. (3)
[0284] The information processing device according to (2), wherein
[0285] Frequency band division unit
[0286] The frequency band of the target sound is divided according to a boundary value set based on the characteristics of the target sound or the characteristics of the environmental noise. (4)
[0288] The information processing device according to (2), wherein
[0289] Frequency band division unit
[0290] The frequency band of the target sound is divided according to the boundary value set based on the auditory characteristics of the user. (5)
[0292] The information processing device according to (2) further comprises:
[0293] The parameter setting unit receives the setting of the boundary value from the user, wherein
[0294] Frequency band division unit
[0295] The frequency band of the target sound is divided according to the boundary value set by the user. (6)
[0297] The information processing device according to (5), wherein
[0298] Parameter setting unit
[0299] Based on the analysis result of the frequency characteristic of the target sound, information on the recommended value recommended as the boundary value is presented to the user. (7)
[0301] The information processing device according to (6), wherein
[0302] Frequency band division unit
[0303] The frequency band of the target sound is divided in the frequency domain based on the analysis result of the frequency characteristic of the target sound. (8)
[0305] The information processing device according to (2), wherein
[0306] Frequency band division unit
[0307] Divide the frequency bands of the target sound in the frequency domain or time domain. (9)
[0309] The information processing device according to (1), further comprising:
[0310] The execution command unit receives an execution command for signal processing sent from an external device under the condition that the sound pressure level of the noise exceeds a predetermined threshold, wherein
[0311] Execute command unit
[0312] Signal processing that partially inverts the phase of the target sound is started at an opportunity where the execution command is received. (10)
[0314] An information processing method executed by a computer, the method comprising:
[0315] Copying the sound signals of the target sound set as processing targets;
[0316] dividing a frequency band of the target sound into an inversion frequency band set as a target of the phase inversion process and a non-inversion frequency band not set as a target of the phase inversion process;
[0317] generating an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band;
[0318] generating a summed signal obtained by adding the inverted signal and a second sound signal corresponding to a non-inverted frequency band;
[0319] temporarily storing the original sound signal of the target sound before processing; and
[0320] The added signal and the original sound signal are transmitted to an external device while being synchronized with each other. (11)
[0322] Information Processing Procedure
[0323] To enable a computer to function as a control unit which:
[0324] Copying the sound signals of the target sound set as processing targets;
[0325] dividing a frequency band of the target sound into an inversion frequency band set as a target of the phase inversion process and a non-inversion frequency band not set as a target of the phase inversion process;
[0326] generating an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band;
[0327] generating a summed signal obtained by adding the inverted signal to a second sound signal corresponding to a non-inverted frequency band;
[0328] temporarily storing the original sound signal of the target sound before processing; and
[0329] The added signal and the original sound signal are transmitted to an external device while being synchronized with each other. (12)
[0331] An information processing system, comprising:
[0332] a signal duplication unit that duplicates a sound signal of a target sound set as a processing target;
[0333] a frequency band dividing unit that divides a frequency band of a target sound into an inversion frequency band set as a target of a phase inversion process and a non-inversion frequency band not set as a target of a phase inversion process;
[0334] a signal inversion unit that generates an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band;
[0335] a signal adding unit that generates a summed signal obtained by adding the inverted signal and a second sound signal corresponding to a non-inverted frequency band;
[0336] a buffer unit that temporarily stores an original sound signal of the target sound before processing; and
[0337] The signal transmission unit transmits the added signal and the original sound signal stored in the buffer unit to an external device while synchronizing the added signal and the original sound signal.
[0338] Reference Numbers List
[0339] 1A, 1B Information Processing System
[0340] 10 Sound output device
[0341] 11 Input unit
[0342] 12 Output Units
[0343] 13 Communication unit
[0344] 14 Storage Unit
[0345] 15 Control Unit
[0346] 15a Noise detection unit
[0347] 15b Signal receiving unit
[0348] 15c First signal output unit
[0349] 15d Second signal output unit
[0350] 30 Communication Terminal
[0351] 31 Input unit
[0352] 32 output units
[0353] 33 Communication unit
[0354] 34 Connection Units
[0355] 35 Storage Units
[0356] 36 Control Unit
[0357] 36a Signal receiving unit
[0358] 50 Headphones
[0359] 51 First signal output unit
[0360] 52 Second signal output unit
[0361] 53 units for right ear
[0362] 54 units for left ear
[0363] 100 Reproduction Device
[0364] 110 Input unit
[0365] 120 Output Units
[0366] 130 Communication Unit
[0367] 140 storage units
[0368] 141 Environmental information storage unit
[0369] 142 parameter information storage unit
[0370] 143 Content storage unit
[0371] 150 Control Unit
[0372] 151 Execution command unit
[0373] 152 Signal Replication Unit
[0374] 153 Band Division Unit
[0375] 154 Signal Inversion Unit
[0376] 155 Signal Adding Unit
[0377] 156 Buffer unit
[0378] 157 Signal Transmission Unit
[0379] 158 Setting Unit
[0380] 158a Environment Setting Unit
[0381] 158b Parameter setting unit
[0382] 200 Information processing device
[0383] 210 Communication Unit
[0384] 220 storage units
[0385] 221 Environmental information storage unit
[0386] 222 parameter information storage unit
[0387] 230 Control Unit
[0388] 231 Signal recognition unit
[0389] 232 Signal Replication Unit
[0390] 233 Band Division Unit
[0391] 234 Signal Inversion Unit
[0392] 235 Signal Adding Unit
[0393] 236 Buffer Unit
[0394] 237 Signal Transmission Unit
[0395] 238 Setting Unit
[0396] 238a Environment Setting Unit
[0397] 238b Parameter setting unit.
Claims
1. An information processing device, comprising: a signal duplication unit that duplicates a sound signal of a target sound set as a processing target; a frequency band dividing unit that divides the frequency band of the target sound into an inversion frequency band that is set as a target of the phase inversion process and a non-inversion frequency band that is not set as a target of the phase inversion process; a signal inversion unit that generates an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band; a signal adding unit that generates a summed signal obtained by adding the inverted signal and a second sound signal corresponding to the non-inverted frequency band; a buffer unit that temporarily stores the original sound signal of the target sound before processing; as well as A signal transmission unit transmits the added signal and the original sound signal stored in the buffer unit to an external device while synchronizing the added signal and the original sound signal.
2. The information processing device according to claim 1, wherein: The frequency band division unit The frequency band of the target sound is divided according to a boundary value set to divide the inversion frequency band and the non-inversion frequency band.
3. The information processing device according to claim 2, wherein: The frequency band division unit The frequency band of the target sound is divided according to the boundary value set based on characteristics of the target sound or characteristics of environmental noise.
4. The information processing device according to claim 2, wherein: The frequency band division unit The frequency band of the target sound is divided according to the boundary value set based on the auditory characteristics of the user.
5. The information processing device according to claim 2, further comprising: A parameter setting unit receives the setting of the boundary value from a user, wherein: The frequency band division unit The frequency band of the target sound is divided according to the boundary value set by the user.
6. The information processing device according to claim 5, wherein: The parameter setting unit Based on the analysis result of the frequency characteristic of the target sound, information on a recommended value recommended as the boundary value is presented to the user.
7. The information processing device according to claim 2, wherein: The frequency band division unit The frequency band of the target sound is divided in a frequency domain based on an analysis result of a frequency characteristic of the target sound.
8. The information processing device according to claim 2, wherein: The frequency band division unit The frequency band of the target sound is divided in the frequency domain or the time domain.
9. The information processing device according to claim 1, further comprising: An execution command unit receives an execution command for signal processing sent from the external device under the condition that the sound pressure level of the noise exceeds a predetermined threshold, wherein: The command execution unit The signal processing of partially inverting the phase of the target sound is started at a timing when the execution command is received.
10. An information processing method executed by a computer, the method comprising: a sound signal that replicates a target sound set as a processing target; dividing a frequency band of the target sound into an inversion frequency band set as a target of phase inversion processing and a non-inversion frequency band not set as a target of phase inversion processing; generating an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band; generating a summed signal obtained by adding the inverted signal and a second sound signal corresponding to the non-inverted frequency band; temporarily storing the original sound signal of the target sound before processing; and The added signal and the original sound signal are transmitted to an external device while being synchronized with each other.
11. An information processing program For causing a computer to function as a control unit, the control unit: a sound signal that replicates a target sound set as a processing target; dividing a frequency band of the target sound into an inversion frequency band set as a target of phase inversion processing and a non-inversion frequency band not set as a target of phase inversion processing; generating an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band; generating a summed signal obtained by adding the inverted signal and a second sound signal corresponding to the non-inverted frequency band; temporarily storing the original sound signal of the target sound before processing; and The added signal and the original sound signal are transmitted to an external device while being synchronized with each other.
12. An information processing system comprising: a signal duplication unit that duplicates a sound signal of a target sound set as a processing target; a frequency band dividing unit that divides the frequency band of the target sound into an inversion frequency band that is set as a target of the phase inversion process and a non-inversion frequency band that is not set as a target of the phase inversion process; a signal inversion unit that generates an inversion signal obtained by inverting the phase of the first sound signal corresponding to the inversion frequency band; a signal adding unit that generates a summed signal obtained by adding the inverted signal and a second sound signal corresponding to the non-inverted frequency band; a buffer unit for temporarily storing the original sound signal of the target sound before processing; as well as A signal transmission unit transmits the added signal and the original sound signal stored in the buffer unit to an external device while synchronizing the added signal with the original sound signal.
