AUDIO PROCESSING SYSTEM, AUDIO PROCESSING METHOD, AND PROGRAM

Abstract

This sound processing system comprises: a sound source that emits a sound, which is based on a prescribed sound source signal; a sound reception unit that acquires a sound reception signal of an arrived sound; a calculation unit that acquires, from the sound reception signal, the impulse response from the sound source to the sound reception unit; and a sound absorption material that covers an object, said object being on the path of a reflected sound that arrives at the sound reception unit at least within a prescribed observation period from the production of the aforementioned sound.

Description

[Technical field]

[0001] The present invention relates to an audio processing system, an audio processing method, and program The present invention relates to a method for acquiring a head related impulse response (HRIR) for reproducing a sound with a three-dimensional effect using, for example, headphones, earphones, etc. [Background technology]

[0002] Methods for reproducing sound with a three-dimensional effect have been proposed. For example, Patent Document 1 discloses a method of convolving the transfer characteristics of sound waves from a sound source to the left and right ears of a listener as head-related transfer functions with source sounds to synthesize two-channel acoustic signals. By presenting sounds based on the acoustic signals obtained by this method to the left and right ears of the listener, sound image localization in the direction of the sound source is realized. The HRIR corresponds to a representation of the head-related transfer function as an impulse response in the time domain. On the other hand, it is known that more than 30% of listeners cannot obtain the desired stereoscopic effect even if a sound signal is synthesized using common HRIRs in advance. Therefore, it is important to obtain HRIRs for each listener.

[0003] Patent Document 1 describes a measurement device that measures the head-related transfer function of a pseudo sound source signal generated around the subject's head by placing microphones at the subject's ears. This measurement device has a sound source storage unit, an output control unit, an output unit, an input unit, a key input unit that supports the start of measurement, an input control unit that records measurement data in the data storage unit, and a reflector. This measurement device is characterized in that, when measuring in a measurement room that has not been subjected to anechoic processing, it starts input in synchronization with the output of the output control unit, ends data input before reverberation reaches the input unit, and a reflector is placed on the path of the reflected sound that reaches the input unit within the measurement time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-307988 Summary of the Invention [Problem to be solved by the invention]

[0005] However, a special environment is required to obtain the HRIR. A special environment refers to an environment without reflection, such as an anechoic chamber, or an environment with so little reflection that its influence can be ignored. If the HRIR includes the influence of reflection, it is not possible to reproduce the sound waves arriving at the left and right ears from the target direction of the sound image localization. In this regard, the measuring device described in Patent Document 1 reduces the influence of reflection by placing a reflector in the path of the reflected sound, thereby blocking the reflection to the microphone. Therefore, placing a reflector in the path of the reflected sound is difficult for anyone other than an acoustic engineer who has knowledge of the path of the reflected sound. [Means for solving the problem]

[0006] The present invention has been made to solve the above-mentioned problems. One example of the object of the present invention is to provide an audio processing system, an audio processing method, and program The purpose of this project is to provide

[0007] According to a first aspect of the present invention, a sound processing system includes a sound source that emits a sound based on a predetermined sound source signal, a sound receiving unit that acquires a received signal of the arriving sound, a calculation unit that acquires an impulse response from the received signal from the sound source to the sound receiving unit, a sound absorbing material that covers an object on a path of a reflected sound that arrives at the sound receiving unit within at least a predetermined observation period from the generation of the sound, and a sound absorbing material in the measurement environment that performs image recognition processing on an image of a measurement environment of the impulse response. of the subject equivalent to Material and the materiala reflectance storage unit that stores reflection characteristic related data including information on the sound reflectance for each material; and a measurement environment determination unit that refers to the reflection characteristic related data, identifies a reflectance corresponding to a specified material, and determines whether or not to measure the impulse response based on an average reflectance within an evaluation area calculated from the distribution of the materials.

[0008] According to a second aspect of the present invention, there is provided an acoustic processing method for an acoustic processing system including a sound absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving unit within at least a predetermined observation period after the generation of the sound from a sound source, the method including a first step in which the sound source emits a sound based on a predetermined sound source signal, a second step in which the sound receiving unit acquires a received signal of the arriving sound, a third step in which an arithmetic processing unit acquires an impulse response from the sound source to the sound receiving unit from the received signal, and a measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to obtain the sound absorbing material in the measurement environment. of the subject equivalent to Material and the material and a fifth step in which the measurement environment determination unit refers to reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, identifies a reflectance corresponding to the identified material, and determines whether or not to measure the impulse response based on an average reflectance within the evaluation area calculated from the distribution of the materials.

[0009] According to a third aspect of the present invention, a program is provided for a computer of an acoustic processing system including a sound absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving unit within at least a predetermined observation period from the generation of a sound from a sound source, the program including a first step in which the sound source emits a sound based on a predetermined sound source signal, a second step in which the sound receiving unit acquires a received signal of the arriving sound, a third step in which an arithmetic processing unit acquires an impulse response from the sound source to the sound receiving unit from the received signal, and a measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to obtain the sound absorbing material in the measurement environment. of the subject equivalent to Material and the materialand a fifth step in which the measurement environment determination unit refers to reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, identifies a reflectance corresponding to the identified material, and determines whether or not to measure the impulse response based on the average reflectance within the evaluation area calculated from the distribution of the materials. Effect of the Invention

[0010] According to one aspect of the present invention, the HRIR can be obtained more easily. [Brief description of the drawings]

[0011] [Figure 1] 1 is a plan view showing an example of the configuration of a sound processing system according to a first embodiment. [Diagram 2] 1 is a side view showing an example of the configuration of a sound processing system according to a first embodiment. [Diagram 3] 1 is a schematic block diagram illustrating an example of a hardware configuration of a sound processing device according to a first embodiment. [Figure 4] 1 is a schematic block diagram illustrating an example of a functional configuration of a sound processing device according to a first embodiment. [Diagram 5] 5 is a flowchart showing an example of HRIR measurement processing according to the first embodiment. [Figure 6] FIG. 11 is a side view showing an example of the configuration of a sound processing system according to a second embodiment. [Figure 7] FIG. 11 is a schematic block diagram illustrating an example of the functional configuration of a control unit according to a second embodiment. [Figure 8] 13 is a flowchart showing an example of a measurement environment determination process according to the second embodiment. [Figure 9] FIG. 2 is an explanatory diagram showing an example of a configuration of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, sound processing systems according to multiple embodiments of the present invention will be described with reference to the drawings. <First embodiment> The sound processing system according to the first embodiment includes a sound source for emitting sound, a sound receiving unit for acquiring a sound receiving signal of the arriving sound, a sound absorbing material for absorbing the arriving sound, and a sound processing device. An example of the functional configuration of the sound processing device 10 will be described later. Fig. 1 is a plan view showing a configuration example of a sound processing system 1 according to this embodiment. Fig. 2 is a side view showing a configuration example of a sound processing system 1 according to this embodiment. Fig. 2 shows a side surface passing through line AA' in Fig. 1 and parallel to the vertical direction.

[0013] In the example shown in FIG. 1 and FIG. 2, the sound processing system 1 is used in an indoor space Rm. The space Rm may be a room in which no particular soundproofing or reverberation equipment is provided as a whole space. That is, the space Rm is not limited to a soundproof room or a laboratory, but may be any of an office, a classroom, a meeting room, a living room, and the like. The space Rm has a substantially rectangular parallelepiped shape and a rectangular shape in a plan view. The space Rm is surrounded by four side surfaces, one bottom surface, and one top surface. The top surface is a surface facing the bottom surface, and is depicted above the bottom surface in FIG. 1 and FIG. 2. The side surfaces, bottom surface, and top surface are collectively referred to as inner surfaces. The side surfaces, bottom surface, and top surface form the wall surface, floor surface, and ceiling, respectively. In the space Rm, two speakers Sp, a sound absorbing material As, and a seat Pf are arranged as sound sources for emitting sound.

[0014] The two speakers Sp are installed at positions close to each other within a predetermined range from one side (hereinafter referred to as a speaker installation side) that is an intersection line between two adjacent side surfaces (hereinafter referred to as adjacent surfaces) among the four side surfaces of the space Rm. However, the two speakers Sp are installed at positions of different heights in the vertical direction as shown in FIG. 2. The two speakers Sp may be distinguished from each other by calling them speakers Sp01, Sp02, etc. In FIG. 1, of the two speakers Sp, the speaker Sp01 is shown, and the speaker Sp02, which is installed at a lower position than the speaker Sp01, is not shown. A reproduction signal is input from the sound processing device 10 to any one of the speakers Sp (speaker Sp01 in the example shown in FIG. 1). The speaker Sp radiates (reproduces) a sound based on a sound source signal input from the sound processing device 10.

[0015] The seats Pf are installed so that their centers face each speaker Sp at a predetermined distance (e.g., 1 to 2 m) on the plane bisecting the angle formed by the adjacent planes. The seats Pf have a seat plate and a backrest, which enable the subject Sb to be supported motionless in a seated position. The subject Sb is a person for whom HRIRs from the speakers Sp to each of the left and right ears are to be measured.

[0016] The sound absorbing material As is installed so as to cover at least a part of the inner surface of the space Rm. In the example shown in FIG. 1 and FIG. 2, the sound absorbing material As is installed so as to cover a part of the bottom surface and a part of the adjacent surface. Although not shown, a sound absorbing material may be installed to cover the seat Pf. The sound absorbing material As includes a material with a sufficiently low reflectance of the incoming sound. The sound absorbing material As may be a material that converts most of the energy of the vibration of the incoming sound into heat, thereby absorbing the energy of the sound and dissipating it as heat. For example, glass wool, rock wool, urethane sponge, etc. can be used as the sound absorbing material As. The sound absorbing material may also be called a sound insulation material, soundproofing material, etc. The area in which the sound absorbing material As is installed will be described later.

[0017] A microphone Mic is attached to each of the left and right ears of the subject Sb as a sound receiving unit. The microphones attached to the left and right ears are distinguished by being called microphones MicL and MicR, respectively. The microphones MicL and MicR each receive sounds arriving at the respective portions and convert the received sounds into received signals. The microphones MicL and MicR each output the converted received signals to the sound processing device 10. In the following description, the received signals output from the microphones MicL and MicR may be called the left received signal and the right received signal, respectively.

[0018] Sound waves emitted from the speaker Sp propagate along various paths to the microphones MicL and MicR. When no object is placed between the speaker Sp and the microphones MicL and MicR as illustrated in Fig. 1, a part of the sound waves emitted from the speaker Sp arrives at the microphones MicL and MicR as direct sound. Another part of the sound waves is reflected by the surface of an object placed on the path and arrives at the microphones MicL and MicR as reflected sound.

[0019] Here, let us assume that the sound wave S emitted by the speaker Sp has a propagation path that includes a reflection point on the surface of an object, and is reflected at that reflection point. In this case, the reflected wave S observed at the sound receiving point where one of the microphones Mic is installed is rn is shown in equation (1).

[0020]

number

[0021] In equation (1), R and L respectively represent the reflection rate of the sound wave at the reflection point and the attenuation rate according to distance. In general, there are an infinite number of reflection points, so the reflected wave Sr arriving at the sound receiving point is expressed as the reflected wave S for each reflection point as shown in formula (2). rn In equation (2), Σ indicates the sum between the reflection points. The set of these reflection points corresponds to the surfaces of various reflectors that reflect the sound coming from the speaker Sp.

[0022]

number

[0023] Therefore, the sound wave S' arriving at the sound receiving point is a combination of the component of the direct wave S from the speaker Sp and the component of the reflected wave Sr, as shown in equation (3).

[0024]

number

[0025] On the other hand, the head impulse response indicates the propagation characteristics of sound waves from a sound source to each ear of the subject Sb. In other words, the head impulse response includes the propagation characteristics due to the reflection and diffraction of sound waves arriving from the sound source on the surface of the subject Sb's head (e.g., the pinna, etc.). To obtain a pure head impulse response, it is expected that the influence of reflected waves will be removed. Conventionally, it has been common to measure the head impulse response in an environment where no reflection of sound waves occurs, such as an anechoic chamber. This is because the contribution of reflected sound from objects installed on the propagation path of the sound waves is not included.

[0026] The sound processing system 1 according to the present embodiment reduces components of reflected sound from other objects during a period in which the head impulse response includes at least components due to reflection and diffraction of sound waves at the head (hereinafter, head transfer period). Therefore, a period including at least the head transfer period is preset as an observation period of the head impulse response in the sound processing device 10. The head transfer period is typically about 1 to 3 ms from the time when the direct wave arriving from the speaker Sp first arrives at the head of the subject Sb, depending on the size of the human head. Therefore, by covering the area of ​​the reflector that brings about the reflected sound arriving at the microphones MicL and MicR at least during the observation period with a sound absorbing material As, the reflected sound components can be reduced. Even when the sound absorbing material As is installed, it does not hinder the use of the space Rm as much as a reflector, and the work load associated with the installation is light. Therefore, as illustrated in FIG. 1, the HRIR can be acquired even in an ordinary environment such as a corner of the space Rm. However, the positional relationship between the sound source and the seat Pf or the subject Sb is not limited to those illustrated in Figures 1 and 2. For example, the seat Pf may be installed in the center of the space Rm. The shape of the space Rm is also not limited to those illustrated in Figures 1 and 2. For example, the shape of the space Rm may be a cylinder.

[0027] In the example shown in FIG. 1 and FIG. 2, the area where the sound absorbing material As is installed includes a square partial area of ​​the bottom surface from the speaker installation side including the seat Pf, and an area extending from a part of each of the two adjacent surfaces that contacts the partial area of ​​the bottom surface to a height that is a predetermined height higher than the height of the higher of at least two speakers Sp01 and Sp02. The time from the time when the speaker emits sound to the start point of the observation period (hereinafter, the observation start time) is calculated based on the distance from the speaker Sp to the subject Sb and the sound speed. The observation start time may be set in advance in the sound processing device 10, or may be calculated based on the position of the speaker Sp and the position of the seat Pf or the subject Sb. In addition, the area where the sound absorbing material As is installed may include a portion where the reflected sound arrives at the microphones MicL and MicR during the observation period, and the size and shape of the area are not limited to those exemplified in FIG. 1 and FIG. 2. For example, the shape of the area where the floor surface is covered with the sound absorbing material As may be a sector shape as exemplified in FIG. 9, and does not necessarily have to include the position of the seat Pf.

[0028] The intensity of the reflected sound arriving at each of the microphones MicL and MicR from the sound absorbing material As is not completely zero, but is sufficiently low compared to the intensity of the direct sound. In general, when two sounds of different intensities are presented to a subject, a phenomenon (masking) is known in which the weaker sound cannot be heard because it is drowned out by the stronger sound. With the sound absorbing material As, the S / N ratio when the direct sound component is the signal component and the reflected sound component is the noise component can be reduced to a level where the reflected sound cannot be perceived. Therefore, the head impulse response acquired by the sound processing system 1 according to this embodiment can also realize a three-dimensional effect on the auditory sense to the same degree as the head impulse response measured in an anechoic chamber.

[0029] Next, the configuration of the sound processing device 10 according to this embodiment will be described. FIG. 3 is a schematic block diagram showing an example of the hardware configuration of the sound processing device 10 according to the present embodiment. The sound processing device 10 includes a control unit 110, a storage unit 130, an input / output unit 140, a display unit 150, and an operation unit 160. The sound processing device 10 may be configured as a dedicated device, or may be realized by a general-purpose computer.

[0030] The control unit 110 performs processing for implementing and controlling various functions of the sound processing device 10. The control unit 110 enables processing instructed by commands written in various programs such as application programs, and executes and controls the operation of each unit constituting the sound processing device 10. The control unit 110 is configured to include, for example, one or more processors such as a CPU (Central Processing Unit).

[0031] The storage unit 130 stores various data used by the control unit 110, various data acquired by the control unit 110, various programs, etc. The storage unit 130 includes, for example, a Read Only Memory (ROM) and a Random Access Memory (RAM).

[0032] The input / output unit 140 is connected to a device separate from the sound processing device 10 in a wired or wireless manner, and inputs and outputs various data. The input / output unit 140 may be connected to a communication network in a wired or wireless manner, and transmits and receives various data to and from the separate device connected to the communication network. The input / output unit 140 is configured to include input / output devices such as an input / output interface and a communication interface, for example.

[0033] Under the control of the control unit 110, the display unit 150 displays information indicated by various display data input from the control unit 110 in a visually recognizable manner. Such display data includes image data, text data, etc. The display unit 150 is configured to include any type of display device, such as a liquid crystal display (LCD) or an organic electro-luminescence display (OLED).

[0034] The operation unit 160 accepts a user's operation, generates an operation signal according to the accepted operation, and outputs the generated operation signal to the control unit 110. The operation signal indicates various information such as a user's instruction or command to the sound processing device 10. The operation unit 160 includes an operation device in any form such as a push button or a stick key. The operation unit 160 may be configured to include a receiver that wirelessly receives (including infrared) signals from a physically separated operation device (e.g., a remote controller).

[0035] Next, an example of the functional configuration of the sound processing device 10 according to the present embodiment will be described. 4 is a schematic block diagram showing an example of the functional configuration of the sound processing device 10 according to this embodiment. The sound processing device 10 includes a control unit 110 including a measurement control unit 112, a recording unit 114, and a calculation unit .

[0036] The measurement control unit 112 performs various controls related to the measurement of the head impulse response. When an operation signal indicating the start of measurement is input from the operation unit 160, the measurement control unit 112 generates a predetermined sound source signal to be used for measurement. The sound source signal is, for example, an impulse signal. The impulse signal is a signal whose signal value at a certain sample time is significantly different from zero (non-zero value) and whose signal value at other times is zero. The measurement control unit 112 outputs the generated sound source signal to the speaker Sp via the input / output unit 140. The speaker Sp emits a sound according to the input sound source signal. The sample time at which the signal value is a non-zero value corresponds to the time at which the impulse sound is emitted from the speaker Sp.

[0037] 2, when there are a plurality of speakers Sp, the measurement control unit 112 may specify the speaker Sp that emits the sound as the output destination of the sound source signal. The speaker Sp that is the output destination may be specified by an operation signal input from the operation unit 160, or may be selected according to a predetermined order.

[0038] The seat Pf may have a horizontal plane parallel to the floor surface and a rotatable seat plate. The subject Sb seated on the seat plate may rotate to change the direction in the horizontal plane relative to the speaker Sp. The subject Sb may operate the operation unit 160 to instruct the start of measurement every time the direction is changed by a predetermined angle (for example, 3 to 60°). This makes it easy to measure HRIRs with different directions in the horizontal plane. The measurement control unit 112 may obtain direction information indicating the direction of the seat Pf, output the obtained direction information to the calculation unit 116, and store the information in the storage unit 130 in association with the HRIR measured by the calculation unit 116.

[0039] The recording unit 114 stores (records) the left and right received sound signals input from the microphones MicL and MicR via the input / output unit 140, respectively. The microphones MicL and MicR receive the sounds that are directly or indirectly transmitted from the speaker Sp. Thus, the left and right received signals are input from the microphones MicL and MicR to the recording unit 114. When an impulse signal is used as the sound source signal, the left and right received signals indicate the HRIR from the speaker Sp to the microphones MicL and MicR, respectively. Thus, the recording unit 114 may start recording the left and right received signals when an operation signal indicating the start of measurement is input from the operation unit 160, and may stop recording after a predetermined recording period (for example, 0.2 to 2 seconds) has elapsed since the start of recording. The recording period may be at least as long as the observation period.

[0040] The calculation unit 116 receives an operation signal from the operation unit 160 indicating the start of measurement, and after the above-mentioned recording period has elapsed, reads the newly recorded received sound signal from the recording unit 114. The calculation unit 116 obtains the HRIR from the read received sound signal. As described above, the reflected sound included in the observation period is significantly attenuated by the sound absorbing material As. However, components of the reflected sound arriving after the above-mentioned observation period may remain in the received sound signal.

[0041] Therefore, the calculation unit 116 may gradually attenuate (fade out) the signal values ​​of the impulse response obtained from the received signal, which are in a period after the observation period. For example, the calculation unit 116 attenuates the signal value of the received signal from the end point of the observation period over a predetermined attenuation time (for example, 10 to 100 ms) depending on the passage of time until it reaches zero, and discards the signal value at the time thereafter. Thus, the influence of the reflected sound (hereinafter, late reflected sound) that appears in the received signal and arrives after the observation period is removed, and the calculation unit 116 can obtain a significant portion as the HRIR. The components of the late reflected sound include, for example, reflected sound from the ceiling, the back of the subject Sb, that is, reflected sound from the opposite direction from the seat Pf to the speaker Sp, and the like. The calculation unit 116 stores the HRIR signal indicating the obtained HRIR in the storage unit 130.

[0042] Next, an example of the HRIR measurement process according to this embodiment will be described. 5 is a flowchart showing an example of the HRIR measurement process according to this embodiment. However, before starting the following process, the reflecting objects (i.e., floor, wall, seat Pf, etc.) at the locations where the reflected sounds arrive in response to the impulse arriving at the microphones MicL and MicR attached to the left and right ears of the subject Sb, respectively, within at least an observation period from the generation of the impulse from the speaker Sp are covered with a sound absorbing material As.

[0043] (Step S102) The measurement control unit 112 outputs an impulse signal to the speaker Sp. The speaker Sp emits an impulse as a measurement sound based on the impulse signal. (Step S104) The microphones MicL and MicR receive the sounds that have arrived at them, and obtain left and right received signals that indicate the received sounds. The microphones MicL and MicR output the left and right received signals to the recording unit 114, respectively. (Step S106) The recording unit 114 records the left and right received signals each including an HRIR. (Step S108) The calculation unit 116 removes late reflections by attenuating and discarding signal values ​​for a period after the observation period from each of the left and right received sound signals newly recorded in the recording unit 114, thereby obtaining an HRIR.

[0044] Note that the received signal may contain noise arriving at the microphone Mic from the surroundings. Therefore, after being instructed to start measurement, the measurement control unit 112 may repeat the HRIR measurement process from step S102 to step S108 a predetermined number of times or more. A predetermined number of times, 2 or more, is preset in the measurement control unit 112 as the number of repetitions. Thus, the calculation unit 116 outputs an HRIR acquisition end notification to the measurement control unit 112 after each HRIR measurement process ends. The measurement control unit 112 counts the number of measurements by adding (incrementing) 1 each time an HRIR acquisition end notification is input. However, the initial value of the number of measurements is set to 0. Then, the measurement control unit 112 determines whether the number of measurements has reached the number of repetitions, and when it is determined that the number has been reached, it outputs a measurement end notification to the calculation unit 116. When it is determined that the number has not been reached, the measurement control unit 112 starts the next HRIR measurement process.

[0045] When the measurement end notification is input from the measurement control unit 112, the calculation unit 116 generates a new HRIR signal including, as a signal value, a sum obtained by adding up the signal values ​​for each time included in the HRIR signal for the acquired number of repetitions between repetitions of the HRIR measurement process for each time (synchronous addition). The calculation unit 116 may generate an HRIR signal including, as a new signal value instead of the original signal value, a quotient obtained by dividing the signal value for each time by the number of repetitions (normalization). The calculation unit 116 stores the generated HRIR in the storage unit 130. The synchronous addition reduces the noise components that randomly overlap between repetitions of the HRIR measurement process relatively to the signal components of the HRIR. Therefore, the signal-to-noise ratio (SNR) is improved, and the influence of noise can be reduced.

[0046] <Second embodiment> Next, the second embodiment will be described, focusing mainly on the differences from the first embodiment. Unless otherwise specified, the same configurations and processes as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be used.

[0047] FIG. 6 is a side view showing an example of the configuration of the sound processing system 1 according to this embodiment. The sound processing system 1 according to the present embodiment includes a speaker Sp as a sound source, microphones Mic L and Mic R (however, Mic R is not shown in FIG. 6) as sound receiving units, a sound absorbing material As, and a sound processing device 10, as well as an image acquisition unit Cm as an example of a measurement environment information acquisition unit. The image acquisition unit Cm captures an image of the measurement environment of the HRIR as an example of measurement environment information. The image acquisition unit Cm is, for example, a digital still camera capable of capturing still images. The measurement environment may include an area (hereinafter, evaluation area) in which reflectors that bring about reflected sounds that arrive at the microphones Mic L and Mic R within an observation period, among the reflected sounds of the sound emitted from the speaker Sp, are distributed or arranged. In other words, the image acquisition unit Cm may be positioned and oriented so as to include the evaluation area in the field of view in the space Rm, and the field of view angle may be set.

[0048] The sound processing device 10 judges the measurement environment, particularly the arrangement of the sound absorbing material in the evaluation area, based on the image captured by the image acquisition unit Cm. The sound processing device 10 estimates the reflection characteristics from the arrangement by referring to the reflection characteristic related data indicating the relationship between the arrangement and the reflection characteristics. Then, the sound processing device 10 judges whether to measure the HRIR based on whether the estimated reflection characteristics are greater than a predetermined reflection intensity. This identifies whether an HRIR that can be used to realize a stereoscopic effect can be measured.

[0049] Next, an example of the functional configuration of the sound processing device 10 according to this embodiment will be described. Fig. 7 is a schematic block diagram showing an example of the functional configuration of the control unit 110 provided in the sound processing system 1 according to this embodiment. The control unit 110 includes a measurement control unit 112 , a sound recording unit 114 , a calculation unit 116 , a reflectance storage unit 124 , and a measurement environment determination unit 126 .

[0050] The reflection characteristic related data is stored in advance in the reflectance storage unit 124. The reflection characteristic related data is data including information indicating, for each material, a material and a corresponding reflectance of the material with respect to a sound wave arriving at the material. The reflectance is the ratio of the intensity of the reflected wave with respect to the incoming sound wave to the intensity of the incoming sound wave. The reflection characteristic related data needs to include information on at least one type of material that can be a sound absorbing material. The reflection characteristic related data may or may not include information on a material that is not a sound absorbing material. A sound absorbing material corresponds to a material whose reflectance is sufficiently lower than a predetermined reflectance (for example, -30 to -50 dB).

[0051] The measurement environment determination unit 126 receives image data representing an image captured by the image acquisition unit Cm. The measurement environment determination unit 126 executes a predetermined image recognition process on the input image data to determine the arrangement state of the sound absorbing material in the measurement environment. More specifically, the measurement environment determination unit 126 identifies the material of an object arranged in each block obtained by subdividing the image shown in the image data into blocks of a predetermined size by the image recognition process. For example, image recognition data indicating the relationship between image feature amount and material is set in advance in the measurement environment determination unit 126. In the image recognition process, for example, the measurement environment determination unit 126 calculates image feature amount for each block based on the pixel value of each pixel arranged in the block, and can specify the material of the subject corresponding to the calculated image feature amount by referring to the image recognition data. Here, a collection of blocks in which the specified material is a material corresponding to the sound absorbing material corresponds to a sound absorbing material area, and the arrangement state of the sound absorbing material is indicated by the distribution of the sound absorbing material area.

[0052] The measurement environment determination unit 126 determines whether or not to measure the HRIR based on the arrangement information of the sound absorbing material. More specifically, the measurement environment determination unit 126 refers to the reflection characteristic related data to determine the reflectance of the material specified for each block included in the evaluation area, and calculates the average value of the specified reflectance among the blocks in the evaluation area as the average reflectance. The measurement environment determination unit 126 determines whether or not to measure the HRIR based on whether or not the calculated average reflectance is equal to or lower than a predetermined upper limit value of the reflectance (for example, -25 to -40 dB). The measurement environment determination unit 126 has evaluation area information indicating the evaluation area in the captured image set in advance. As described above, the evaluation area is determined based on the positional relationship between the speaker Sp and the seat Pf, the distribution of reflectors (walls, sides, seats Pf, etc.) in the space Rm, the position, orientation, and size of the viewing angle of the image acquisition unit Cm.

[0053] The measurement environment determination unit 126 generates display data including determination information corresponding to a determination result indicating, for example, whether or not to measure the HRIR, and outputs the generated display data to the display unit 150. The display unit 150 notifies the user as a determination result whether or not the measurement environment is an appropriate environment, that is, whether or not the environment is capable of measuring an HRIR that allows the listener to fully perceive the stereoscopic effect. Here, the user may be an operator involved in measuring the HRIR, or the subject Sb himself / herself.

[0054] The measurement environment determination unit 126 may output determination information indicating the determination result to the measurement control unit 112. When the determination information input from the measurement environment determination unit 126 indicates that the HRIR is to be measured, the measurement control unit 112 executes the above-mentioned HRIR measurement process, and when the determination information indicates that the measurement environment does not measure the HRIR, the measurement control unit 112 may not execute the above-mentioned HRIR measurement process. As a result, when the measurement environment is inappropriate, the measurement of the HRIR is restricted, and when the measurement environment is appropriate, the HRIR is measured. This ensures the quality of the measured HRIR.

[0055] Next, an example of the measurement environment determination process according to this embodiment will be described. Fig. 8 is a flowchart showing an example of the measurement environment determination process according to this embodiment. However, before starting the execution of the process shown in Fig. 8, reflection characteristic related data is saved in advance. The reflection characteristic related data includes information indicating the reflectance of each material.

[0056] (Step S202) The image acquisition unit Cm captures an image of the measurement environment, and outputs image data indicating the captured image to the sound processing device . (Step S204) The measurement environment determination unit 126 performs image recognition processing on the image data input from the image acquisition unit Cm to identify the material of the subject in the measurement environment represented in the field of view for each block into which the image is divided. The measurement environment determination unit 126 determines the reflectance of the material identified for each block by referring to the reflection characteristic related data.

[0057] (Step S206) The measurement environment determination unit 126 determines an average reflectance as an example of a representative value of the reflectance in the evaluation area based on the reflectance determined for each block in the predetermined evaluation area. The measurement environment determination unit 126 determines whether the determined average reflectance is equal to or less than a predetermined upper limit of the reflectance. When the measurement environment determination unit 126 determines that the average reflectance is equal to or less than the upper limit, it determines that the measurement environment measures the HRIR, and when the measurement environment determination unit 126 determines that the average reflectance is greater than the upper limit, it determines that the measurement environment does not measure the HRIR. The measurement environment determining section 126 may output determination information indicating the determination result to the display section 150 or to the measurement control section 112.

[0058] In addition, when judging whether or not the measurement environment is an environment for measuring HRIR, the measurement environment judgment unit 126 determines the reflectance in the evaluation area as an example of an index of the arrangement state of the sound absorbing material, but this is not limited to this. For example, the measurement environment judgment unit 126 may determine the coverage rate of a sound absorbing material of a predetermined material in the evaluation area as an example of an index of the arrangement state of the sound absorbing material. Here, the measurement environment judgment unit 126 may set in advance sound absorbing material information indicating whether or not the member is a sound absorbing material for each member. In this case, it is not necessary to store reflection characteristic related data in advance in the reflectance storage unit 124.

[0059] The measurement environment determination unit 126 performs image recognition processing on the image data as described above, and identifies the material of the subject for each block. The measurement environment determination unit 126 can specify whether or not the identified material is a sound absorbing material by referring to the sound absorbing material information set in the measurement environment determination unit 126. The measurement environment determination unit 126 determines a set of blocks in which the identified material is determined to be a sound absorbing material as a sound absorbing area. The measurement environment determination unit 126 can calculate the ratio of the area of ​​the evaluation area to the area of ​​the sound absorbing area included in the evaluation area as the sound absorbing area ratio. Then, the measurement environment determination unit 126 determines whether or not the sound absorbing area ratio is equal to or greater than a lower limit of a predetermined ratio (for example, 0.8 to 0.98) as an index value indicating the arrangement state of the sound absorbing material. When the measurement environment determination unit 126 determines that the sound absorbing area ratio is equal to or greater than the predetermined lower limit, it determines that the measurement environment is an environment in which the HRIR is measured, and when the measurement environment determination unit 126 determines that the sound absorbing area ratio is equal to or less than the predetermined lower limit, it determines that the measurement environment is an environment in which the HRIR is not measured.

[0060] Next, an example of the configuration of the above embodiment will be described. FIG. 9 is an explanatory diagram showing an example of the configuration. The sound processing system 1 according to the above embodiment includes a sound source (e.g., a speaker Sp), a sound receiving unit (e.g., a microphone Mic), a calculation unit 116, and a sound absorbing material As. The sound source emits a sound based on a predetermined sound source signal. The sound receiving unit acquires a received sound signal of the arriving sound. The calculation unit 116 acquires an impulse response from the sound source to the sound receiving unit from the acquired received sound signal. The sound absorbing material covers an object on the path of a reflected sound that arrives within at least a predetermined observation period from the generation of the sound. According to this configuration, the installation of the sound-absorbing material reduces the intensity of the reflected sound component that arrives at least within the observation period in the impulse response compared to when the sound-absorbing material is not installed. In addition, it is sufficient for the sound-absorbing material to cover the objects related to the propagation path of the sound wave from the sound source to the sound receiving unit, at least the propagation path of the reflected sound that arrives within the observation period. Therefore, even in an ordinary space that does not have special equipment such as an anechoic chamber, it is possible to easily obtain the impulse response to be used as the HRIR. Here, the burden of installing the sound-absorbing material is relatively small, and the space can be used effectively.

[0061] Furthermore, the calculation section 116 may attenuate the signal value of the acquired impulse response after the observation period has elapsed. According to this configuration, the part of the acquired impulse response within the observation period is extracted, and the part after the observation period is removed. Therefore, the part important for the perception of the three-dimensional effect of the sound is maintained as the HRIR within the observation period, and the influence of the late reflections arriving after the observation period can be avoided.

[0062] The sound processing system 1 may further include a measurement control unit 112 that repeats the generation of sound from a sound source, the acquisition of a received sound signal, and the acquisition of an impulse response multiple times. The calculation unit 116 may synchronously add up the multiple acquired impulse responses. According to this configuration, the noise components added are relatively reduced compared to the components of the impulse response added by synchronous addition, so that a more accurate impulse response can be obtained.

[0063] The sound processing system 1 may further include a measurement environment information acquisition unit (for example, an image acquisition unit Cm) and a measurement environment determination unit 126. The measurement environment information acquisition unit acquires information on the measurement environment of the impulse response. The measurement environment determination unit refers to preset reflection characteristic related data, determines the arrangement state of the sound absorbing material based on the acquired information, and determines whether the measurement environment is suitable for measuring the impulse response based on the determined arrangement state of the sound absorbing material. According to this configuration, it is judged whether the measurement environment is appropriate for obtaining an impulse response that allows the listener to fully perceive the three-dimensional effect based on the arrangement of the sound absorbing material, and whether or not it is necessary to measure the impulse response is judged according to the judgment result. Therefore, the acquisition of an inappropriate impulse response is restricted, and the acquisition of an appropriate impulse response is encouraged.

[0064] The sound processing system 1 may also include a reflectance storage unit 124. The reflectance storage unit 124 stores reflection characteristic related data including information associating materials with information on the reflectance of sound. The measurement environment determination unit 126 identifies the material of the subject identified from the image as the acquired information, refers to the reflection characteristic related data to identify the reflectance corresponding to the identified material, and determines whether or not to measure an impulse response based on the identified reflectance. According to this configuration, the material of the subject shown in the acquired image is identified, and whether or not to measure the impulse response is determined based on the reflectance determined from the identified material. Therefore, the reflectance in the measurement environment of the impulse response is estimated without performing actual measurement, and therefore it is possible to determine whether or not to measure the impulse response without changing the acoustic environment due to the actual measurement.

[0065] Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as the scope and spirit of the invention.

[0066] This application claims priority based on Japanese Patent Application No. 2020-156368, filed on September 17, 2020, the disclosure of which is incorporated herein in its entirety. [Industrial Applicability]

[0067] The present invention may be applied to an audio processing system, an audio processing method, and a recording medium. [Explanation of symbols]

[0068] 1. Sound processing system 10...Sound processing device 110...Control unit 112... Measurement control unit 114…Recording section 116...Arithmetic section 124...Reflectance storage section 126...Measurement environment determination section 130...Storage section 140…Input / output section 150...Display section 160...Operation unit As…sound absorbing material Cm: Image acquisition section Mic (Mic L, Mic R)...Microphone Sp…Speaker

Claims

1. a sound source that emits a sound based on a predetermined sound source signal; A sound receiving unit for acquiring a received signal of an incoming sound; A calculation unit that obtains an impulse response from the sound source to the sound receiving unit from the received sound signal; a sound absorbing material that covers an object on a path of a reflected sound that arrives at the sound receiving unit within at least a predetermined observation period from the generation of the sound; a measurement environment information acquisition unit that performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of a subject corresponding to a sound absorbing material in the measurement environment and a distribution of the material; A reflectance storage unit that stores data related to reflection characteristics including information on the sound reflectance for each material; By referring to the reflection characteristic related data, a reflectance corresponding to the identified material is identified; a measurement environment determination unit that determines whether or not to measure the impulse response based on an average reflectance in the evaluation area calculated from the distribution of the material. Sound processing system.

2. The calculation unit is Attenuating the signal value after the observation period of the impulse response has elapsed. The sound processing system of claim 1 .

3. a measurement control unit that repeats the generation of sound from the sound source, the acquisition of the received sound signal, and the acquisition of the impulse response a plurality of times; The calculation unit is A plurality of the impulse responses are synchronously added. The sound processing system according to claim 1 or 2.

4. A sound processing method for a sound processing system including a sound absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving section within at least a predetermined observation period from the generation of the sound from a sound source, the method comprising: a first step in which the sound source emits a sound based on a predetermined sound source signal; A second step in which the sound receiving unit acquires a received signal of the arriving sound; a third step of acquiring an impulse response from the sound source to the sound receiving unit from the received sound signal by the calculation processing unit; a fourth step in which the measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of the object corresponding to a sound absorbing material in the measurement environment and a distribution of the material; The measurement environment determination unit refers to the reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, and identifies the reflectance corresponding to the identified material; and a fifth step of determining whether or not to measure the impulse response based on an average reflectance in the evaluation area calculated from the distribution of the material. Acoustic processing methods.

5. a sound absorbing material that covers an object on a path of a reflected sound that arrives at the sound receiving unit within at least a predetermined observation period from the generation of the sound from the sound source, A first step in which the sound source emits a sound based on a predetermined sound source signal; A second step in which the sound receiving unit acquires a received signal of the arriving sound; a third step of acquiring an impulse response from the sound source to the sound receiving unit from the received sound signal by the calculation processing unit; a fourth step in which the measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of the object corresponding to a sound absorbing material in the measurement environment and a distribution of the material; The measurement environment determination unit refers to the reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, and identifies the reflectance corresponding to the identified material; a fifth step of determining whether or not to measure the impulse response based on an average reflectance within an evaluation area calculated from the distribution of the material; A program that executes the following.

Images