Recording device, sound reproduction device, recording method, and sound reproduction method

Abstract

To provide a recording device or the like which allows a user to listen to a highly realistic sound in which timing of sound output from a plurality of speakers corresponds to timing of actual sound generation.SOLUTION: A recording device includes a recording signal processing unit that adds position information of one or more microphones or direction information of the one or more microphones input by a microphone information input unit to one or more sound signals acquired by one or more microphones, and the recording signal processing associates the position information of the one or more microphones or the direction information of the one or more microphones with the one or more sound signal in a one-to-one relationship, and causes a recording storage unit to store the information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a sound recording device, a sound reproducing device, a sound recording method, and a sound reproducing method. [Background technology]

[0002] As disclosed in Patent Documents 1 to 3, recording devices and sound playback devices that capture sound with a microphone and play back the sound with a speaker are being developed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-211047 A [Patent Document 2] JP 2006-287606 A [Patent Document 3] JP 2009-044261 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the techniques disclosed in Patent Documents 1 to 3, even if the direction from the microphone to the assumed listening position is the same as the direction from the actual speaker to the actual listener, the distance from the microphone to the assumed listening position may differ from the distance from the speaker to the actual listener. In this case, a discrepancy occurs between the virtual listening timing at which the sound actually generated is estimated to be heard at the assumed listening position and the actual listening timing at which the listener hears the sound output from the speaker. Therefore, even if the sound captured by the microphone is reproduced by the speaker, the listener cannot hear a highly realistic sound.

[0005] The present disclosure has been made in consideration of the above problems. An object of the present disclosure is to provide a recording device, a sound reproducing device, a recording method, and a sound reproducing method that enable a user to hear a highly realistic sound even when a sound captured by a microphone is reproduced by a speaker. [Means for solving the problem]

[0006] A recording device of one embodiment of the present disclosure includes a recording signal processing unit that adds position information of the one or more microphones or directional information of the one or more microphones input by a microphone information input unit to one or more sound signals acquired by the one or more microphones, and the recording signal processing unit associates the position information of the one or more microphones or the directional information of the one or more microphones with the one or more sound signals in a one-to-one relationship and stores them in a recording memory unit.

[0007] A sound playback device of one embodiment of the present disclosure includes a sound playback control unit that determines one or more delay times for one or more sound signals transmitted from a sound playback memory unit that stores one or more sound signals based on position information of one or more speakers input by a speaker position input unit, a delay time imparting unit that imparts the one or more delay times determined by the sound playback control unit to each of the one or more sound signals, and a speaker driving unit that delays one or more sounds corresponding to the one or more sound signals by the one or more delay times and generates them in one or more speakers.

[0008] A recording method according to one embodiment of the present disclosure includes the steps of: adding position information of the one or more microphones or directional information of the one or more microphones input by a microphone information input unit to one or more sound signals acquired by the one or more microphones; and storing the position information of the one or more microphones or the directional information of the one or more microphones in a one-to-one association with the one or more sound signals.

[0009] A sound reproduction method according to one embodiment of the present disclosure includes the steps of determining one or more delay times for one or more sound signals based on position information of one or more speakers input by a speaker position input unit, imparting the one or more delay times to the one or more sound signals, respectively, and generating one or more sounds corresponding to the one or more sound signals from one or more speakers, each delayed by the one or more delay times. [Brief description of the drawings]

[0010] [Figure 1] 1 is a functional block diagram of a recording device according to a first embodiment. [Diagram 2] 2 is a schematic diagram showing the relationship between the positions of one or more microphones in the sound recording device of the first embodiment and the reference position where a listener is located. FIG. [Diagram 3] 2 is a diagram showing an example of a data string in which one or more pieces of position information are associated with one or more sound signals in a one-to-one relationship in the sound recording device and sound reproducing device of the first embodiment. FIG. [Figure 4] FIG. 11 is a diagram showing another example of a data string in which one or more pieces of position information are associated with one or more sound signals in a one-to-one relationship in the sound recording device and the sound reproducing device of the first embodiment. [Diagram 5] FIG. 1 is a first schematic diagram for explaining XYZ coordinates for specifying position information of one or more microphones. [Figure 6] FIG. 2 is a second schematic diagram for explaining XYZ coordinates for specifying position information of one or more microphones. [Figure 7] FIG. 11 is a third schematic diagram for explaining XYZ coordinates for identifying position information of one or more microphones. [Figure 8] FIG. 11 is a schematic diagram for explaining polar coordinates for identifying position information of one or more microphones. [Figure 9] 1 is a functional block diagram of a sound reproduction device according to a first embodiment. [Figure 10] 1 is a schematic diagram showing an example of the positional relationship between one or more speakers in the sound reproduction device of the first embodiment and a reference position where a listener is located. [Figure 11]FIG. 11 is a diagram showing an example of the overall configuration of one or more microphones of a sound recording device according to a second embodiment. [Figure 12] 13 is a diagram for explaining that information capable of identifying the overall configuration of one or more microphones is associated with directional information of the one or more microphones in the second embodiment. FIG. [Figure 13] 11 is a schematic diagram showing an example of the positional relationship between one or more speakers in a sound reproduction device according to a second embodiment and a reference position where a listener is present. FIG. [Figure 14] FIG. 11 is a diagram showing an example of a data string in which one or more pieces of position information are associated with one or more sound signals in a one-to-one relationship in the sound recording device and sound reproducing device of the third embodiment. [Figure 15] FIG. 13 is a diagram showing an example of a data string in which one or more pieces of position information are associated with one or more sound signals in a one-to-one relationship in the sound recording device and sound reproducing device of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following describes a recording device and a sound reproducing device according to an embodiment of the present disclosure with reference to the drawings. Note that in the drawings, the same or equivalent elements are designated by the same reference numerals, and redundant description will be omitted.

[0012] (Embodiment 1) The sound recorder SREC and the sound reproducing device SREG according to the first embodiment will be described with reference to FIGS.

[0013] The recording device SREC of this embodiment is assumed to be used for recording sounds that occur in outdoor natural environments, such as the sound of wind, the sound of a flowing river, the sound of thunder, or the cries of animals, etc. However, it may also be used for recording music being played indoors.

[0014] Fig. 1 is a functional block diagram of a recording device SREC that records one or more sound signals, more specifically, one or more audio signals. As shown in Fig. 1, the recording device SREC includes one or more microphones Mic(K) (hereinafter, K=1, 2, 3,...,N: N is a natural number). The recording device SREC includes one or more analog-digital converters A / D(K) (hereinafter, K=1, 2, 3,...,N: N is a natural number).

[0015] The sound recording device SREC includes a sound recording signal processing unit SP1. As described above, if the sound recording device SREC includes the sound recording signal processing unit SP1, the above-mentioned microphone Mic(K) and analog-digital conversion unit A / D(K), as well as the sound recording storage unit R1 and the microphone information input unit MINP described below are not essential components. The microphone Mic(K) and the analog-digital conversion unit A / D(K) may be replaceable with those having different performance and structure depending on the purpose of use of the sound recording device SREC. The sound recording storage unit R1 and the microphone information input unit MINP described below may be included in a device other than the sound recording device SREC.

[0016] Each of the one or more microphones Mic(K) receives sound, i.e., air vibrations, with a diaphragm or the like and converts the air vibrations received by the diaphragm or the like into an electrical signal. Each of the one or more analog-to-digital converters A / D(K) converts the analog electrical signal generated from the air vibrations into a digital signal.

[0017] Each of the one or more analog-digital conversion units A / D(K) transmits the digital signal to the recording signal processing unit SP1. Hereinafter, the digital signal is referred to as Audio Stream(K) (K=1, 2, 3,..., N: N is a natural number) (see FIG. 9). However, the digital signal is sometimes simply referred to as sound signal SS(K) (K=1, 2, 3,..., N: N is a natural number).

[0018] The microphone information input unit MINP is an operation unit for a user of the sound recording device SREC to input position information (X, Y, Z) of each of one or more microphones Mic(K) on predetermined coordinates. However, as described in the second embodiment, the microphone information input unit MINP also functions as an operation unit for a user of the sound recording device SREC to input direction information (α, β) of each of one or more predetermined microphones Mic(K). The microphone information input unit MINP is not an essential component of the sound recording device SREC, and may be any input device such as an exchangeable touch panel, keyboard, or mouse.

[0019] The recording control unit CT1 controls the recording signal processing unit SP1. In this embodiment, the recording control unit CT1 transmits the position information (X, Y, Z) of one or more microphones Mic(K) inputted at the microphone information input unit MINP to the recording signal processing unit SP1.

[0020] The recording signal processing unit SP1 adds position information (X, Y, Z) of one or more microphones Mic(K) input by the microphone information input unit MINP to one or more sound signals acquired by one or more microphones Mic(K). The one or more sound signals are acquired by one or more microphones Mic(K). The recording memory unit R1 associates the position information (X, Y, Z) of one or more microphones Mic(K) with one or more sound signals SS(K) in a one-to-one relationship and stores them. In other words, the recording signal processing unit SP1 associates the position information (X, Y, Z) of one or more microphones Mic(K) with one or more sound signals SS(K) in a one-to-one relationship and stores them in the recording memory unit R1.

[0021] In this specification, the one-to-one relationship may be one between the position information (X, Y, Z) of the microphone Mic(K) and the sound signal SS(K). In this case, even if other signals or information associated with the position information (X, Y, Z) of the microphone Mic(K) and the sound signal SS(K) are stored in the recording storage unit R1, the one-to-one relationship is established.

[0022] In the present embodiment, each of the position information (X, Y, Z) of one or more microphones Mic(K) is coordinate information indicating orthogonal coordinates. However, each of the position information (X, Y, Z) of one or more microphones Mic(K) may be coordinate information indicating polar coordinates.

[0023] FIG. 2 is a schematic diagram showing the relationship between the position information (X, Y, Z) of one or more microphones Mic(K) in the recording device SREC of Embodiment 1 and the reference position P where the listener exists. Using FIG. 2, for example, a method for determining the position information (X, Y, Z) of the microphone Mic(K) when recording a sound generated outdoors using, for example, 24 microphones Mic(K) <K is a natural number from 1 to 24> will be described.

[0024] As shown in FIG. 2, it is assumed that the recording device SREC of the present embodiment includes 24 microphones Mic(K) composed of, for example, 8 upper layers, 8 middle layers, and 8 lower layers. The reference position P is set at the center position of the 24 microphones Mic(K). The reference position P indicates a virtual position where a listener is assumed to listen when playing back one or more sound signals SS(K) recorded using the recording device SREC using a sound playback device SREG described below. The relative positional relationship from the reference position P to the position of the microphone Mic(K) is the position information (X, Y, Z).

[0025] The position information (X, Y, Z) of the microphone Mic(K) is input using the microphone information input unit MINP as follows. For example, along each of the three axes in the X-axis direction, Y-axis direction, and Z-axis direction, first, the distance from the reference position P where the listener is assumed to listen to each of the one or more microphones Mic(K) is measured using a measuring instrument or the like. As a result, the distances along the three axes in the X-axis direction, Y-axis direction, and Z-axis direction are measured respectively. The distances along these three axes are transmitted as the position information (X, Y, Z) of the microphone Mic(K) to the recording signal processing unit SP1 via the recording control unit CT1 by the operation of the microphone information input unit MINP by the user.

[0026] Furthermore, when each of the one or more microphones Mic(K) is equipped with a GPS (Global Positioning System) and a display (not shown), the following may be done. For example, a transmitter that transmits its position information (X, Y, Z) is installed at a reference position P. As a result, the one or more microphones Mic(K) receive information on the reference position P transmitted by the transmitter (not shown), and the coordinates in a three-axis coordinate system of the one or more microphones Mic(K) relative to the reference position P may be displayed on a display (not shown) as position information (X, Y, Z).

[0027] Furthermore, the difference between the current position of Mic(K) and the desired planned installation position of Mic(K) may be displayed on the display (not shown) of Mic(K). In this case, guide information for encouraging the user to move Mic(K) to the desired planned installation position of Mic(K), for example, the direction in which Mic(K) should be moved, may be displayed on the display of Mic(K).

[0028] Thereafter, the user operates the microphone information input unit MINP, whereby the position information (X, Y, Z) of one or more microphones Mic(K) is input to the recording signal processing unit SP1 via the recording control unit CT1.

[0029] 3 is a diagram showing an example in which one or more pieces of position information (X, Y, Z) are associated with one or more sound signals SS(K) in a one-to-one relationship. The one-to-one relationship here means that, as long as there is a correspondence between one piece of position information (X, Y, Z) and one sound signal SS(K), one or more other pieces of information or one or more other signals may be added to the combination of one piece of position information (X, Y, Z) and one sound signal SS(K).

[0030] In this embodiment, the recording storage unit R1 stores a sound signal SS(K) to which position information (X, Y, Z) of the microphone Mic(K) is added in the format shown in Fig. 3. After outputting each of the position information (X, Y, Z) of one or more microphones Mic(K), the recording storage unit R1 outputs each of the one or more sound signals SS(K) in an order associated with each other in a one-to-one relationship. In this manner, the recording storage unit R1 stores the position information (X, Y, Z) of the microphone Mic(K) and the sound signal SS(K).

[0031] Specifically, first, position information (X,Y,Z) of microphone Mic(1), position information (X,Y,Z) of microphone Mic(2), ... position information (X,Y,Z) of microphone Mic(K), ..., position information (X,Y,Z) of microphone Mic(24) are output in this order. Then, sound signal (1), sound signal (2), ..., sound signal SS(K), ..., sound signal (24) are output in this order. In other words, after position information (X,Y,Z) of microphone Mic(K) is output in the order of K digits (natural numbers), sound signal SS(K) is output in the order of K digits (natural numbers).

[0032] It should be noted that the sound playback signal processor SP2, which will be described later, needs to identify whether the position information (X, Y, Z) or the sound signal has been transmitted. For this reason, in this embodiment, the sound recording signal processor SP1 specifies the transmission format in the first data of the content to be output. For example, the first data of the content specifies that the next data is a data string to which the position information (X, Y, Z) of the microphone Mic (K) has been added. The first data of the content also specifies that the specification is followed by the first Y pieces of X-byte data of position information (X, Y, Z), and that this is followed by Y pieces of Z-byte audio stream.

[0033] However, instead of the above-mentioned designation, the recording signal processing unit SP1 may transmit position information output start information indicating that output of the position information (X, Y, Z) of the microphone Mic(K) is to be started when output of the position information (X, Y, Z) of the microphone Mic(K) is to be started. This position information output start information may be included in the position information (X, Y, Z) of each microphone Mic(K). Also, when output of the sound signal SS(K) is to be started, sound signal output start information indicating that output of the sound signal SS(K) is to be started may be transmitted. This sound signal output start information may be included in the sound signal SS(K) of each microphone Mic(K). By outputting the above-mentioned position information output start signal and sound signal output start information, the sound reproduction signal processing unit SP2 on the receiving side, which will be described later, can easily identify what kind of data has been transmitted.

[0034] 4 is a diagram showing another example in which one or more pieces of position information (X, Y, Z) are associated with one or more sound signals SS(K) in a one-to-one relationship. The one-to-one relationship here means that one or more pieces of other information or one or more other signals may be added to one piece of position information (X, Y, Z) and one sound signal SS(K) as long as there is a corresponding relationship between the piece of position information (X, Y, Z) and one sound signal SS(K).

[0035] The recording storage unit R1 may store the sound signal SS(K) to which the position information (X, Y, Z) of the microphone Mic(K) is added in the format shown in Fig. 4. More specifically, the recording storage unit R1 can alternately output any one of the position information (X, Y, Z) of one or more microphones Mic(K) and any one of the one or more sound signals SS(K) in an order associated in a one-to-one relationship.

[0036] Specifically, position information (X, Y, Z) of microphone Mic(1), sound signal (1), position information (X, Y, Z) of microphone Mic(2), sound signal (2), ..., position information (X, Y, Z) of microphone Mic(24) and sound signal (24) are output in this order. That is, position information (X, Y, Z) of microphone Mic(K) and sound signal SS(K) are output alternately in the order of the value of K. In other words, a combination of position information (X, Y, Z) of microphone Mic(K) and sound signal SS(K) following position information (X, Y, Z) of microphone Mic(K) is output in the order of the digits of K (natural number).

[0037] 5 to 7 are diagrams for explaining XYZ coordinates for identifying position information of one or more microphones. As shown in FIG. 5 to FIG. 7, in this embodiment, the position information (X, Y, Z) of one or more microphones Mic(K) is expressed using a three-axis coordinate system consisting of the X-axis, the Y-axis, and the Z-axis. Specifically, the left-right position of the position information (X, Y, Z) of one or more microphones Mic(K) is expressed in the words Left / Center / Right (see FIG. 5). The up-down position of the position information (X, Y, Z) of one or more microphones Mic(K) is expressed in the words Top / Middle / Bottom (see FIG. 6). The depth position of the position information (X, Y, Z) of one or more microphones Mic(K) is expressed in the words Fornt / Side / Back (see FIG. 7).

[0038] Fig. 8 is a schematic diagram for explaining polar coordinates for identifying position information (R, α, β) of one or more microphones Mic(K). As shown in Fig. 8, the position information (X, Y, Z) of the microphone Mic(K) may be expressed in the form of polar coordinates using a distance R from a reference position P on a reference plane RP, a rotation angle α from a reference direction RD, and an elevation angle β from the reference direction RD.

[0039] FIG. 9 is a functional block diagram of the sound reproducing device SREG according to the present embodiment.

[0040] The sound playback device SREG includes a sound playback memory unit R2 and a sound playback signal processing unit SP2. The sound playback device SREG includes a delay time imparting unit DL(K) <K = 1, 2, 3, ···, N: N is a natural number>, a speaker driving unit SDR(K) <hereinafter, K = 1, 2, 3, ··· N: N is a natural number>, and a speaker SP(K) <K = 1, 2, 3, ···, N: N is a natural number>.

[0041] The sound playback memory unit R2 stores one or more sound signals SS(K) that are the same as the one or more sound signals SS(K) stored in the recording memory unit R1 of the recording device SREC. That is, the position information (X, Y, Z) of one or more microphones Mic(K) and one or more sound signals SS(K) stored in the recording memory unit R1 are stored in the sound playback memory unit R2 via a telecommunication information network or a storage medium. Therefore, the sound playback memory unit R2 stores the position information (X, Y, Z) of one or more microphones Mic(K) and one or more sound signals SS(K) in a one-to-one relationship. Note that the sound playback device SREG may not have the sound playback memory unit R2. For example, a server installed on a telecommunication network such as the Internet may include the sound playback memory unit R2. In this case, one or more sound signals may be distributed to the sound playback devices SREG in each household by streaming from the server. That is, the sound playback device SREG may be anything as long as it includes a sound playback control unit CT2, a delay time imparting unit DL(K), and a speaker driving unit SDR(K).

[0042] The one or more sound signals SS(K) are so-called multi-channel audio data. Each of the one or more sound signals SS(K) is transmitted from the sound playback memory unit R2 to the delay time imparting unit DL(K).

[0043] The sound playback device SREG includes a sound playback signal processing unit SP2, a microphone information reading unit MIR, a sound playback control unit CT2, and a speaker position input unit SINP. The sound playback device SREG includes one or more delay time imparting units DL(K) <K = 1, 2, 3, ···, N: N is a natural number> and a speaker driving unit SDR(K). The sound playback device SREG includes one or more speakers SP(K).

[0044] The sound playback memory unit R2 transmits the position information (X, Y, Z) of one or more microphones Mic(K) and one or more sound signals SS(K) to the sound playback signal processing unit SP2. The sound playback signal processing unit SP2 transmits one or more sound signals SS(K) to one or more delay time imparting units DL(K). Also, the sound playback signal processing unit SP2 transmits the position information (X, Y, Z) of the microphone Mic(K) to the sound playback control unit CT2 via the microphone information reading unit MIR.

[0045] The user transmits the position information (x, y, z) of one or more speakers SP(K) to the sound playback control unit CT2 by operating the speaker position input unit SINP.

[0046] Based on the position information (X, Y, Z) of one or more microphones Mic(K) and the position information (x, y, z) of one or more speakers SP(K), the sound playback control unit CT2 determines one or more delay times DT(K) <K = 1, 2, 3, ···, N: N is a natural number> of one or more sound signals SS(K). The sound playback control unit CT2 transmits one or more delay times DT(K) to one or more delay time imparting units DL(K). The delay time imparting unit DL(K) imparts one or more delay times DT(K) to one or more sound signals (K) respectively.

[0047] The speaker driver SDR(K) generates one or more sounds corresponding to one or more sound signals SS(K) in one or more speakers SP(K) (hereinafter, K=1, 2, 3,...N: N is a natural number) by delaying each of the one or more sounds by one or more delay times DT(K). The speaker driver SDR(K) includes one or more amplifiers Amp(K) (hereinafter, K=1, 2, 3,...N: N is a natural number) corresponding to each of the one or more speakers SP(K).

[0048] More specifically, when the sound reproducing device SREG reproduces one or more sound signals SS(K), position information (X, Y, Z) of one or more microphones Mic(K) at the time when the sound reproducing device SREC recorded the one or more sound signals SS(K) is read from the sound reproducing memory unit R2. The sound reproducing control unit CT2 then determines one or more delay times DT(K) corresponding to the position information (X, Y, Z) of one or more microphones Mic(K) and the position information (x, y, z) of one or more speakers SP(K). The delay time imparting unit DL(K) imparts one or more delay times DT(K) to the one or more sound signals SS(K). The speaker driving unit SDR(K) delays the generation of sounds based on one or more sound signals SS(K) acquired by one or more microphones Mic(K) by the calculated one or more delay times DT(K).

[0049] Therefore, the timing when the sound is expected to be heard at the reference position P (see FIG. 2) of one or more microphones Mic(K) coincides with the timing when the listener actually hears the sound at the reference position P (see FIG. 10). As a result, it is possible to improve the sense of realism of the sound generated by one or more speakers SP(K). In other words, by using the recording device SREC and the sound reproducing device SREG of the above-mentioned embodiment, the listener can hear a highly realistic sound.

[0050] FIG. 10 is a schematic diagram showing an example of the positional relationship between one or more speakers SP(K) in the sound reproduction device SREG according to the first embodiment and a reference position P where a listener is present.

[0051] In Fig. 10, a circle representing the position of the microphone Mic(2) when recording with the recording device SREC is placed at a predetermined distance from the reference position P, for example, 3 m in each of the X-axis and Z-axis directions, and 0 m in the Y-axis direction. Also in Fig. 10, a double circle representing the position of the speaker SP(2) is placed at a specific distance from the reference position P, for example, 1 m in each of the X-axis and Z-axis directions, and 0 m in the Y-axis direction. The position of the speaker SP(2) is the position when the sound of the microphone Mic(2) recorded with the recording device SREC is played back with the sound playback device SREG, and each of the one or more speakers SP(K) is placed on a straight line extending from the reference position P to one of the corresponding one or more microphones Mic(K).

[0052] The following methods can be used to input the position information (x, y, z) of the speaker SP(K). For example, first, along each of the three axes, the X-axis direction, the Y-axis direction, and the Z-axis direction, the distance from a reference position P where the listener is expected to listen to each of the one or more speakers SP(K) is measured with a tape measure or the like. This allows the distances along the three axes, the X-axis direction, the Y-axis direction, and the Z-axis direction, to be measured. The distances along the three axes are transmitted to the sound reproduction signal processing unit SP2 as the position information (x, y, z) of the speaker SP(K) by the user operating the speaker position input unit SINP.

[0053] Furthermore, if each of the one or more speakers SP(K) is equipped with a GPS (Global Positioning System) and a display (not shown), the following may be done. For example, a transmitter (not shown) that transmits position information (x, y, z) is installed at a reference position P. The one or more speakers SP(K) may then receive information on the reference position P transmitted by the transmitter, and the coordinates of the one or more speakers SP(K) relative to the reference position P in a three-axis coordinate system may be displayed on the display as position information (x, y, z).

[0054] Furthermore, the difference between the current position of the speaker SP(K) and the desired planned installation position of the desired speaker SP(K) may be displayed on the display of the speaker SP(K). In this case, the direction in which the speaker SP(K) should be moved may be displayed on the display of the speaker SP(K) to prompt the speaker SP(K) to move to the desired planned installation position of the speaker SP(K).

[0055] For example, the position information (-3m, 0m, +3m) of the microphone Mic(2) assigned to the sound signal (2) corresponding to the FrontLeft position and the position information (-1m, 0m, +1m) of the speaker SP(2) located in the FrontLeft position are used. The distance D between the microphone Mic(K) and the speaker SP(K) is calculated from the position information (-3m, 0m, +3m) of the microphone Mic(2) and the position information (-1m, 0m, +1m) of the speaker SP(2).

[0056] If the position information of the microphone Mic(2) is (Xm, Ym, Zm) and the position information of the speaker SP(2) is (Xs, Ys, Zs), the distance D is √(Xm^2+Zm^2)-√(Xs^2+Zs^2). Therefore, specifically, the delay time DT(K) of the FrontLeft channel in the delay time providing unit DL(2) shown in FIG. 9 is √(3^2+3^2)-√(1^2+1^2)=√18-√2=2.82 m.

[0057] Using the calculated distance D, the time t that the sound signal (2) travels is calculated as t = D [m] / 340 [m / s]. The delay time t in the delay time providing unit DL (2) shown in Figure 9 is 2.82 [m] / 340 [m / s] = 8.29 [ms].

[0058] The sound reproduction control unit CT2 determines one or more delay times DT(K) for each of one or more sound signals SS(K) emitted by one or more speakers SP(K). The delay time imparting unit DL(K) imparts one or more delay times DT(K) to each of the one or more sound signals SS(K). The sounds emitted by the one or more speakers SP(K) are delayed by the one or more delay times DT(K) determined by the one or more delay time imparting units DL(K).

[0059] As a result, the following can be said about a sound traveling from a position on an extension line of the direction from the reference position P toward the microphone Mic(K) toward the reference position P. In other words, the timing at which the sound actually emitted from the speaker SP(K) reaches the reference position P coincides with the timing at which the actual sound reaches the reference position P. Therefore, according to the recording device SREC and the sound playback storage unit R2 of this embodiment, even if the sound captured by one or more microphones Mic(K) is played back by one or more speakers SP(K), it is possible to hear a highly realistic sound.

[0060] (Embodiment 2) The recording device SREC and the sound reproducing device SREG of the second embodiment will be described. Note that, in the following, the description of the same points as the recording device SREC and the sound reproducing device SREG of the above-mentioned embodiment will not be repeated. The recording device SREC and the sound reproducing device SREG of this embodiment differ from the overall configuration of one or more microphones Mic(K) of the first embodiment in the overall configuration of one or more microphones Mic(K).

[0061] 11, in the overall configuration of one or more microphones Mic(K), multiple microphones Mic(K) are installed at the tips of multiple rod-shaped members that extend radially from the center, that is, protrude in all directions. Each of the multiple Mic(K) is assumed to be provided on a virtual spherical surface.

[0062] According to the overall configuration of the microphone Mic(K) shown in Fig. 11, the center position of the virtual sphere on which one or more microphones Mic(K) are provided becomes the reference position P, which is the listening position of the listener when sound is reproduced by the sound reproduction device SREG. Therefore, the positions of the tips of one or more rod-shaped members become the position information (X, Y, Z) of one or more microphones Mic(K). For example, if the length of each of one or more rod-shaped members is 10 cm, the coordinates of one or more microphones Mic(K) are any positions on the virtual sphere with a radius of 10 cm centered on the reference position P.

[0063] When the overall configuration of one or more microphones Mic(K) is miniaturized so that the length of each of the one or more rod-shaped members is approximately 0 cm, the position information (X, Y, Z) of each of the one or more microphones Mic(K) becomes (0, 0, 0). In this case, even if there is no position information (X, Y, Z) of one or more microphones Mic(K), if there is directional information (α, β) of one or more microphones Mic(K), it is possible to determine position information (x, y, z) of one or more speakers SP(K) and one or more delay times DT(K). The directional information (α, β) of the microphone Mic(K) is expressed using a rotation angle α and an elevation angle β in polar coordinates shown in FIG. 8.

[0064] FIG. 12 is a diagram for explaining that information capable of identifying the overall configuration of one or more microphones Mic(K) according to the second embodiment is associated with directional information (α, β) of the one or more microphones Mic(K).

[0065] Therefore, as shown in FIG. 12, a voice stream as a sound signal SS(K) The information added to the data of the Audio Stream is the [single point recording flag / multiple point recording flag] and [microphone direction information].

[0066] Which information, [single-point recording flag] or [multiple-point recording flag], is added to the sound signal SS(K) is determined by the user's selection using the microphone information input unit MINP. [Single-point recording flag] is a flag indicating that the overall configuration of one or more microphones Mic(K) can be assumed to be concentrated at approximately one point, as shown in Fig. 11. [Multiple-point recording flag] is a flag indicating that the overall configuration of one or more microphones Mic(K) is such that the positions of one or more microphones Mic(K) are dispersed, as shown in Fig. 2.

[0067] When the information added to the sound signal SS(K) is a [single-point recording flag], the information that follows it is [microphone directional information]. [Microphone directional information] is information that indicates the directional information (α, β) of the microphone Mic(K) relative to the reference position P. The directional information (α, β) is indicated by the rotation angle α and the elevation angle β, which are elements of the polar coordinate system. [Microphone directional information] is information that is input by the user using the microphone information input unit MINP.

[0068] In this embodiment, the directional information (α, β) of one or more microphones Mic(K) may be input by the user using the microphone information input unit MINP, based on the direction of one or more microphones Mic(K) indicated by a compass set at the reference position P. Each of the input directional information (α, β) of one or more microphones Mic(K) is transmitted from the microphone information input unit MINP via the recording control unit CT1 to the recording signal processing unit SP1 as directional information (α, β) of each microphone Mic(K).

[0069] However, if one or more microphones Mic(K) are equipped with a GPS (Global Positioning System) and a display (not shown), the following may be used. For example, a transmitter (not shown) that transmits its position information (X, Y, Z) is installed at a reference position P. Then, one or more microphones Mic(K) receive information on the reference position P transmitted by the transmitter, and directional information (α, β) of one or more microphones Mic(K) relative to the reference position P may be calculated from the position information of the reference position P and the position information of Mic(K) and displayed on a display. The directional information (α, β) of one or more microphones Mic(K) may be transmitted from the microphone information input unit MINP to the recording signal processing unit SP1 by a user operation.

[0070] Furthermore, when the information added to the sound signal SS(K) is [multi-point recording], the information that follows is position information of the microphone Mic(K), as shown in Figures 3 and 4. The position information of the microphone Mic(K) is information indicating the position information (X, Y, Z) of the microphone Mic(K) relative to the reference position P, as described in the first embodiment. The position information (X, Y, Z) of one or more microphones Mic(K) is determined by the same method as in the first embodiment.

[0071] FIG. 13 is a schematic diagram showing an example of the positional relationship between one or more speakers SP(K) in the sound reproduction device SREG according to the second embodiment and a reference position P where a listener is present.

[0072] In this embodiment, the position of the microphone Mic(K) of the sound recording device SREC can be considered to be the same as the reference position P, which is the position of the listener in the sound reproducing device SREG. Therefore, the position information (X, Y, Z) of the microphone Mic(K) is (0, 0, 0). In this embodiment, for example, the position information (x, y, z) of the speaker SP(2) of the sound reproducing device SREG is (-1m, 0, 1m) when the reference position P is (0, 0, 0).

[0073] Each of the one or more speakers SP(K) is disposed on an imaginary line extending from a reference position P toward a position where a microphone Mic(K) having a value of K corresponding to the speaker SP(K) is located.

[0074] For example, based on the position information (-1m, 0m, +1m) of the speaker SP(2) installed in the FrontLeft position of the sound reproduction device SREG, the distance D between the reference position P and the speaker SP(2) is calculated. Then, a delay time (2)=t is calculated in the same manner as described in the first embodiment. Then, these are set in the delay time applying unit DL(2) in FIG. 8. This makes the timing of the sound heard at the reference position P match the timing of the sound actually heard, improving the sense of realism.

[0075] To summarize the above, the following can be said:

[0076] The recording device SREC of this embodiment includes a recording signal processor SP1 and a recording storage unit R1, as in the first embodiment. The recording signal processor SP1 adds directional information (α, β) of one or more microphones Mic(K) input by the microphone information input unit MINP to one or more sound signals SS(K) acquired by one or more microphones Mic(K). The recording storage unit R1 associates the directional information (α, β) of one or more microphones Mic(K) with one or more sound signals SS(K) in a one-to-one relationship and stores them. In other words, the recording signal processor SP1 associates the directional information (α, β) of one or more microphones Mic(K) with one or more sound signals SS(K) in a one-to-one relationship and stores them in the recording storage unit R1.

[0077] The sound reproducing device SREG includes a sound reproducing storage unit R2, a delay time providing unit DL(K), and a speaker driving unit SDR(K). The sound reproducing storage unit R2 stores one or more sound signals SS(K). The sound reproducing control unit CT2 determines one or more delay times DT(K) for one or more sound signals SS(K) based on one or more speaker SP(K) position information (x, y, z) input by the speaker position input unit SINP. The delay time providing unit DL(K) provides one or more delay times DT(K) to the one or more sound signals SS(K). The speaker driving unit SDR(K) delays one or more actual sounds corresponding to the one or more sound signals SS(K) by one or more delay times DT(K) and generates them from one or more speakers SP(K).

[0078] The sound reproduction storage unit R2 stores direction information (α, β) of one or more microphones Mic(K) in a one-to-one relationship in association with one or more sound signals SS(K). When direction information (α, β) of one or more microphones Mic(K) is transmitted from the sound reproduction storage unit R2, the sound reproduction control unit CT2 determines one or more delay times of one or more sound signals SS(K) based only on one or more speaker SP(K) position information (x, y, z). When the information is transmitted from the sound reproduction storage unit R2, this includes the case where the information is transmitted from the sound reproduction storage unit R2 via other parts such as the sound reproduction signal processing unit SR2 and the microphone information reading unit MIR. The sound reproduction control unit CT2 determines whether or not direction information (α, β) of one or more microphones Mic(K) is transmitted from the sound reproduction storage unit R2. The sound playback control unit CT2 performs the above-mentioned judgment based on [single-point recording / multiple-point recording] as configuration information indicating the type of the overall configuration of one or more microphones Mic(K) added to the directional information (α, β) of one or more microphones Mic(K).

[0079] The recording device SREC and sound playback memory unit R2 of this embodiment also enable the listening of highly realistic sound by playing back sounds captured by one or more microphones Mic(K) through one or more speakers SP(K).

[0080] (Embodiment 3) The recording device SREC and the sound playback device SREG of Embodiment 3 will be described. Note that, in the following, descriptions of the same points as those of the recording device SREC and the sound playback device SREG of the foregoing embodiment will not be repeated. The recording device SREC and the sound playback device SREG of this embodiment are different from the recording device SREC and the sound playback device SREG of the embodiment in the following points.

[0081] One or more sound signals SS(K) recorded by the recording device SREC are added with another sound signal (N + 1), such as narration voice or music <N is a natural number>, in the sound playback device SREG. In this case, the other sound signal (N + 1), such as narration voice or music, is transmitted as a separate audio stream independent of the audio stream of the sound signal SS(K) recorded by the recording device SREC. Thereby, mixing with one or more sound signals SS(K) recorded by the recording device SREC for narration or music can be avoided.

[0082] In this embodiment, the position information of the microphone Mic(N + 1) given to the other sound signal (N + 1), such as narration voice or music, that is, the coordinates are (0, 0, 0) representing the reference position P.

[0083] However, it is not necessary to give the other sound signal (N + 1), such as narration or music, its position information (X, Y, Z), that is, the playback coordinates. In this case, for example, the data given to the audio stream is, as shown in FIG. 14, [microphone coordinate flag], [recording flag], and [microphone coordinate data]. The [microphone coordinate flag] is a flag that enables the subsequent microphone coordinate data. The [recording flag] is a flag indicating that the subsequent sound signal is another sound signal (N + 1) other than one or more sound signals SS(K) acquired by one or more microphones Mic(K), and is a flag that invalidates the subsequent [microphone coordinate data]. The [microphone coordinate data] is the coordinate information of the microphone Mic(K), that is, the [microphone coordinate data] functions as the position information (X, Y, Z) of Mic(K).

[0084] If there is no [Microphone Coordinate Flag], the [Microphone Coordinate Data] following the [Dubbing Flag] is invalidated. In other words, if there is no [Microphone Coordinate Flag], the position information (X, Y, Z) of the microphone Mic(K) is considered to be nonexistent. Therefore, the other sound signal (N+1) generates another sound in the speaker SP(K) without the delay time DT(K) based on the position information (X, Y, Z) of the microphone Mic(K).

[0085] In this embodiment, the following can be said.

[0086] The sound reproduction storage unit R2 stores another sound signal SS(K) different from the one or more sound signals SS(K). The speaker driving unit SDR(K) generates another sound corresponding to the other sound signal (N+1) from at least one of the one or more speakers SP(K) without a delay time based on the one or more delay times DT(K) described above. The other sound signal (N+1) is stored in the sound reproduction storage unit R2 in association with a flag as identification information that allows the other sound signal (N+1) to be identified from the one or more sound signals SS(K).

[0087] In the recording device SREC of this embodiment, the recording storage unit R1 stores other sound signals (N+1) that are not associated with position information of one or more microphones Mic(K) or directional information of one or more microphones Mic(K). The other sound signals (N+1) are stored in the recording storage unit R1 in association with flags that serve as identification information that allows the other sound signals (N+1) to be distinguished from the one or more sound signals SS(K).

[0088] (Embodiment 4) The sound recorder SREC and the sound player SREG of the fourth embodiment will be described. Note that, in the following, the description of the sound recorder SREC and the sound player SREG similar to those of the above-mentioned embodiments will not be repeated. The sound recorder SREC and the sound player SREG of the present embodiment differ from the sound recorder SREC and the sound player SREG of the embodiment in the following points.

[0089] When playing back sound using the sound player SREG, a narration voice or music is added as another sound signal (N+1) to the sound signal SS(K) recorded by the recorder SREC. In this case, the process for selecting the position for outputting the narration voice or music, i.e., the speaker SP(K), is as follows.

[0090] When playing back sound from the speaker SP(K), to specify the position where the narration or music can be heard, the position information where the narration or music can be heard is added to another sound signal (N+1) such as the narration or music as the "playback coordinates." The other sound signal (N+1) is also an audio stream.

[0091] In this case, in addition to the [microphone coordinate flag], a [playback position flag] and [playback coordinates] are added to the audio stream as shown in Fig. 15. The [microphone coordinate flag] in this embodiment is the same as the [microphone coordinate flag] in the third embodiment.

[0092] The [playback position flag] is a flag indicating that it is followed by [playback coordinates] that can specify the playback position of another sound signal SS(K) such as narration or music. The [playback coordinates] is coordinate information that can specify the playback position of the other sound signal (N+1). By specifying the playback position of the other sound, the position of the speaker SP(K) that emits the other sound, which is the narration or music specified by the other sound signal (N+1), can be specified.

[0093] The "playback coordinates" are not the position information of the speaker SP(K) of the recording device SREC itself, but are data that can identify the position from which the sound is being heard. Therefore, as long as the direction from which the sound is being heard and the distance from the source of the sound to the reference position P can be identified, the narration or music information may be generated by two or more speakers SP(K).

[0094] However, the [playback coordinates] may be the position information (X, Y, Z) of the speaker SP(K) that emits the sound specified by the other sound signal (N+1) added in the sound playback device SREG. In other words, the [playback coordinates] may be the coordinate data of the speaker SP(K) that emits other sounds such as narration or music when the sound is played by the sound playback device SREG, that is, the position information (x, y, z).

[0095] As can be seen from the above, the sound reproduction storage unit R2 stores information capable of identifying at least one speaker SP(K) that emits another sound corresponding to another sound signal (N+1) different from one or more sound signals SS(K). The speaker driving unit SDR(K) causes at least one speaker SP(K) identified by the information capable of identifying the at least one speaker SP(K) to generate another sound corresponding to the other sound signal (N+1). [Explanation of symbols]

[0096] DL(K) Delay time adding section DT(K) Delay Time Mic(K) MINP Microphone information input section SP1 Recording signal processing section SREC Recording device R1 Recording memory section R2 Sound playback storage section SDR(K) speaker driver SREG Sound Reproducing Device

Claims

1. a recording signal processing unit that adds directional information of the one or more microphones inputted by a microphone information input unit to one or more sound signals acquired by the one or more microphones, the recording signal processing unit causes the one or more microphone direction information and the one or more sound signals to be stored in a one-to-one relationship in a recording storage unit; The recording signal processing unit includes: adding configuration information indicating a type of an overall configuration of the one or more microphones input by the microphone information input unit to the directional information of the one or more microphones, and storing the directional information of the one or more microphones and the configuration information in the recording storage unit in association with each other; A recording device in which the types of the overall configuration of the one or more microphones include a single-point recording overall configuration in which the one or more microphones can be assumed to be concentrated at approximately one point, or a multi-point recording overall configuration in which the positions of the one or more microphones are distributed.

2. The recording device according to claim 1 , wherein each of the position information of the one or more microphones is coordinate information indicating rectangular coordinates or polar coordinates.

3. The recording device according to claim 1 or 2, wherein the recording signal processing unit associates other sound signals not associated with directional information of the one or more microphones with identification information capable of distinguishing the other sound signals from the one or more sound signals, and stores the other sound signals in the recording memory unit.

4. a sound reproduction control unit that determines one or more delay times of the one or more sound signals transmitted from a sound reproduction storage unit that stores one or more sound signals, based on position information of the one or more speakers input by a speaker position input unit; a delay time imparting unit that imparts the one or more delay times determined by the sound reproduction control unit to the one or more sound signals, respectively; a speaker driving unit that causes one or more speakers to generate one or more sounds corresponding to the one or more sound signals by delaying the one or more delay times, The directional information of one or more microphones is associated with the one or more sound signals in a one-to-one relationship; A sound playback device in which the sound playback control unit determines one or more delay times of the one or more sound signals based on position information of the one or more speakers when directional information of the one or more microphones is transmitted from the sound playback memory unit.

5. The position information of one or more microphones is associated with the one or more sound signals in a one-to-one relationship; The sound reproducing device according to claim 4 , wherein the sound reproducing control unit determines each of the one or more delay times based on position information of the one or more microphones and position information of the one or more speakers.

6. The sound reproduction storage unit is further provided, The sound reproducing device according to claim 5 , wherein the sound reproducing storage unit outputs each of the one or more microphone position information items, and then outputs each of the one or more sound signals in an order associated with each other in the one-to-one relationship.

7. The sound reproduction storage unit is further provided, The sound reproduction device according to claim 5 , wherein the sound reproduction storage unit alternately outputs any one of the one or more microphone position information and any one of the one or more sound signals in an order associated with the one-to-one relationship.

8. The sound playback device described in claim 5, wherein the sound playback control unit determines whether or not directional information of the one or more microphones has been transmitted from the sound playback memory unit based on information that can identify the type of overall configuration of the one or more microphones that is added to the directional information of the one or more microphones.

9. The sound reproduction device according to any one of claims 4 to 8, wherein the speaker driving unit generates another sound corresponding to another sound signal different from the one or more sound signals from at least one of the one or more speakers without a delay time based on the one or more delay times.

10. The sound reproducing device according to claim 9 , wherein the other sound signal is associated with identification information that is identifiable from the one or more sound signals.

11. The sound reproduction device according to claim 9 or 10, wherein the speaker driving unit generates the other sound in at least one of the one or more speakers identified by information capable of identifying at least one of the one or more speakers.

12. adding directional information of the one or more microphones inputted by a microphone information input unit to one or more sound signals acquired by the one or more microphones; and storing the directional information of the one or more microphones in a one-to-one association with the one or more sound signals, In the storing step, configuration information indicating a type of an overall configuration of the one or more microphones input by the microphone information input unit is added to the directional information of the one or more microphones, and the directional information of the one or more microphones and the configuration information are stored in association with each other; A recording method in which the type of overall configuration of the one or more microphones includes a single-point recording overall configuration in which the one or more microphones can be assumed to be concentrated at approximately one point, or a multi-point recording overall configuration in which the positions of the one or more microphones are distributed.

13. determining one or more delay times of one or more sound signals based on position information of one or more speakers input by a speaker position input unit; imparting the one or more delay times to the one or more sound signals, respectively; and generating one or more sounds corresponding to the one or more sound signals from one or more speakers, each of the sounds being delayed by the one or more delay times; The directional information of one or more microphones is associated with the one or more sound signals in a one-to-one relationship; A sound reproduction method in which, in the step of determining the delay time, when directional information of the one or more microphones is transmitted, one or more delay times of the one or more sound signals are determined based on position information of the one or more speakers.

Images