Signal processing device, control method for signal processing device, and program

Abstract

To provide a signal processing device, a control method for the signal processing device, and a program capable of synchronizing time information with sound signals even when the sound signals do not contain time codes.SOLUTION: A signal processing device includes: signal processing means for processing a sound signal; measuring means for measuring a delay time caused by signal processing by the signal processing means based on first time information output from timing means and second time information output from the signal processing means; and synchronizing means for synchronizing the time information output from the timing means with the sound signal processed by the signal processing means based on the delay time measured by the measuring means.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a signal processing apparatus, a control method thereof, and a program.

Background Art

[0002] Virtual viewpoint acoustic technology that generates an acoustic signal corresponding to a specified virtual viewpoint using acoustic data obtained by collecting sound with a plurality of microphones has attracted attention. Patent Document 1 discloses a technique for associating and recording acoustic data and a time code in order to generate virtual viewpoint acoustics.

[0003] In addition, a method of synchronizing acoustic data and a time code during an editing operation by sharing the time code among a plurality of devices is known. Patent Document 2 discloses a method of synchronizing acoustic content separately recorded together with a time code according to the time code obtained by demodulating a time code superimposed acoustic signal when reproducing main content in which the time code superimposed acoustic signal is recorded together with a video signal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 does not disclose a method for correcting the delay of acoustic data and the deviation of the time code that occur when the collected acoustic signal is delayed in lip sync adjustment or the like. Further, in Patent Document 2, it is necessary to prepare an acoustic track for the time code in the sound collection device. An object of the present invention is to enable synchronization of time information and an acoustic signal even when the acoustic signal does not include a time code.

Means for Solving the Problem

[0006] The signal processing apparatus according to the present invention includes signal processing means for signal-processing an acoustic signal, first time information output from timing means, and second time information output from the signal processing means, and measurement means for measuring a delay time caused by signal processing in the signal processing means based on the first time information and the second time information, and synchronization means for synchronizing the time information output from the timing means and the acoustic signal processed by the signal processing means based on the delay time measured by the measurement means.

Effect of the Invention

[0007] According to the present invention, it is possible to synchronize time information and an acoustic signal.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Also, the same or similar configurations, etc. are denoted by the same reference numerals, and duplicate explanations are omitted.

[0010] For each embodiment, as an example, an example in which baseball games are the target of sound collection will be described. Specifically, competitive sounds such as hitting sounds and catching sounds generated in the stadium, cheering sounds from the spectator stands, and furthermore, in the case of television broadcasts, sounds such as live commentary and explanations will be the target of sound collection, and an example of synchronously recording the collected acoustic signals with time codes will be mainly described. However, the application target of the present invention is not limited to baseball games, and may be, for example, other competitions or music concerts, etc.

[0011] <Embodiment 1> In this embodiment, an example of synchronously recording acoustic data based on an acoustic signal obtained from a general relay system and time information based on a time code for a television relay of a baseball game will be described.

[0012] FIG. 1 is a block diagram showing a functional configuration example of a signal processing apparatus 100 in Embodiment 1. The signal processing apparatus 100 includes a time code generation unit 110, a sound collection unit 120, a signal processing unit 130, a synchronization unit 140, and a recording unit 150. The synchronization unit 140 includes a control unit 141, a measurement unit 142, and a time code correction unit 143.

[0013] In this embodiment, the signal processing apparatus 100 has a measurement mode and a recording mode as operation modes. In the measurement mode, the signal processing apparatus 100 measures the delay amount (delay time) due to signal processing in the signal processing unit 130, and calculates a time code correction value based on the measured delay amount (delay time). Further, in the recording mode, the signal processing apparatus 100 corrects the time code based on the time code correction value calculated in the measurement mode, and synchronizes and records the time information based on the time code and the acoustic data based on the acoustic signal. In baseball games, the signal processing apparatus 100 can synchronize and record the time information and the acoustic data by operating in the measurement mode, for example, before the start of the game, and in the recording mode during the game.

[0014] The time code generation unit 110 generates a time code indicating a time position, and outputs the generated time code to the signal processing unit 130, the measurement unit 142, and the time code correction unit 143. The time code generation unit 110 is, for example, a grand master or a device synchronized with the grand master. Here, the time code may be, for example, LTC (Linear Timecode, Longitudinal Timecode) standardized by SMPTE. The time code generation unit 110 is an example of a timing means. The time code generated by the time code generation unit 110 is an example of the first time information.

[0015] The sound collection unit 120 collects the sound emitted by the sound collection target, and outputs it to the signal processing unit 130 as an acoustic signal composed of one or more channels. The sound collection unit 120 may be composed of, for example, a plurality of microphones, or may be composed of different types of microphones. In this embodiment, the acoustic signal output from the sound collection unit 120 is not provided with an acoustic track for the time code and does not include the time code.

[0016] The signal processing unit 130 receives the time code from the time code generation unit 110 or the acoustic signal from the sound collection unit 120 and performs signal processing. In the present embodiment, in the measurement mode, the time code from the time code generation unit 110 is input to the signal processing unit 130, and in the recording mode, the acoustic signal is input from the sound collection unit 120. The signal processing unit 130 outputs the input time code or the acoustic signal subjected to signal processing to the measurement unit 142 and the time code correction unit 143. In the present embodiment, the signal processing unit 130 outputs the input time code as it is. In the following description, the time code output from the signal processing unit 130 is also referred to as the "measured time code". The time code (measured time code) output from the signal processing unit 130 is an example of the second time information.

[0017] Here, the signal processing unit 130 is, for example, an audio mixer, a frame synchronizer, or the like. Further, the signal processing unit 130 may be a group of devices combining a plurality of them. In the signal processing unit 130, as shown in an example in FIG. 2, various signal processes are performed on the input signal. The processes performed by the signal processing unit 130 include, for example, analog-to-digital conversion processing (ADC) 201, high-pass filter processing (HPF) 202, and equalization processing (EQ) 203. Further, for example, delay processing (Delay) 204, mixing processing (Mix) 205, digital-to-analog conversion processing (DAC) 206, and the like are included. For example, when the signal processing unit 130 performs lip sync for video and audio in applications such as television relay, delay processing is applied to the acoustic signal. Note that the signal processing unit 130 shown in FIG. 2 is an example, and other processes may be included, or some of the processes shown in FIG. 2 may not be included.

[0018] Here, when the signal processing unit 130 performs the above-described signal processing on the acoustic signal, the time at which the acoustic signal is output is delayed (delay time) from the time code output from the time code generation unit 110 at that time. For example, when the signal processing unit 130 applies delay processing to the acoustic signal, the acoustic signal output from the signal processing unit 130 is delayed by the amount of delay processing with respect to the time code output from the time code generation unit 110 at the same time.

[0019] The synchronization unit 140 corrects the deviation between the time code from the time code generation unit 110 and the acoustic signal from the signal processing unit 130, and performs processing related to the synchronization of the time code and the acoustic signal. In other words, the synchronization unit 140 performs processing so that the acoustic signal can be recorded with a time code reflecting the processing time (delay time) of the signal processing unit 130. As described above, the synchronization unit 140 includes a control unit 141, a measurement unit 142, and a time code correction unit 143.

[0020] The control unit 141 outputs control information and the like based on an input by a user operation or the like, and controls the operation of the signal processing apparatus 100. For example, the control unit 141 controls whether to perform processing related to measurement of the delay amount (delay time) due to signal processing in the signal processing unit 130 (operation in the measurement mode) or processing related to synchronization between the time information based on the time code and the acoustic signal (operation in the recording mode). Further, for example, the control unit 141 measures the delay amount (delay time) due to signal processing in the signal processing unit 130 and outputs a delay measurement request for causing the calculation of the time code correction value to the measurement unit 142. Further, the control unit 141 sets the time code correction value acquired from the measurement unit 142 to the time code correction unit 143.

[0021] Based on the delay measurement request from the control unit 141, the measurement unit 142 measures the delay amount (delay time) due to signal processing in the signal processing unit 130 and calculates the time code correction value. The measurement unit 142 measures the delay amount (delay time) in the signal processing unit 130 by comparing the time code acquired from the time code generation unit 110 with the time code (measured time code) acquired from the signal processing unit 130. Further, the measurement unit 142 calculates a time code correction value for synchronously recording the acoustic signal and the time code from the measurement result, and outputs the calculated time code correction value to the control unit 141.

[0022] The time code correction unit 143 synchronizes the time code and the acoustic signal based on the time code acquired from the time code generation unit 110, the acoustic signal acquired from the signal processing unit 130, and the time code correction value acquired from the control unit 141. Then, the time code correction unit 143 uses the synchronized time code as time information, and outputs the acoustic signal cut out for each time information as acoustic data to the recording unit 150. This synchronized time code is an example of the third time information. The time code correction unit 143 cuts out the acoustic signal in a predetermined unit (for example, a predetermined frame unit, a predetermined data sample unit, etc.) according to the time information to obtain the acoustic data. Therefore, for example, when cutting out the acoustic signal in a predetermined frame unit or a predetermined data sample unit, the unit of the time information is an integer multiple of the time width of these frame units or data sample units. The recording unit 150 records the time information and the acoustic data input from the time code correction unit 143.

[0023] In the example shown in FIG. 1, the signal processing apparatus 100 is configured to have the time code generation unit 110, but the configuration is not limited thereto. For example, the signal processing apparatus 100 may be connected to a time code generation apparatus outside the signal processing apparatus 100 via a cable or the like, and acquire the time code from the time code generation apparatus.

[0024] Also, in the example shown in FIG. 1, the signal processing apparatus 100 is configured to include the sound collection unit 120, but the configuration is not limited to this. For example, the signal processing apparatus 100 may be connected to a sound collection device such as a microphone outside the signal processing apparatus 100 via a cable or the like, and acquire an acoustic signal from the sound collection device.

[0025] FIG. 3 is a block diagram showing an example of the hardware configuration of the signal processing apparatus according to the present embodiment. The signal processing apparatus according to the present embodiment includes a CPU 301, a ROM 302, a RAM 303, an auxiliary storage device 304, a display unit 305, an acoustic input unit 306, an operation unit 307, a communication I / F 308, and a bus 309. The CPU 301, the ROM 302, the RAM 303, the auxiliary storage device 304, the display unit 305, the acoustic input unit 306, the operation unit 307, and the communication I / F 308 are connected to be communicable with each other via the bus 309. Note that the signal processing apparatus may further include other configurations.

[0026] The CPU (Central Processing Unit) 301 controls the entire signal processing apparatus using programs (computer programs) and data stored in the ROM 302 and the RAM 303, thereby realizing each function of the signal processing apparatus shown in FIG. 1. The CPU 301 realizes processing according to each flowchart described later, for example, by reading and executing a program stored in the ROM 302 or the like. Note that the signal processing apparatus may have one or more dedicated hardware different from the CPU 301, and at least a part of the processing by the CPU 301 may be executed by the dedicated hardware. Examples of the dedicated hardware include an ASIC (Application Specific Integrated Circuit) and the like.

[0027] The ROM (Read Only Memory) 302 stores various programs and various data, etc. The ROM 302 stores, for example, programs related to the control of the signal processing device by the CPU 301, etc. The RAM (Random Access Memory) 303 temporarily stores programs and data supplied from the auxiliary storage device 304, etc., and data, etc. supplied from the outside via the communication I / F 308. The auxiliary storage device 304 is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), etc., and stores various data such as acoustic signals and setting information.

[0028] The display unit 305 is composed of, for example, a liquid crystal display, an LED, etc., and displays a GUI (Graphical User Interface) etc. for the user to operate the system. The acoustic input unit 306 is composed of, for example, a microphone, an A / D converter, etc., and inputs the collected acoustic signal. The operation unit 307 is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, etc., and receives an operation by the user and inputs various instructions to the CPU 301. The CPU 301 operates as a display control unit that controls the display unit 305 and an operation control unit that controls the operation unit 307.

[0029] The communication I / F (interface) 308 is used for communication with an external device. For example, when the signal processing device is connected to an external device by wire, a communication cable is connected to the communication I / F 308. Also, for example, when the signal processing device has a function of wireless communication with an external device, the communication I / F 308 is provided with an antenna. The bus 309 connects each part and transmits information.

[0030] FIG. 4 is a flowchart showing an example of processing of the signal processing apparatus according to Embodiment 1. The processing shown in FIG. 4 starts, for example, at the timing when the signal processing apparatus 100 receives an instruction to start processing related to recording of acoustic data. The instruction to start the processing may be given by a user operation via the operation unit 307 or the like of the signal processing apparatus 100, or an instruction may be input from another apparatus. Note that the timing to start the execution of the processing shown in FIG. 4 is not limited to the timing described above. The processing shown in FIG. 4 is realized by the CPU 301 expanding and executing a program stored in the ROM 302 or the like in the RAM 303. Note that at least a part of the processing shown in FIG. 4 may be executed by one or a plurality of dedicated hardware different from the CPU 301.

[0031] In step S401, the time code correction unit 143 initializes the time code correction value. The time code correction value is a value indicating time, and may be, for example, in units of one frame. Also, the initial value of the time code correction value may be 0 (zero) or a predetermined value.

[0032] In step S402, the time code generation unit 110 starts generating a time code. The time code generation unit 110 outputs the generated time code to the signal processing unit 130, the measurement unit 142, and the time code correction unit 143.

[0033] In step S403, the control unit 141 receives a user operation or the like via the operation unit 307 and outputs control information to the measurement unit 142 and the time code correction unit 143. Here, the control information is information regarding the operation control of the signal processing apparatus 100, and is information indicating operation modes such as the measurement mode and the recording mode, and the end of the operation. In the measurement mode, the signal processing apparatus 100 measures the delay amount (delay time) due to signal processing in the signal processing unit 130 to obtain a time code correction value, and performs an operation of setting the obtained time code correction value. Further, in the recording mode, the signal processing apparatus 100 corrects the time code based on the set time code correction value, and performs an operation of synchronizing and recording the time information based on the time code and the acoustic data based on the acoustic signal.

[0034] In step S404, the measurement unit 142 receives the control information from the control unit 141 and determines whether the control information indicates the measurement mode. If the measurement unit 142 determines that the control information indicates the measurement mode (YES in step S404), the process of step S405 is executed. On the other hand, if the measurement unit 142 determines that the control information does not indicate the measurement mode (NO in step S404), the process of step S407 is executed.

[0035] In step S405, the measurement unit 142 measures the delay amount (delay time) due to signal processing in the signal processing unit 130 and calculates a time code correction value. The measurement unit 142 receives the time code from the time code generation unit 110 and the time code from the signal processing unit 130 (measured time code), and sets the value obtained by subtracting the time code from the measured time code as the time code correction value. The time code correction value may be in units of frames. For example, when the time code is "13:00:15;20" and the measured time code is "13:00:15;15", the time code correction value is (-5) frames. The measurement unit 142 outputs the value calculated in this way to the control unit 141 as the time code correction value.

[0036] Note that the signal level of the measured time code may decrease with respect to the time code due to the signal processing by the signal processing unit 130, which may cause a problem in the analysis of the measured time code by the measurement unit 142. Therefore, a level correction unit (not shown) for correcting the signal level of the measured time code may be provided between the measurement unit 142 and the signal processing unit 130.

[0037] In step S406, the control unit 141 receives the input of the time code correction value from the measurement unit 142 and sets the time code correction value in the time code correction unit 143. After the process of step S406 is performed, the process of step S403 is executed.

[0038] In step S407, the measurement unit 142 receives the control information from the control unit 141 and determines whether the control information indicates the recording mode. If the measurement unit 142 determines that the control information indicates the recording mode (YES in step S407), the process of step S408 is executed. On the other hand, if the measurement unit 142 determines that the control information does not indicate the recording mode (NO in step S407), the process of step S410 is executed.

[0039] In step S408, the time code correction unit 143 receives the time code from the time code generation unit 110 and the acoustic signal from the signal processing unit 130, and corrects the time code based on the time code correction value set in step S406. Further, the time code correction unit 143 uses the corrected time code as time information, cuts out the acoustic signal based on the time information, and outputs the cut-out acoustic signal as acoustic data together with the time information to the recording unit 150. For example, the time code correction unit 143 includes the time information in a predetermined channel of the acoustic signal composed of a plurality of channels and outputs the time information and the acoustic data.

[0040] A method for correcting a time code using a time code correction value will be described with reference to FIG. 5. First, it is assumed that an acoustic signal is cut out by a time code. As shown in FIG. 5(a), the time code that should originally be assigned to the acoustic signal t1 is the time code TC501 of "13:00:01;15". Similarly, the time code that should be assigned to the acoustic signal t2 is the time code TC502 of "13:00:01;16", and the time code that should be assigned to the acoustic signal t3 is the time code TC503 of "13:00:01;17".

[0041] Here, as described above, it is assumed that the acoustic signal is delayed by signal processing (for example, delay processing) in the signal processing unit 130. For example, when the delay amount DL is one frame of the time code, as shown in FIG. 5(b), the time code TC511 of "13:00:01;16" advanced by one frame is associated with the acoustic signal t1. Similarly, the time code TC512 of "13:00:01;17" advanced by one frame is associated with the acoustic signal t2.

[0042] The value for correcting this deviation is the time code correction value. As shown in FIG. 5(c), by correcting the time code by minus one frame according to the time code correction value, it becomes possible to synchronize and record the time code and the acoustic data. For example, in the example shown in FIG. 5(c), the time code TC521 of "13:00:01;15" corrected by minus one frame with respect to the time code ("13:00:01;16") from the time code generation unit 110 is assigned to the acoustic signal t1. Similarly, the time code TC522 of "13:00:01;16" corrected by minus one frame with respect to the time code ("13:00:01;17") from the time code generation unit 110 is assigned to the acoustic signal t2. In this way, the time code is corrected by adding the time code correction value to the time code from the time code generation unit 110 and assigned to the acoustic signal.

[0043] Note that the time code correction value does not necessarily have to be a time code in units of one frame, and may be, for example, time information in units of 1 ms. In that case, for a deviation less than one frame, by correcting the acoustic signal in the time direction, it becomes possible to more accurately synchronize the time code and the acoustic data.

[0044] In step S409, the recording unit 150 receives the input of the time information and the acoustic data by the time code correction unit 143, and records the time information and the acoustic data. After the process of step S409 is performed, the process of step S403 is executed.

[0045] In step S410, the measurement unit 142 receives the control information from the control unit 141, and determines whether the control information is an end instruction. If the measurement unit 142 determines that the control information is an end instruction (YES in step S410), the process shown in FIG. 4 is ended. On the other hand, if the measurement unit 142 determines that the control information is not an end instruction (NO in step S410), the process of step S403 is executed.

[0046] Note that in the recording mode, since it is not necessary to input the time code to the signal processing unit 130, it may be possible to prevent the time code from being mixed into the acoustic signal to be recorded. Specifically, a switch (not shown) linked to the control information output by the control unit 141 may be provided between the time code generation unit 110 and the signal processing unit 130, and the input of the time code from the time code generation unit 110 to the signal processing unit 130 may be blocked during the recording mode. Alternatively, the signal processing unit 130 may be linked to the control unit 141 so that the time code input is muted during the recording mode.

[0047] According to the present embodiment, the measurement unit 142 measures the delay amount (delay time) due to signal processing in the signal processing unit 130 based on the time code and the measured time code, and calculates a time code correction value. Then, the time code correction unit 143 corrects the time code based on the time code correction value calculated by the measurement unit 142, and synchronizes the time information based on the corrected time code with the acoustic signal. As a result, even if the acoustic signal does not include a time code, it is possible to correct the delay of the acoustic signal caused by signal processing and the deviation of the time code, and synchronize the time information with the acoustic signal. For example, it is possible to synchronize and record the time information and the acoustic data without recording the time code on the acoustic track.

[0048] <Embodiment 2> In Embodiment 2, a method for synchronously recording time information and acoustic data for a plurality of acoustic signals having different delay amounts (delay times) will be described. For example, it can be applied to a case where Company A is in charge of the conventional stereo broadcast sound and Company B is in charge of the stereophonic sound for a baseball game TV relay. In this case, the sound collection device and the signal processing device used by Company A and the sound collection device and the signal processing device used by Company B are composed of different devices and may be used simultaneously. In such a case, although there may be a difference in the delay amount due to signal processing or the like between the acoustic signal by Company A and the acoustic signal by Company B, it will be explained that the time information and the acoustic data can be synchronously recorded according to the present embodiment.

[0049] FIG. 6 is a block diagram showing a functional configuration example of the signal processing device 600 in Embodiment 2. In FIG. 6, components having the same functions as those shown in FIG. 1 are denoted by the same reference numerals, and redundant descriptions are omitted. The signal processing device 600 includes a time code generation unit 110, a sound collection unit 120, a signal processing unit 130, a synchronization unit 610, and a recording unit 150. The synchronization unit 610 includes a control unit 611, a measurement unit 612, a time code correction unit 613, and a delay correction unit 614.

[0050] In the signal processing apparatus 600, the sound collection unit 120 is composed of S sound collection units from the sound collection unit 120-1 to the sound collection unit 120-S. Hereinafter, any sound collection unit is represented as the sound collection unit 120-i. Further, the signal processing unit 130 is composed of S signal processing units from the signal processing unit 130-1 to the signal processing unit 130-S corresponding to each of the sound collection units from the sound collection unit 120-1 to the sound collection unit 120-S. Hereinafter, any signal processing unit is represented as the signal processing unit 130-i.

[0051] The sound collection unit 120-i has each microphone collect the sound (voice) emitted by a sound collection target such as a subject, and outputs the obtained acoustic signal to the signal processing unit 130-i. The signal processing unit 130-i receives and processes the acoustic signal from the time code generation unit 110 or the corresponding sound collection unit 120-i. In the present embodiment, in the signal processing unit 130-i, the time code from the time code generation unit 110 is input in the measurement mode, and the acoustic signal is input from the sound collection unit 120-i in the recording mode. The signal processing unit 130-i outputs the input time code (measured time code) or the acoustic signal subjected to signal processing to the measurement unit 612 and the time code correction unit 613. Note that, in the present embodiment, the processing that the signal processing unit 130-i can perform on the time code and the acoustic signal is the same as that of the signal processing unit 130 in the first embodiment.

[0052] The synchronization unit 610 corrects the delay amount between the acoustic signals from the signal processing unit 130-i and the deviation between the time code from the time code generation unit 110 and the acoustic signal, and performs processing related to the synchronization of the time code and the acoustic signal. As described above, the synchronization unit 610 includes a control unit 611, a measurement unit 612, a time code correction unit 613, and a delay correction unit 614.

[0053] The control unit 611 outputs control information and the like based on inputs such as user operations, and controls the operation of the signal processing apparatus 100. For example, the control unit 611 controls whether to perform processing related to measurement of the delay amount (delay time) by signal processing in the signal processing unit 130-i (operation in the measurement mode), or processing related to synchronization of time information based on the time code and the acoustic signal (operation in the recording mode). Further, for example, the control unit 611 measures the delay amount (delay time) by signal processing in the signal processing unit 130-i and outputs a delay measurement request to the measurement unit 612 to cause calculation of a delay correction value and a time code correction value. Also, the control unit 611 sets the time code correction value acquired from the measurement unit 612 in the time code correction unit 613, and outputs the delay correction value acquired from the measurement unit 612 to the delay correction unit 614. Here, the delay correction value is for correcting the delay difference between the acoustic signals output from the signal processing unit 130-i.

[0054] Based on the delay measurement request from the control unit 611, the measurement unit 612 measures the delay amount (delay time) by signal processing in the signal processing unit 130-i, and calculates a delay correction value and a time code correction value based on the measurement result. The delay correction value and the time code correction value calculated by the measurement unit 612 are output to the control unit 611.

[0055] Based on the time code acquired from the time code generation unit 110, the acoustic signal whose delay difference between acoustic signals is corrected by the delay correction unit 614, and the time code correction value acquired from the control unit 611, the time code correction unit 613 synchronizes the time code and the acoustic signal. Then, the time code correction unit 613 uses the synchronized time code as time information, and outputs the acoustic signal cut out for each time information as acoustic data to the recording unit 150.

[0056] The delay correction unit 614 receives the input of the acoustic signal by the signal processing unit 130 and the delay correction value by the control unit 611, corrects the delay amount for each acoustic signal, and outputs the acoustic signal with the delay difference between the acoustic signals corrected to the time code correction unit 613.

[0057] In the example shown in FIG. 6, the signal processing apparatus 600 is configured to include the time code generation unit 110, but it is not limited to this configuration. For example, the signal processing apparatus 600 may be connected to a time code generation device outside the signal processing apparatus 600 via a cable or the like, and acquire a time code from the time code generation device.

[0058] Also, in the example shown in FIG. 6, the signal processing apparatus 600 is configured to include the sound collection unit 120, but it is not limited to this configuration. For example, the signal processing apparatus 600 may be connected to a sound collection device such as a microphone outside the signal processing apparatus 600 via a cable or the like, and acquire an acoustic signal from the sound collection device. Note that the hardware configuration of the signal processing apparatus in Embodiment 2 is the same as that of the signal processing apparatus in Embodiment 1 shown in FIG. 3.

[0059] In the measurement mode, the signal processing apparatus 600 in the present embodiment calculates a delay correction value for correcting the delay amount between a plurality of acoustic signals in addition to the time code correction value. Further, in the recording mode, the signal processing apparatus 600 corrects the delay amount between acoustic signals based on the delay correction value, corrects the time code based on the time code correction value, and synchronizes and records the time information and the acoustic data.

[0060] Specifically, in the measurement mode, the signal processing apparatus 600 calculates a delay amount i that is the delay amount of the signal processing unit 130-i, and a maximum delay amount max that is the largest among them. Subsequently, the signal processing apparatus 600 calculates a delay correction value i for correcting the delay amount of the acoustic signal i obtained from the signal processing unit 130-i and a time code correction value based on the delay amount i and the maximum delay amount max.

[0061] Then, in the recording mode, the signal processing device 600 corrects the time shift between acoustic signals by correcting the delay amount of the acoustic signal i based on the delay correction value i in the delay correction unit 614. Further, the signal processing device 600 corrects the time code shift based on the time code correction value in the time code correction unit 613, synchronizes the time information and the acoustic data, and records them.

[0062] FIG. 7 is a diagram for explaining the flow of correction of the delay amount of each acoustic signal and the time code. FIG. 7(a) shows that each acoustic signal 702-1 to 702-S output from the signal processing units 130-1 to 130-S is delayed by different delay amounts with respect to the time code 701. In FIG. 7(a), 711 indicates the delay amount (the maximum delay amount of the acoustic signal) with respect to the time code 701 of the acoustic signal with the largest delay amount (in this example, the acoustic signal <2> 702-2). FIG. 7(b) shows an image of the delay amount correction of each acoustic signal by the delay correction value calculated by the measurement unit 612. In the example shown in FIG. 7(b), based on each delay correction value, the acoustic signal <1> 702-1 is delayed by the delay amount 721, and the acoustic signal <s>By delaying 702-S by the delay amount 722, the delay difference between the acoustic signals is corrected so that they have the same delay amount. FIG. 7(c) shows an image of correcting the time code for the acoustic signals with the delay difference between the acoustic signals corrected. Based on the time code correction value, the time code is corrected by offsetting it by the correction amount 723 corresponding to the maximum delay amount 711 shown in FIG. 7(a), thereby synchronizing the time code 701 and the acoustic signals 702-1 to 702-S.

[0063] FIG. 8 is a flowchart showing an example of processing of the signal processing apparatus according to Embodiment 2. The processing shown in FIG. 8 starts, for example, at the timing when the signal processing apparatus 600 receives an instruction to start processing related to recording of acoustic data. The instruction to start the processing may be given by a user operation via the operation unit 307 of the signal processing apparatus 600 or the like, or an instruction may be input from another apparatus. Note that the timing to start the execution of the processing shown in FIG. 8 is not limited to the timing described above. The processing shown in FIG. 8 is realized by the CPU 301 expanding the program stored in the ROM 302 or the like into the RAM 303 and executing it. Note that at least a part of the processing shown in FIG. 8 may be executed by one or a plurality of dedicated hardware different from the CPU 301.

[0064] In step S801, the time code correction unit 613 initializes the time code correction value. In step S802, the time code generation unit 110 starts generating the time code and outputs the generated time code to the signal processing unit 130 (signal processing unit 130-i), the measurement unit 612, and the time code correction unit 613. In step S803, the control unit 611 receives a user operation or the like via the operation unit 307 and outputs control information to the measurement unit 612, the time code correction unit 613, and the delay correction unit 614.

[0065] In step S804, the measurement unit 612 receives the control information from the control unit 611 and determines whether the control information is information indicating the measurement mode. If the measurement unit 612 determines that the control information is information indicating the measurement mode (YES in step S804), the process of step S805 is executed. On the other hand, if the measurement unit 612 determines that the control information is not information indicating the measurement mode (NO in step S804), the process of step S811 is executed.

[0066] In step S805, the measurement unit 612 initializes the value of the delay amount max and the parameter i indicating the signal processing unit 130 to be measured to 0 (zero). After performing the process of step S805, the measurement of the delay amount (delay time) of each signal processing unit 130 is started, and the processes from step S806 to step S809 are executed for each of the signal processing units 130-i.

[0067] In step S806, the measurement unit 612 receives the time code from the time code generation unit 110 and the time code (measured time code) i from the signal processing unit 130-i, and calculates the delay amount i of the signal processing unit 130-i. The measurement unit 612 calculates the difference between the time code from the time code generation unit 110 and the time code (measured time code) i, and holds the calculated value as the delay amount i of the acoustic signal i by signal processing in the signal processing unit 130-i.

[0068] In step S807, the measurement unit 612 compares the delay amount i calculated in step S806 with the delay amount max, and determines whether the delay amount i is greater than the delay amount max. If the measurement unit 612 determines that the delay amount i calculated in step S806 is greater than the delay amount max (YES in step S807), the process of step S808 is executed. On the other hand, if the measurement unit 612 determines that the delay amount i calculated in step S806 is less than or equal to the delay amount max (NO in step S807), the process of step S809 is executed.

[0069] In step S808, the measurement unit 612 overwrites the delay amount max with the delay amount i. In step S809, the measurement unit 612 increments the value of parameter i by 1.

[0070] As described above, the measurement unit 612 repeats the processes from step S806 to step S809 S times, which is the number of signal processing units 130. As a result, the delay amounts 1 to S and the maximum delay amount corresponding to the signal processing units 130-1 to 130-S are obtained. When the processes from step S806 to step S809 are executed for all the signal processing units 130-i and the measurement of the delay amount (delay time) of each signal processing unit 130 is completed, the process of step S810 is executed.

[0071] In step S810, the measurement unit 612 calculates a delay correction value i for correcting the delay amount of each acoustic signal and a time code correction value, and outputs them to the control unit 611. The measurement unit 612 calculates the delay correction value i for each acoustic signal so that the delay amounts of the acoustic signals match. Specifically, the measurement unit 612 sets the value obtained by subtracting the delay amount i from the maximum delay amount max obtained as described above as the delay correction value i. Further, the measurement unit 612 sets the maximum delay amount max obtained as described above as the time code correction value. Then, the control unit 611 receives the input of the delay correction value i and the time code correction value from the measurement unit 612, outputs the received delay correction value i to the delay correction unit 614, and sets the time code correction value in the time code correction unit 613. After the process of step S810 is performed, the process of step S803 is executed.

[0072] In step S811, the measurement unit 612 receives control information from the control unit 611 and determines whether the control information indicates the recording mode. When the measurement unit 612 determines that the control information indicates the recording mode (YES in step S811), the process of step S812 is executed. On the other hand, when the measurement unit 612 determines that the control information does not indicate the recording mode (NO in step S811), the process of step S815 is executed.

[0073] In step S812, the delay correction unit 614 delays the acoustic signal i from the signal processing unit 130-i based on the delay correction value i received from the control unit 611, and outputs the delayed acoustic signal i to the time code correction unit 613.

[0074] In step S813, the time code correction unit 613 receives the time code from the time code generation unit 110 and the acoustic signal from the delay correction unit 614, and corrects the time code based on the time code correction value. Further, the time code correction unit 613 uses the corrected time code as time information, cuts out the acoustic signal based on the time information, and outputs the cut-out acoustic signal as acoustic data together with the time information to the recording unit 150.

[0075] In step S814, the recording unit 150 receives the input of the time information and the acoustic data by the time code correction unit 613, and records the time information and the acoustic data. After the process of step S814 is performed, the process of step S803 is executed.

[0076] In step S815, the measurement unit 612 receives the control information from the control unit 611, and determines whether the control information is an end instruction. If the measurement unit 612 determines that the control information is an end instruction (YES in step S815), the process shown in FIG. 8 is terminated. On the other hand, if the measurement unit 612 determines that the control information is not an end instruction (NO in step S815), the process of step S803 is executed.

[0077] According to the present embodiment, it is possible to correct the delay of the acoustic signal and the deviation of the time code, and synchronize the time information and the acoustic signal, even for a plurality of acoustic signals having different delay amounts (delay times) by signal processing.

[0078] (Other embodiments of the present invention) The present invention can also be implemented by supplying a program that realizes one or more functions of the foregoing embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. It can also be implemented by a circuit (for example, ASIC) that realizes one or more functions.

[0079] Note that the foregoing embodiments are merely examples of implementation when carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0080] The disclosure of this embodiment includes the following configurations and methods, etc. (Configuration 1) Signal processing means for signal-processing an acoustic signal; Measurement means for measuring a delay time due to signal processing in the signal processing means based on first time information output from timing means and second time information output from the signal processing means; A signal processing apparatus, comprising: synchronization means for synchronizing time information output from the timing means and the acoustic signal signal-processed by the signal processing means based on the delay time measured by the measurement means. (Configuration 2) The synchronization means outputs the acoustic signal cut out based on third time information synchronized by the synchronization means as acoustic data, and outputs the third time information and the acoustic data; The signal processing apparatus according to Configuration 1, further comprising recording means for recording the third time information and the acoustic data output from the synchronization means. (Configuration 3) The synchronization means cuts out the acoustic signal in a predetermined unit corresponding to the third time information to obtain the acoustic data, according to the signal processing apparatus according to Configuration 2. (Configuration 4) The synchronization means outputs the third time information and the acoustic data by including time information in a predetermined channel of the acoustic signal, according to the signal processing apparatus described in Configuration 2 or 3, which is characterized in that. (Configuration 5) The signal processing apparatus according to any one of Configurations 1 to 4, characterized in that it has control means for controlling whether to perform processing for measuring the delay time by the measurement means or processing for synchronizing the time information and the acoustic signal by the synchronization means. (Configuration 6) The measurement means calculates the delay time by subtracting the first time information from the second time information, The synchronization means corrects the first time information by adding the delay time calculated by the measurement means, according to the signal processing apparatus described in any one of Configurations 1 to 5, which is characterized in that. (Configuration 7) The acoustic signal is a plurality of acoustic signals collected by a plurality of sound collection means, It has correction means for correcting the delay time for each of the plurality of acoustic signals signal-processed by the signal processing means based on the delay time measured by the measurement means, The synchronization means synchronizes the time information output from the timing means and the acoustic signal corrected by the correction means, according to the signal processing apparatus described in any one of Configurations 1 to 6, which is characterized in that. (Configuration 8) The correction means corrects the delay time of the acoustic signal signal-processed by the signal processing means based on the largest delay time among the delay times measured by the measurement means, according to the signal processing apparatus described in Configuration 7, which is characterized in that. (Configuration 9) The correction means corrects the delay time of the acoustic signal signal-processed by the signal processing means so that the largest delay time among the delay times measured by the measurement means is obtained, according to the signal processing apparatus described in Configuration 8, which is characterized in that. (Method 1) A signal processing step of signal-processing an acoustic signal, A measuring step of measuring a delay time due to signal processing in the signal processing step based on the first time information output from the timing means and the second time information output from the timing means through the signal processing step; A control method for a signal processing apparatus, comprising: a synchronization step of synchronizing the time information output from the timing means and the acoustic signal signal-processed in the signal processing step based on the delay time measured in the measuring step. (Program 1) A program for causing a computer of a signal processing apparatus to execute a signal processing step of signal-processing an acoustic signal, a measuring step of measuring a delay time due to signal processing in the signal processing step based on the first time information output from the timing means and the second time information output from the timing means through the signal processing step, and a synchronization step of synchronizing the time information output from the timing means and the acoustic signal signal-processed in the signal processing step based on the delay time measured in the measuring step.

Explanation of Signs

[0081] 100, 600: Signal processing apparatus 110: Time code generation unit 120: Sound collection unit 130: Signal processing unit 140, 610: Synchronization unit 141, 611: Control unit 142, 612: Measurement unit 143, 613: Time code correction unit 150: Recording unit 614: Delay correction unit< / s>

Claims

1. signal processing means for processing an acoustic signal; measuring means for measuring a delay time due to signal processing in the signal processing means based on first time information output from the timing means and second time information output from the signal processing means; A signal processing apparatus, comprising: synchronization means for synchronizing the time information output from the timing means and the acoustic signal signal-processed by the signal processing means based on the delay time measured by the measuring means.

2. The synchronization means outputs the acoustic signal cut out based on third time information synchronized by the synchronization means as acoustic data, and outputs the third time information and the acoustic data; The signal processing apparatus according to claim 1, further comprising recording means for recording the third time information and the acoustic data output from the synchronization means.

3. The signal processing apparatus according to claim 2, wherein the synchronization means cuts out the acoustic signal in a predetermined unit corresponding to the third time information to obtain the acoustic data.

4. The signal processing apparatus according to claim 2, wherein the synchronization means includes time information in a predetermined channel of the acoustic signal and outputs the third time information and the acoustic data.

5. The signal processing apparatus according to claim 1, further comprising control means for controlling whether to perform a process of measuring the delay time by the measuring means or a process of synchronizing time information and an acoustic signal by the synchronization means.

6. The measuring means calculates the delay time by subtracting the first time information from the second time information; The signal processing apparatus according to claim 1, wherein the synchronization means corrects the first time information by adding the delay time calculated by the measuring means.

7. The acoustic signal is a plurality of acoustic signals picked up by a plurality of sound pickup means, correction means for correcting the delay time for each of the plurality of acoustic signals signal-processed by the signal processing means based on the delay time measured by the measuring means; The signal processing apparatus according to claim 1, wherein the synchronization means synchronizes the time information output from the timing means and the acoustic signal corrected by the correction means.

8. The signal processing apparatus according to claim 7, wherein the correction means corrects the delay time of the acoustic signal signal-processed by the signal processing means based on the largest delay time among the delay times measured by the measurement means.

9. The signal processing apparatus according to claim 8, wherein the correction means corrects the delay time of the acoustic signal signal-processed by the signal processing means so as to be the largest delay time among the delay times measured by the measurement means.

10. A signal processing step of signal-processing an acoustic signal; A measurement step of measuring a delay time due to signal processing in the signal processing step based on first time information output from a timekeeping means and second time information output from the timekeeping means after passing through the signal processing step; A control method for a signal processing apparatus, comprising: a synchronization step of synchronizing the time information output from the timekeeping means and the acoustic signal signal-processed in the signal processing step based on the delay time measured in the measurement step.

11. A program for causing a computer of a signal processing apparatus to execute a signal processing step of signal-processing an acoustic signal; a measurement step of measuring a delay time due to signal processing in the signal processing step based on first time information output from a timekeeping means and second time information output from the timekeeping means after passing through the signal processing step; and a synchronization step of synchronizing the time information output from the timekeeping means and the acoustic signal signal-processed in the signal processing step based on the delay time measured in the measurement step.

Images