Data processing device, data processing method and program

Abstract

To provide a data processing device, a data processing method, and a program that enable AGC of sound data while preventing an analysis accuracy of the sound data from degrading.SOLUTION: The data processing device includes: a sound data copy unit 111 that generates sound data 107 obtained by copying sound data 106 input through a microphone 102; a gain processing unit 112 that outputs the sound data 106 multiplied by an AGC gain, and outputs the sound data 107 not multiplied by the AGC gain; a sound data distribution processing unit 104 that distributes the sound data 106 multiplied by the AGC gain; and a sound data analysis processing unit 105 that analyzes the sound data 107 not multiplied by the AGC gain.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data processing device, a data processing method, and a program. [Background technology]

[0002] Cameras that are controlled via a network, dedicated line, or remote control may be equipped with an audio input function. The audio input function has an AGC (Auto Gain Control) function that automatically adjusts the gain so that the audio level is always at an appropriate volume. With this AGC function, the gain decreases when a loud sound is input, and increases when a quiet sound is input. Cameras equipped with voice input functionality have several built-in voice-based analysis functions known as voice recognition functions (such as raising an event if the volume exceeds a set value). Voice-based analysis functions may not function properly if the gain fluctuates due to the AGC function.

[0003] Patent Document 1 discloses a technology that estimates the noise emitted by the engine and its frequency from the engine speed, and applies a bandpass filter to reduce the noise, in order to prevent the engine noise, which is background noise, from being amplified by the AGC function.

[0004] Patent Document 2 discloses a technique for turning off AGC when an application that uses voice recognition on an amplified audio signal is executed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 5-336590 [Patent Document 2] Patent No. 5817368 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the technology disclosed in Patent Document 1, when a band-pass filter is applied to reduce engine noise, the level of the sound that is originally intended to be detected in the same frequency band as the noise also decreases. As a result, if there is a sound that is intended to be detected in the same frequency band as the noise, the detection performance of the sound analysis decreases. In the technology disclosed in Patent Document 2, the AGC function is turned off when an application that uses voice recognition is executed, so the volume of the audio distribution may be too loud and cause saturation, or too quiet and inaudible. The problem to be solved by the present invention is to enable AGC of sound data while suppressing a decrease in the accuracy of analysis of the sound data. [Means for solving the problem]

[0007] A data processing device according to one aspect includes: AGC (Auto Gain Control) gain is applied to the first sound data, and a negative gain that cancels out the AGC gain is applied to the second sound data, which is a copy of the first sound data after the AGC gain is applied. a processing means for applying the a distribution means for distributing the first sound data to which the AGC gain has been applied by the processing means; and an analysis means for analyzing the second sound data to which the negative gain has been applied by the processing means. Equipped with. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to perform AGC on sound data while suppressing a decrease in the accuracy of analysis of the sound data. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the schematic configuration of a data processing device according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the schematic configuration of an imaging apparatus according to a first embodiment. [Figure 3] FIG. 1 is a block diagram showing an example of the arrangement of a data processing device according to a first embodiment. [Figure 4] 10 is a flowchart showing a sound data copy process according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing an example of the schematic configuration of a data processing device according to a second embodiment. [Figure 6]FIG. 11 is a diagram showing an example of gain setting in the data processing device according to the third embodiment. [Figure 7] FIG. 10 is a block diagram showing an example of the hardware configuration of a data processing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention, and not all combinations of features described in the embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). The technical scope of the present invention is determined by the claims and is not limited by the individual embodiments below.

[0011] First Embodiment FIG. 1 is a block diagram showing an example of a schematic configuration of a data processing device according to the first embodiment. For the software-implemented functions of the functional blocks shown in FIG. 1, a program for providing the function of each functional block is stored in a memory such as a read-only memory (ROM). The program is then loaded into a random-access memory (RAM) and executed by a central processing unit (CPU). For the hardware-implemented functions, a dedicated circuit may be automatically generated on an FPGA from a program for implementing the function of each functional block using, for example, a predetermined compiler. FPGA stands for Field Programmable Gate Array. Alternatively, a gate array circuit may be formed in a similar manner to an FPGA to implement the function as hardware. Alternatively, the function may be implemented using an application-specific integrated circuit (ASIC). Note that the functional block configuration shown in FIG. 1 is merely an example; multiple functional blocks may constitute a single functional block, or one functional block may be divided into blocks that perform multiple functions.

[0012] 1, the data processing device includes a sound data processing unit 101, a sound data distribution processing unit 104, and a sound data analysis processing unit 105. The sound data processing unit 101 includes a sound data copy unit 111 and a gain processing unit 112. The gain processing unit 112 includes an AGC unit 103.

[0013] The sound data processing unit 101 processes sound data input via the microphone 102 to generate two pieces of sound data with different volumes for the same input sound, and outputs the two pieces of sound data to the sound data distribution processing unit 104 and the sound data analysis processing unit 105. At this time, the sound data processing unit 101 can apply a gain to at least one of the two pieces of sound data. For example, the sound data processing unit 101 generates sound data to which an AGC gain has been applied and sound data to which an AGC gain has not been applied. The sound data processing unit 101 can then output the sound data to which an AGC gain has been applied to the sound data distribution processing unit 104, and output the sound data to which an AGC gain has not been applied to the sound data analysis processing unit 105.

[0014] The sound data copy unit 111 copies the sound data 106 input via the microphone 102 to generate sound data 107 . The gain processing unit 112 applies an AGC gain to the sound data 106 and outputs the data, and also applies no AGC gain to the sound data 107 and outputs the data. The AGC unit 103 applies an AGC gain to the sound data 106.

[0015] The sound data distribution processing unit 104 distributes the sound data 106 to which the AGC gain has been applied. A network or a dedicated line may be used to distribute the sound data 106 to which the AGC gain has been applied. The destination of the sound data 106 to which the AGC gain has been applied is, for example, an information processing device connected to the imaging device.

[0016] The sound data analysis processing unit 105 analyzes sound data 107 to which no AGC gain has been applied. The analysis processing of the sound data 107 may include a recognition processing of the sound data 107, or may include a frequency analysis processing of the sound data 107. The recognition processing of the sound data 107 may include a recognition processing of an abnormal sound such as the sound of glass breaking, for example.

[0017] Here, the gain processing unit 112 applies an AGC gain to the sound data 106 output to the sound data distribution processing unit 104, thereby preventing the volume of the sound data when distributed from being too loud and causing saturation, or too quiet and inaudible. Furthermore, by not applying AGC gain to the sound data 107 output to the sound data analysis processing unit 105, the gain processing unit 112 can prevent the data to be analyzed from being suppressed, thereby suppressing a decrease in the analysis accuracy of the sound data 107. At this time, the sound data copying unit 111 copies the sound data 106 before the AGC gain is applied, thereby suppressing a decrease in the analysis accuracy of the sound picked up by the microphone 102 and realizing distribution at an optimized volume.

[0018] Fig. 2 is a block diagram showing a schematic configuration example of an imaging device according to the first embodiment. In Fig. 2, a pan-tilt camera with a microphone is taken as an example of the imaging device. This imaging device can be used as a surveillance camera, for example.

[0019] 2, camera 200 includes an imaging unit 201, a pan / tilt driving unit 202, a microphone 203, an arithmetic processing unit 204, a pan / tilt control unit 205, and a system control unit 207. Camera 200 is connected to a client device 208 in a state in which they can communicate with each other. Client device 208 is, for example, an information processing device such as a personal computer. A user can send various commands to camera 200 from client device 208.

[0020] The imaging unit 201 captures an image based on light from a subject. At this time, the imaging unit 201 converts the light collected on the imaging surface into an electrical signal for each pixel and outputs the signal to the arithmetic processing unit 204. The imaging unit 201 includes imaging lenses including a focus lens, a zoom lens, etc., an imaging element, a mechanical drive system and a drive circuit for driving these elements, etc. The imaging element is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0021] The pan / tilt driving unit 202 performs pan (horizontal rotation) driving and tilt (vertical rotation) driving of the camera 200. The pan / tilt driving unit 202 includes a mechanical driving system for performing pan / tilt operations, a motor and a motor driver as a driving source, and the like.

[0022] The arithmetic processing unit 204 performs image processing such as noise removal and gamma correction on the electrical signals converted by the imaging unit 201 to generate image data and transmits the image data to the system control unit 207. The arithmetic processing unit 204 also processes commands received from the system control unit 207. For example, when the arithmetic processing unit 204 receives an instruction to change the zoom position or focus position from the system control unit 207, it drives the focus lens or zoom lens to the instructed position. When the arithmetic processing unit 204 receives an instruction to adjust image quality from the system control unit 207, it adjusts the image quality. The arithmetic processing unit 204 also performs calculations related to pan / tilt position information to be transmitted to the pan / tilt control unit 205.

[0023] Furthermore, the arithmetic processing unit 204 performs calculations on the sound data input via the microphone 203 and performs sound data analysis processing. At this time, the arithmetic processing unit 204 can copy the sound data input via the microphone 203 and generate sound data with AGC gain applied and sound data without AGC gain applied. Then, the arithmetic processing unit 204 can distribute the sound data with AGC gain applied and analyze the sound data without AGC gain applied.

[0024] The pan / tilt control unit 205 processes commands related to pan / tilt control received by the arithmetic processing unit 204 via the system control unit 207, and controls the pan / tilt driving unit 202. For example, the pan / tilt control unit 205 controls the driving amount, speed, and acceleration / deceleration of the pan / tilt driving unit 202, and performs initialization operations of the pan / tilt driving unit 202, based on instructions of the commands related to pan / tilt control.

[0025] The system control unit 207 controls the entire pan-tilt camera 200. For example, the system control unit 207 distributes image data generated by the arithmetic processing unit 204 to the client device 208. The system control unit 207 also analyzes camera control commands sent from the client device 208 and transmits commands related to the arithmetic processing unit 204 to the imaging unit 201. The system control unit 207 also transmits responses to the camera control commands to the client device 208.

[0026] Furthermore, the system control unit 207 distributes the sound data output from the calculation processing unit 204 to the client device 208. At this time, the system control unit 207 may distribute the sound data collected by the microphone 203 when the imaging unit 201 captures an image together with the image data to the client device 208. Alternatively, the system control unit 207 may distribute the sound data collected by the microphone 203 alone to the client device 208. The system control unit 207 may also notify the client device 208 of the occurrence of an event detected based on analysis processing of sound data without AGC gain. For example, when the sound of glass breaking is detected based on analysis processing of sound data without AGC gain, the system control unit 207 may notify the client device 208 of the occurrence of an event.

[0027] It should be noted that camera 200 in this embodiment is not limited to the configuration shown in Fig. 2. For example, camera 200 may include a video output terminal such as an SDI (Serial Digital Interface) or an HDMI (High-Definition Multimedia Interface) (registered trademark). Camera 200 may also include an audio data input / output unit or an external device input / output unit.

[0028] FIG. 3 is a block diagram showing an example of the configuration of the data processing device according to the first embodiment. 3, the arithmetic processing unit 204 in Fig. 2 includes an AD conversion unit 302, a filter processing unit 303, a PCM (Pulse Code Modulation) conversion unit 304, an AGC unit 305, a sound data compression unit 306, and a sound data distribution processing unit 307. The arithmetic processing unit 204 also includes a sound data copy unit 308 and a sound data analysis processing unit 308.

[0029] In the AD conversion unit 302, the sound data acquired by the microphone 301 is converted from an analog signal to a digital signal. Next, in the filter processing unit 303, unnecessary high-frequency components and low-frequency components are cut from the sound data converted into a digital signal. Next, in the PCM conversion unit 304 , the sound data output from the filter processing unit 303 is converted into a PCM signal, and the converted sound data P 1 is output to the AGC unit 305 and the sound data copy unit 308 .

[0030] Next, the AGC unit 305 applies an AGC gain to the sound data P1, and generates sound data P3 in which the volume of the sound data P1 is optimized. Next, in the sound data compression unit 306, the sound data P3 to which the AGC gain has been applied is subjected to data compression to ensure a bandwidth for distribution, and is distributed via the sound data distribution processing unit 307.

[0031] Meanwhile, sound data P2 is generated by copying the sound data P1 converted into a PCM signal in sound data copy unit 308. Then, sound data P2 copied by sound data copy unit 308 is sent to sound data analysis processing unit 308 without AGC gain being applied, and analysis processing is performed thereon.

[0032] FIG. 4 is a flowchart showing the sound data copy process according to the first embodiment. Each step in FIG. 4 is realized by the arithmetic processing unit 204 reading and executing a program stored in the storage unit of the imaging device 200 in FIG. 2. At least a part of the flowchart shown in FIG. 4 may be realized by hardware. When realized by hardware, for example, a predetermined compiler may be used to automatically generate a dedicated circuit on an FPGA from a program for realizing each step. Also, a gate array circuit may be formed in the same manner as an FPGA, and realized as hardware. Also, it may be realized by an ASIC. In this case, each block in the flowchart shown in Fig. 4 can be regarded as a hardware block. Note that multiple blocks may be combined into one hardware block, or one block may be configured as multiple hardware blocks.

[0033] 4, the arithmetic processing unit 204 determines whether or not to perform sound data analysis processing (step S41). If the arithmetic processing unit 204 does not perform sound data analysis processing, it terminates the processing. On the other hand, if the arithmetic processing unit 204 performs sound data analysis processing, it determines whether or not the AGC function is ON (step S42). If the AGC function is OFF, the arithmetic processing unit 204 terminates the processing. On the other hand, if the AGC function is ON, the arithmetic processing unit 204 performs sound data copy processing (step S43).

[0034] 4 is preferably applied when the sound data copy process is performed on hardware in order to reduce power consumption. If software processing or the like is used and there is little concern about increased power consumption, the arithmetic processing unit 204 may constantly perform the copy process of the sound data input via the microphone 203.

[0035] Second Embodiment FIG. 5 is a block diagram showing an example of a schematic configuration of a data processing device according to the second embodiment. 5, this data processing device includes an audio data copying unit 501 and a gain control unit 502 instead of the audio data copying unit 308 in FIG.

[0036] The sound data copy unit 501 copies sound data input via the microphone 301. At this time, the sound data copy unit 501 may copy sound data P1 before the AGC gain is applied by the AGC unit 305, or may copy sound data P3 after the AGC gain is applied by the AGC unit 305.

[0037] The gain control unit 502 can apply a gain different from the AGC gain applied by the AGC unit 305 to the sound data copied by the sound data copy unit 501. For example, the gain control unit 502 may store the AGC gain used during calibration and apply the AGC gain used during calibration to the sound data copied by the sound data copy unit 501. During the calibration period when internal settings for the sound data analysis process are performed, the sound data P3 after the gain has been applied by the AGC unit 305 is copied and used for the sound data analysis process. At this time, the gain control unit 502 saves the AGC gain used during calibration and applies the same AGC gain after the calibration. In this case, the gain control unit 502 uses the sound data P1 before the gain has been applied by the AGC unit 305 to copy the sound data.

[0038] This allows the gain control unit 502 to apply a constant gain stored during the calibration period to the sound data used in the data analysis process data, thereby optimizing the volume of the sound data used in the data analysis process and preventing a decrease in the accuracy of the sound data analysis process.

[0039] Furthermore, the gain control unit 502 may also use the sound data P3 after the calibration period after the gain has been applied by the AGC unit 305. At this time, in order to cancel the AGC gain applied by the AGC unit 305, the gain control unit 502 may apply a negative gain, the sign of which is opposite to that of the AGC gain, to the sound data P3.

[0040] This allows the sound data analysis processing unit 308 to analyze sound data with a fixed gain, even in a system in which sound data P1 before gain is applied by the AGC unit 305 cannot be copied and sound data P3 to which gain is applied by the AGC unit 305 must be copied.

[0041] Furthermore, the gain control unit 502 may change the detection threshold for the sound data P3 in accordance with the change in gain for the sound data P3 after the gain has been applied by the AGC unit 305. For example, when the sound data analysis processing unit 308 detects that the volume of the sound data P3 is equal to or greater than a certain level, the gain control unit 502 can lower the detection threshold for the volume by the amount of the gain applied by the AGC unit 305.

[0042] This allows the sound data analysis processing unit 308 to perform sound data analysis while adapting to the gain applied by the AGC unit 305, even when the analysis target changes and it is desirable to lower the detection level of the sound data analysis.

[0043] <Third embodiment> FIG. 6 is a diagram showing an example of gain setting in the data processing device according to the third embodiment. 6, the gain control unit 502 of FIG. 5 applies a negative gain to sound data during operation of the internal drive unit in accordance with the internal drive unit of the camera 200 of FIG. 2. The internal drive units of the camera 200 are, for example, a pan drive unit (Pan), a tilt drive unit (Tilt), and a zoom unit (Zoom) and focus unit (Focus) of the imaging unit 201. A negative gain can be set for each of these internal drive units. The negative gain can be set according to the volume of the drive sound emitted by the internal drive unit during operation.

[0044] This makes it possible to reduce the drive noise superimposed on the sound data used in the analysis process even when drive noise is generated while the internal drive unit is operating, thereby preventing malfunction of the sound data analysis process due to internal noise of the camera 200.

[0045] <Fourth embodiment> FIG. 7 is a block diagram showing an example of the hardware configuration of a data processing device according to the fourth embodiment. 7, data processing device 10 can realize, on camera 200, functions that are realized by software among the functions of camera 200 in FIG. 2. In this case, data processing device 10 can realize the functions of arithmetic processing unit 204 in FIG.

[0046] The data processing device 10 includes a processor 11, a communication control unit 12, a communication interface 13, a main memory unit 14, an auxiliary memory unit 15, and an input / output interface 17. The processor 11, the communication control unit 12, the communication interface 13, the main memory unit 14, the auxiliary memory unit 15, and the input / output interface 17 are interconnected via an internal bus 16. The main memory unit 14 and the auxiliary memory unit 15 are accessible from the processor 11.

[0047] In addition, an image sensor 20, a microphone 21, and a driving unit 22 are provided outside the data processing device 10. The image sensor 20, the microphone 21, and the driving unit 22 are connected to the internal bus 16 via the input / output interface 17. The image sensor 20 is, for example, a CCD sensor or a CMOS sensor. The microphone 21 is, for example, the microphone 203 in FIG. 2. The driving unit 22 is, for example, the pan / tilt driving unit 205, zoom unit, and focus unit of the imaging unit 201 in FIG. 2.

[0048] The processor 11 controls the overall operation of the data processing device 10. The processor 11 may be a CPU or a GPU (Graphics Processing Unit). The processor 11 may be a single-core processor or a multi-core processor. The processor 11 may include a hardware circuit (for example, an FPGA or an ASIC) such as an accelerator that speeds up part of the processing.

[0049] The main memory unit 14 can be configured, for example, from a semiconductor memory such as an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory). The main memory unit 14 can store programs being executed by the processor 11 and can provide a work area for the processor 11 to execute programs.

[0050] The auxiliary storage unit 15 is a non-volatile storage device, such as a ROM, a hard disk drive, or an SSD (Solid State Drive). The auxiliary storage unit 15 can store executable files for various programs and data used to execute the programs. For example, the auxiliary storage unit 15 can store a data processing program 15A. The data processing program 15A may be software that can be installed in the camera 200, or may be incorporated into the camera 200 as firmware.

[0051] The communication control unit 12 is hardware having a function of controlling communication with the outside world. The communication control unit 12 is connected to a network 19 via a communication interface 13. The network 19 may be the Internet, a WAN, a LAN such as WiFi or Ethernet, or a combination of the Internet, a WAN, and a LAN.

[0052] The input / output interface 17 converts data input from the image sensor 20, the microphone 21, and the driving unit 22 into a data format that can be processed by the processor 11. The input / output interface 17 also converts data output from the processor 11 into a data format that can be processed by the image sensor 20 and the driving unit 22.

[0053] The processor 11 can realize sound data copy processing, sound data gain processing, and sound data analysis processing by reading out the data processing program 15A stored in the auxiliary storage unit 15 into the main storage unit 14 and executing it.

[0054] The execution of the programs for realizing the sound data copy processing, sound data gain processing, and sound data analysis processing may be shared among multiple processors or computers. Alternatively, processor 11 may instruct a cloud computer or the like via network 19 to execute all or part of the programs for realizing the sound data copy processing, sound data gain processing, and sound data analysis processing, and receive the execution results.

[0055] <Other embodiments> In the present invention, a program that realizes one or more functions of the above-described embodiments may be supplied to a system or device via a network or a storage medium. One or more functions of the above-described embodiments may also be realized by one or more processors in a computer of the system or device reading and executing the program. Alternatively, one or more functions may also be realized by a circuit (e.g., an FPGA or ASIC) that realizes one or more functions. While preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and variations are possible within the scope of the spirit thereof. [Explanation of symbols]

[0056] 101 sound data processing unit, 102 microphone, 103 AGC unit, 104 sound data distribution processing unit, 105 sound data analysis processing unit, 111 sound data copy unit, 112 gain processing unit

Claims

1. A processing means for applying an AGC (Auto Gain Control) gain to first sound data, and for applying a negative gain to second sound data, which is a copy of the first sound data after the AGC gain has been applied, to cancel out the AGC gain; a distribution means for distributing the first sound data to which the AGC gain has been applied by the processing means, and an analysis means for analyzing the second sound data to which the negative gain has been applied by the processing means. A data processing device characterized by:

2. 2. The data processing apparatus according to claim 1, wherein said processing means stores an AGC gain at the time of calibration of said analyzing means, and multiplies said second sound data used by said analyzing means by the AGC gain at the time of calibration.

3. the second sound data is the first sound data copied after the AGC gain is applied, 2. The data processing apparatus according to claim 1, wherein said processing means changes a detection threshold for said first sound data in response to a change in an AGC gain applied to said first sound data.

4. The sound data output from the microphone is input to the imaging device, 2. The data processing device according to claim 1, wherein the processing means applies a negative gain to the second sound data while the driving section is driving in accordance with the driving section of the imaging device.

5. A step of applying an AGC (Auto Gain Control) gain to first sound data, and applying a negative gain to second sound data, which is a copy of the first sound data after the AGC gain has been applied, to cancel out the AGC gain; Distributing the first sound data to which the AGC gain has been applied; analyzing the second sound data to which the negative gain has been applied; A data processing method comprising:

6. A program for causing a computer to operate as the data processing device according to any one of claims 1 to 4.

Images