Imaging apparatus, sound pickup apparatus, and imaging system

The imaging system combines two amplification conversions to generate audio data in a float format, addressing the challenge of accurately representing sound collection results, enhancing video shooting by simplifying the recording process and reducing distortion.

JP2024158944APending Publication Date: 2024-11-08PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023074596
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing imaging systems struggle to easily obtain audio data in a predetermined data format that accurately represents sound collection results, requiring complex configurations and manual adjustments during video shooting.

Method used

An imaging system comprising a digital camera and a sound collection device that performs two types of amplification conversions on audio signals, which are then combined and processed to generate audio data in a float format, ensuring a wide dynamic range and high precision without the need for additional arithmetic circuits in the sound collection device.

Benefits of technology

The system enables easy and accurate capture of audio data in a float format, simplifying the recording process and reducing the risk of sound distortion or volume issues during video shooting, allowing users to focus on composition without manual adjustments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158944000001_ABST
    Figure 2024158944000001_ABST
Patent Text Reader

Abstract

To provide an imaging apparatus, a sound pickup apparatus, and an imaging system capable of making sound data showing a sound pickup result in a predetermined data format be easily obtained using the imaging apparatus and the sound pickup apparatus.SOLUTION: An imaging apparatus (100) comprises: an imaging section (115) for imaging a subject image to create image data; a voice processing section (170) for creating voice data to be associated with the image data; and a reception section (160) for receiving a first voice signal (A2) and a second voice signal (A3) from an external sound pickup apparatus (200). The first voice signal shows a result of first amplification conversion performed on input voice by the sound pickup apparatus. The second voice signal shows a result of second amplification conversion, which is different from the first amplification conversion, performed on the input voice by the sound pickup apparatus. The voice processing section takes, as input, the first and second voice signals received from the reception section, performs processing of combining the signals each other in a predetermined data format, and creates voice data (A10) on a sound pickup result showing the input voice in the predetermined data format.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an imaging device that receives an audio signal from a sound collecting device, a sound collecting device, and an imaging system including these. [Background technology]

[0002] Patent Document 1 discloses a data processing device that performs arithmetic processing of floating-point data suitable for processing audio signals. This data processing device is provided with a mantissa register and an exponent register that store the mantissa and exponent of the floating-point data, respectively. The data processing device performs arithmetic processing of audio signals, which are fixed-point data for two channels (left and right), to generate floating-point data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 63-282800 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an imaging device, a sound collection device, and an imaging system that can easily obtain audio data indicating the sound collection results in a predetermined data format using a sound collection device and an imaging device. [Means for solving the problem]

[0005] The imaging device according to the present disclosure includes an imaging unit that captures an image of a subject and generates image data, an audio processing unit that generates audio data associated with the image data, and a receiving unit that receives a first audio signal and a second audio signal from an external sound collection device. The first audio signal indicates the result of a first amplification conversion performed on the input audio in the sound collection device. The second audio signal indicates the result of a second amplification conversion performed on the input audio in the sound collection device, the second amplification conversion being different from the first amplification conversion. The audio processing unit performs a process of combining the first and second audio signals received from the receiving unit as inputs, and generates audio data of the sound collection result indicating the input audio in a predetermined data format.

[0006] The sound collection device in the present disclosure is a sound collection device that transmits a sound signal to an imaging device. The sound collection device includes a sound input unit that acquires an input sound, a first signal processing unit that performs a first amplification conversion on the input sound to generate a first sound signal, a second signal processing unit that performs a second amplification conversion different from the first amplification conversion on the input sound to generate a second sound signal, and a transmission unit that transmits the first sound signal and the second sound signal to the imaging device.

[0007] The imaging system of the present disclosure includes the above-mentioned sound collection device, and an imaging device that receives first and second audio signals from the sound collection device, performs a process of combining the received first and second audio signals as inputs, and generates audio data of the sound collection result indicating the input audio in a predetermined data format. Effect of the Invention

[0008] According to the imaging device, sound collection device, and imaging system of the present disclosure, it is possible to easily obtain audio data indicating the sound collection results in a predetermined data format using the sound collection device and imaging device. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining an imaging system according to a first embodiment of the present disclosure. [Diagram 2] FIG. 1 shows a configuration of a digital camera in an imaging system. [Diagram 3] FIG. 1 is a diagram showing a configuration of a sound collecting device in an imaging system. [Figure 4] A diagram showing the circuit configuration for float recording in an imaging system. [Diagram 5] Waveform diagram for explaining the float recording operation in the imaging system [Figure 6] Diagram for explaining the data structure of float format [Figure 7] FIG. 1 is a sequence diagram illustrating a communication operation in an imaging system; [Figure 8] FIG. 1 is a diagram showing an example of a structure of a communication packet in an imaging system; [Figure 9] FIG. 13 is a diagram showing a display example of a digital camera according to a modified example; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings as appropriate. However, detailed descriptions of already well-known matters and duplicate descriptions of substantially identical configurations may be omitted. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0011] (Embodiment 1) 1. Configuration An imaging system according to a first embodiment of the present disclosure will be described with reference to FIG.

[0012] 1-1. System Overview 1, an imaging system 10 according to this embodiment includes a digital camera 100 and a sound collection device 200. In this system 10, the digital camera 100 and the sound collection device 200 are connected by wireless communication such as Bluetooth.

[0013] The present system 10 realizes float recording using a sound collection device 200 when, for example, shooting a video with a digital camera 100. Float recording is a recording function that can ensure the resolution of each sound from relatively loud sounds to quiet sounds by using a predetermined data format such as the float format.

[0014] In realizing such float recording, the imaging system 10 of this embodiment utilizes the calculation function of the digital camera 100 and adopts a simple configuration for the sound collection device 200. With this system 10, for example, a user of the digital camera 100 can easily obtain high dynamic range and high accuracy sound recording results in video shooting by preparing the sound collection device 200 with a simple configuration.

[0015] The configurations of the digital camera 100 and the sound collecting device 200 in the system 10 will be described below.

[0016] 1-2. Digital camera configuration The configuration of the digital camera 100 according to this embodiment will be described with reference to FIG.

[0017] 2 is a diagram showing the configuration of a digital camera 100 according to this embodiment. The digital camera 100 of this embodiment includes an image sensor 115, an image processing engine 120, a display monitor 130, and a control unit 135. The digital camera 100 also includes a buffer memory 125, a card slot 140, a flash memory 145, an operation unit 150, a communication module 160, and an audio processing engine 170. The digital camera 100 also includes, for example, an optical system 110 and a lens driving unit 112.

[0018] The optical system 110 includes a focus lens, a zoom lens, an optical image stabilization lens (OIS), an aperture, a shutter, etc. The focus lens is a lens for changing the focus state of a subject image formed on the image sensor 115. The zoom lens is a lens for changing the magnification of a subject image formed by the optical system. The focus lens, etc. are each composed of one or more lenses.

[0019] Lens driving unit 112 drives a focus lens and the like in optical system 110. Lens driving unit 112 includes a motor, and moves the focus lens along the optical axis of optical system 110 under the control of control unit 135. The configuration for driving the focus lens in lens driving unit 112 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.

[0020] The image sensor 115 captures an image of a subject formed via the optical system 110 and generates imaging data. The imaging data constitutes image data indicative of an image captured by the image sensor 115. The image sensor 115 generates new frame image data at a predetermined frame rate (e.g., 30 frames / second). The timing of generating imaging data and the electronic shutter operation in the image sensor 115 are controlled by the control unit 135. The image sensor 115 can be any of various image sensors, such as a CMOS image sensor, a CCD image sensor, or an NMOS image sensor.

[0021] The image sensor 115 performs imaging operations of moving images, still images, through images, etc. The through images are mainly moving images, and are displayed on the display monitor 130 so that the user can determine, for example, a composition for capturing a still image. The through images, moving images, and still images are each an example of a captured image in this embodiment. The image sensor 115 is an example of an imaging section in this embodiment.

[0022] The image processing engine 120 performs various processes on the imaging data output from the image sensor 115 to generate image data, and performs various processes on the image data to generate an image to be displayed on the display monitor 130. The various processes include, but are not limited to, white balance correction, gamma correction, YC conversion processing, electronic zoom processing, compression processing, and expansion processing. The image processing engine 120 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer, processor, etc. using a program.

[0023] The display monitor 130 is an example of a display unit that displays various information. For example, the display monitor 130 displays an image (through image) represented by image data captured by the image sensor 115 and processed by the image processing engine 120. The display monitor 130 also displays a menu screen or the like that allows the user to perform various settings on the digital camera 100. The display monitor 130 can be configured, for example, by a liquid crystal display device or an organic EL device.

[0024] Operation unit 150 is a general term for hard keys such as operation buttons and operation levers provided on the exterior of digital camera 100, and accepts operations by the user. Operation unit 150 includes, for example, a release button, a mode dial, a touch panel, a cursor button, and a joystick. When operation unit 150 accepts an operation by the user, it transmits an operation signal corresponding to the user operation to control unit 135.

[0025] The control unit 135 centralizes and controls the overall operation of the digital camera 100. The control unit 135 includes a CPU and the like, and the CPU executes a program (software) to realize a predetermined function. For example, the control unit 135 functions as a decoder 165 that decodes a signal received from the communication module 160.

[0026] The decoder 165 does not have to be realized by the function of the control unit 135, and may be incorporated in the communication module 160, for example. The control unit 135 may include a processor configured with a dedicated electronic circuit designed to realize a predetermined function, instead of a CPU. That is, the control unit 135 can be realized by various processors such as a CPU, an MPU, a GPU, a DSU, an FPGA, and an ASIC. The control unit 135 may be configured with one or more processors. Also, the control unit 135 may be configured with the image processing engine 120 and the like as a single semiconductor chip.

[0027] The buffer memory 125 is a recording medium that functions as a work memory for the image processing engine 120 and the control unit 135. The buffer memory 125 is realized by a dynamic random access memory (DRAM) or the like. The flash memory 145 is a non-volatile recording medium. Although not shown, the control unit 135 may have various types of internal memory, for example, a built-in ROM. The ROM stores various programs executed by the control unit 135. The control unit 135 may also have a built-in RAM that functions as a work area for the CPU.

[0028] The card slot 140 is a means for inserting a removable memory card 142. The card slot 140 is capable of electrically and mechanically connecting the memory card 142. The memory card 142 is an external memory equipped with an internal recording element such as a flash memory. The memory card 142 can store data such as image data generated by the image processing engine 120.

[0029] The communication module 160 is a module (circuit) that connects to an external device such as the sound collection device 200 in accordance with a predetermined communication standard such as Bluetooth Low Energy (BLE). For example, the communication module 160 performs wireless communication of an audio signal in accordance with the LE Audio standard. The communication by the communication module 160 is not limited to wireless communication and may be wired communication. Furthermore, the communication standard of the communication module 160 is not particularly limited and may be, for example, USB, HDMI (registered trademark), IEEE802.11, Wi-Fi, etc. The communication module 160 is an example of a receiving unit of the digital camera 100 in this embodiment, and may be an example of a transmitting unit or a communication unit of the digital camera 100.

[0030] The audio processing engine 170 performs various types of audio processing on, for example, an audio signal acquired from outside the digital camera 100, and generates audio data as a processing result. The audio processing engine 170 is an example of the audio processing unit in this embodiment.

[0031] 2, the audio processing engine 170 of this embodiment includes a data conversion unit 172, an amplification unit 174, and a combination unit 176. Each of the units 172 to 176 is a functional configuration that performs arithmetic processing to realize float recording (details will be described later). The audio processing engine 170 may be configured integrally with either or both of the image processing engine 120 and the control unit 135.

[0032] The digital camera 100 may include a built-in microphone, and audio processing engine 170 may process audio picked up by the microphone. The digital camera 100 may also include an input terminal for connecting an external microphone, a signal processing circuit, and the like.

[0033] 1-3. Configuration of the sound pickup device The configuration of the sound collection device 200 in this embodiment will be described with reference to FIGS.

[0034] Fig. 3 shows the configuration of the sound collection device 200 in this system. Fig. 4 shows the circuit configuration for float recording in this system 10.

[0035] 3, the sound collection device 200 of this embodiment includes an audio input unit 210, a plurality of signal processing units 220 and 230, a control unit 240, a storage unit 250, and a communication unit 260. The sound collection device 200 is a device that collects sound for float recording using, for example, an external microphone.

[0036] The audio input unit 210 includes an input terminal for connecting, for example, one or more external microphones. The audio input unit 210 inputs an analog signal representing audio picked up by, for example, one monaural microphone to the sound collection device 200, thereby acquiring one channel of input audio.

[0037] The audio input unit 210 is connected to two signal processing units 220 and 230 in parallel for one channel of input audio. The audio input unit 210 may acquire input audio of multiple channels, for example, a stereo input. The sound collection device 200 may be integrated with a microphone. In this case, the audio input unit 210 may be the microphone of the sound collection device 200.

[0038] 4, each of the signal processing units 220, 230 is configured with a signal processing circuit including an amplifier 222, 232 and an analog / digital (A / D) converter 224, 234, respectively. The multiple signal processing units 220, 230 include a high (H) level signal processing unit 220 and a low (L) level signal processing unit 230, in which separate gains Ga, Gb are set so as to share the entire dynamic range of the sound collection device 200.

[0039] In the amplifier 222 of the H-level signal processing unit 220, a gain Ga is set so as to reduce the influence of circuit noise, for example, from the viewpoint of accurately collecting input sound of a relatively low volume. For example, the gain Ga is larger than the gain Gb of the L-level signal processing unit 230. Thus, the H-level signal processing unit 220 has a dynamic range on the low volume side in the sound collection device 200.

[0040] In the amplifier 232 of the L-level signal processing unit 230, a gain Gb is set so as to suppress saturation distortion of the signal waveform from the viewpoint of accurately collecting input sound of a relatively large volume, for example. Thus, in the sound collection device 200, the L-level signal processing unit 230 has a dynamic range on the low volume side.

[0041] The dynamic range of the H-level signal processing unit 220 and the dynamic range of the L-level signal processing unit 230 may be continuous or partially overlapped. The A / D converters of the signal processing units 220 and 230 have common circuit characteristics such as resolution. One or more of the signal processing units 220 and 230 may be integrated on an IC chip.

[0042] The control unit 240, for example, controls the overall operation of the sound collection device 200. The control unit 240 includes, for example, a CPU or MPU that cooperates with software to realize a predetermined function. For example, the control unit 240 functions as an encoder 245 that encodes a signal to be transmitted from the communication unit 260.

[0043] The encoder 245 does not have to be realized by a function of the control unit 240, and may be incorporated in the communication unit 260, for example. The control unit 240 may be a dedicated electronic circuit designed to realize a predetermined function or a hardware circuit such as a reconfigurable electronic circuit. The control unit 240 may be configured with various semiconductor integrated circuits such as a CPU, an MPU, a microcomputer, a DSP, an FPGA, and an ASIC.

[0044] The storage unit 250 includes a ROM and a RAM that store programs and data necessary to realize the functions of the sound collection device 200. The storage unit 250 stores information indicating the gains of each of the multiple signal processing units 220 and 230, for example.

[0045] The communication unit 260 is a module (circuit) that connects to an external device according to a predetermined communication standard such as BLE. For example, the communication unit 260 performs wireless communication of an audio signal in accordance with the LE Audio standard. The communication by the communication unit 260 is not limited to wireless communication and may be wired communication. Furthermore, the communication standard of the communication unit 260 is not particularly limited and may be, for example, USB, HDMI, IEEE802.11, Wi-Fi, etc. The communication unit 260 is an example of a transmission unit of the sound collection device 200 in this embodiment, and may be an example of a reception unit of the sound collection device 200.

[0046] 2. Operation The operation of the imaging system 10 configured as above will be described below.

[0047] The sound collection device 200 (FIG. 3) of this embodiment acquires input sound, such as environmental sound in the shooting environment of the system 10, from the sound input unit 210, and performs two types of amplification conversion on the input sound by the H-level signal processing unit 220 and the L-level signal processing unit 230. The sound collection device 200 of this embodiment generates a data signal including the results of these two types of amplification conversion, and transmits it from the communication unit 260 to the digital camera 100 (FIG. 1).

[0048] In this embodiment, when the digital camera 100 receives the above-mentioned data signal from the sound collection device 200 while taking a video, for example, it executes a float recording process, which is a calculation process for performing float recording from the results of two types of amplification conversion. Details of the float recording operation by the sound collection device 200 and the digital camera 100 of this system 10 and details of the communication operation will be described later.

[0049] During the above float recording operation, in digital camera 100, image sensor 115 captures each frame of the video, and image processing engine 120 sequentially generates image data for each frame of the video. In digital camera 100 of this embodiment, control unit 135 performs encoding or the like to sequentially associate the float recording audio data obtained as described above with the image data for each frame, thereby generating a video file. The generated video file is recorded, for example, from card slot 140 to memory card 142.

[0050] According to the above operation of the present system 10, for example, when a user shoots a video, the digital camera 100 can perform float recording, making it easier to avoid recording accidents such as sound distortion or insufficient volume during video shooting. In addition, it also eliminates the need for complicated adjustments to the recording level settings that were previously required with conventional digital cameras when shooting such videos, and the user can easily obtain highly accurate sound collection results while concentrating on, for example, the composition of the video.

[0051] Furthermore, with the present system 10, the user can obtain float recording audio data in a video file obtained as a result of, for example, shooting a video with the digital camera 100. This saves the user the trouble of editing after shooting by replacing audio data picked up by a separate device with the audio data in the video file, as compared to, for example, a case in which separate equipment for performing float recording is prepared, making it easier for the user to use float recording.

[0052] 2-1.Float recording operation The float recording operation in the present system 10 will be described in detail with reference to FIGS.

[0053] Figures 5(A) to (F) are waveform diagrams for explaining the float recording operation in the present system 10. Figure 6 is a diagram for explaining the data structure of the float format.

[0054] In the sound collection device 200 of the present system 10, an input sound signal A1 indicating an input sound acquired by the sound input unit 210 is input to the H-level signal processing unit 220 and the L-level signal processing unit 230, as shown in Fig. 4. An example of the input sound signal A1 is shown in Fig. 5(A). In the waveform diagram of Fig. 5(A), the horizontal axis indicates time and the vertical axis indicates sound volume (same below).

[0055] In the H-level signal processing unit 220, the amplifier 222 amplifies the input audio signal A1 at a gain Ga that is set to a relatively large value. The A / D converter 224 performs A / D conversion to convert the amplification result of the input audio signal A1 in the amplifier 222 from an analog signal to a digital signal, thereby generating an H-level audio signal A2. Such processing of the input audio signal A1 in the H-level signal processing unit 220 is an example of a first amplification conversion in this embodiment.

[0056] Fig. 5(B) illustrates an example of an H-level audio signal A2 obtained from the input audio signal A1 in the example of Fig. 5(A). In the example of Fig. 5(B), waveform distortion occurs in the H-level audio signal A2 near the maximum value Ma that can be output by each of the signal processing units 220 and 230. On the other hand, the H-level audio signal A2 has a high signal-to-noise ratio because the gain Ga is large.

[0057] In the L-level signal processing unit 230, the amplifier 232 amplifies the input audio signal A1 at a gain Gb that is set to a relatively small value. The A / D converter 234 A / D converts the amplification result of the input audio signal A1 in the amplifier 232 from an analog signal to a digital signal to generate an L-level audio signal A3. Such processing of the input audio signal A1 in the L-level signal processing unit 230 is an example of a second amplification conversion in this embodiment.

[0058] Fig. 5C illustrates an example of an L-level audio signal A3 obtained from the input audio signal A1 in the example of Fig. 5A. The L-level audio signal A3 is less likely to cause distortion of the signal waveform than the H-level audio signal A2. On the other hand, the signal-to-noise ratio is low because the gain Gb is small.

[0059] The audio signals A2 and A3 generated in the sound collection device 200 are fixed-format digital signals that represent audio in digital values ​​within a predetermined dynamic range (±Ma) and resolution. In the present system 10, the remaining calculation processing for float recording of the two types of audio signals A2 and A3 generated as described above is performed in the digital camera 100.

[0060] In the digital camera 100 of this embodiment, the sound processing engine 170 performs float calculation processing by carrying out corresponding calculations as a data conversion section 172, an amplification section 174, and a combination section 176, for example, as shown in FIG.

[0061] In the sound processing engine 170 of the digital camera 100, first, the data conversion unit 172 converts each of the sound signals A2, A3 from the sound collection device 200 from fixed format to float format sound data. The data structure of the float format will be described with reference to FIG.

[0062] The float format is a data format in which data values ​​are expressed in floating-point format. The data structure of the float format includes a sign field 50, an exponent field 51, and a mantissa field 52, as shown in FIG.

[0063] The sign field 50 is a portion that indicates a positive or negative sign in a bit string that indicates a float format data structure, for example. The sign field 50 may be omitted from the float format data structure as appropriate.

[0064] The exponent field 51 is a portion of a bit string in float format that indicates the exponent of the data value in exponential notation. The exponential notation is, for example, a binary number with a base of 2. The exponent field 51 manages the volume level corresponding to the position of the decimal point in such notation.

[0065] The mantissa part 52 is a part that indicates the significant digits of the data value in the bit string in the float format. For example, the more bits allocated to the exponent part 52, the higher the resolution of the audio data.

[0066] In the float format, a predetermined number of bits defining the bit string is allocated and set in advance among the sign field 50, the mantissa field 52, and the exponent field 51. For example, in a 32-bit float recording, the sign field 50 has 1 bit, the exponent field 51 has 7 bits, and the mantissa field 52 has 24 bits. With such float format audio data, the resolution of the number of bits of the mantissa field 52 can be ensured over the volume level of the number of bits of the exponent field 51.

[0067] The audio processing engine 170 as the data conversion unit 172 sequentially calculates the values ​​of the exponent part 51 and the mantissa part 52 so as to normalize the data values ​​indicated by the H-level audio signal A2 at each time in a floating-point format, and generates audio data in float format. Similarly, the audio processing engine 170 performs a floating-point normalization calculation on the L-level audio signal A3 to generate audio data in float format. The audio data thus generated indicates a louder volume as the value of the exponent part 51 increases. The normalization is performed, for example, by sequentially increasing the value of the exponent part 51 until the most significant digit of the mantissa part 52 is not zero.

[0068] Returning to FIG. 4, in the sound processing engine 170, the amplifier 174 performs an amplification operation based on gain information indicating the gains Ga and Gb of the amplifiers 222 and 232 in the sound collection device 200, for example, to offset the difference between the gains Ga and Gb.

[0069] For example, in the audio processing engine 170, the amplifier 174 calculates H-level audio data A20 by amplifying the conversion result of the H-level audio signal A2 in the float format by the lower gain Gb in floating-point arithmetic. Similarly, the amplifier 174 calculates L-level audio data A30 by amplifying the conversion result of the L-level audio signal A3 by the higher gain Ga.

[0070] Fig. 5(D) illustrates H-level audio data A20 calculated from the H-level audio signal A2 in Fig. 5(B). Fig. 5(E) illustrates L-level audio data A30 calculated from the L-level audio signal A3 in Fig. 5(C). According to the calculation of the amplifier 174, it is possible to make the volume of the H-level audio data A20 and the L-level audio data A30 uniform, for example, as shown in Figs. 5(D) and 5(E).

[0071] Next, the combining unit 176 in the audio processing engine 170 generates float-recorded audio data A10 by a calculation process that combines, for example, the H-level audio data A20 and the L-level audio data A30 while switching between them. Fig. 5(F) shows an example of float-recorded audio data A10 generated from the audio data A20 and A30 in Figs. 5(D) and (E).

[0072] For example, the combining unit 176 compares the magnitude (absolute value) of the L level audio data A30 with a predetermined threshold value Mt, and when the L level audio data A30 is equal to or greater than the threshold value Mt, the combining unit 176 adopts the L level audio data A30 as the float recording audio data A10. On the other hand, when the L level audio data A30 is less than the threshold value Mt, the combining unit 176 adopts the H level audio data A20 as the float recording audio data A10. The threshold value Mt is set, for example, to a value equal to or less than the value Mb corresponding to the maximum value Ma of the output of each of the signal processing units 220 and 230 in each of the audio data A20 and A30.

[0073] According to the above-mentioned float recording operation, in the present system 10, the sound collection device 200 and the digital camera 100 cooperate with each other to obtain float recorded audio data A10 that accurately represents the input audio picked up by the sound collection device 200. Also, for example, it is not necessary to provide an arithmetic circuit capable of performing floating point arithmetic in the sound collection device 200, and the device configuration of the sound collection device 200 can be simplified.

[0074] According to the process of the combining unit 176, by using the L-level audio data A30 for threshold value judgment, it is easy to avoid the influence of signal distortion in the H-level audio data A20. Also, for example, as shown in Figures 5(E) and (F), in the range of sound volume where signal distortion may occur in the H-level audio data A20, the L-level audio data A30 is adopted, and the influence of signal distortion in the float-recorded audio data A10 can be suppressed. Furthermore, for example, as shown in Figures 5(D) and (F), the H-level audio data A20 is adopted as the sound pickup result except for the above-mentioned range of sound volume, and the signal-to-noise ratio is easily improved.

[0075] The combining unit 176 may perform various calculation processes to combine the H level audio data A20 and the L level audio data A30, and may provide hysteresis to the threshold determination as described above. In addition, when the H level audio data A20 and the L level audio data A30 are frequently switched by the threshold determination, the audio data may be temporarily fixed to the L level audio data A30, for example. By performing such calculation processes, it is possible to reduce the sense of discomfort felt by the sound data A10 recorded by the float recording.

[0076] In this embodiment, the combining unit 176 may perform a calculation process such as a floating point calculation to combine the H level audio data A20 and the L level audio data A30 at a predetermined combination ratio, instead of the calculation process to combine the audio data A20 and A30 while switching between them. Even by such a calculation process, the digital camera 100 of this embodiment can obtain the float recorded audio data A10.

[0077] Furthermore, in the amplifier 174, the amplification process may be performed by correcting the amplification factor by, for example, comparing the gains Ga and Gb indicated by the gain information with the difference between the two audio signals A2 and A3. This results in a robust system that does not affect the coupling even if a difference in the amplification factor occurs due to manufacturing variations in the amplifier.

[0078] 2-2.Communication operation Details of the communication operation between the sound collection device 200 and the digital camera 100 in the present system 10 will be described with reference to Figures 7 and 8. In the following, an example will be described in which the digital camera 100 operates as a master in BLE communication in the present system 10, and the sound collection device 200 operates as a slave.

[0079] Fig. 7 is a sequence diagram illustrating a communication operation in the present system 10. Fig. 8 is a diagram illustrating an example of a structure of a communication packet in the present system 10.

[0080] For example, before communication is established in the system 10, the sound collection device 200 performs advertising by broadcasting device information indicating the specifications of the device itself (S1). The digital camera 100 performs scanning to search for devices to which BLE communication is applicable (S2). In this way, the digital camera 100 receives the device information from the sound collection device 200, and a communication connection between the sound collection device 200 and the digital camera 100 is established (S3).

[0081] For example, when communication is established in the present system 10 (S1 to S3), the control unit 240 of the sound collection device 200 causes the communication unit 260 to transmit (S1) gain information indicating the gains Ga and Gb of the amplifiers 222 and 232 stored in the storage unit 250, which is included in the device information. In this way, the communication module 160 of the digital camera 100 receives the gain information from the sound collection device 200 (S2). The device information may include identification information of the sound collection device 200 and whether or not the device itself is applicable to the float recording operation.

[0082] Next, the control unit 135 of the digital camera 100 notifies the sound collection device 200 of communication setting information indicating various settings in the established communication connection (S4). The communication setting information includes, for example, a packet length and packet allocation settings (see FIG. 8) for performing a float recording operation. The communication unit 260 of the sound collection device 200 receives the communication setting information from the digital camera 100 (S4).

[0083] Thereafter, the digital camera 100 starts shooting a moving image in response to, for example, a user operation (S5), and instructs the sound collection device 200 to start collecting sound. The notification of the communication setting information (S4) may be performed, for example, at the start of shooting a moving image (S5), or may be performed together with the instruction to start collecting sound. The sound collection device 200 performs amplification conversion and encoding for float recording in response to, for example, the instruction from the digital camera 100 (S5) (S6).

[0084] In step S6, each of the signal processors 220, 230 (FIG. 4) in the sound collection device 200 performs two types of amplification conversion in parallel for each channel of the sound input signal A1, for example, to generate an H-level sound signal A2 and an L-level sound signal A3 for each channel. Furthermore, in the sound collection device 200 of this embodiment, the data signal is encoded into a communication packet so that the generated sound signals A2, A3 are transmitted to the digital camera 100 at the same timing.

[0085] Fig. 8 shows an example of the structure of a communication packet in the present system 10. In the example of Fig. 8, a communication packet for two-channel input audio signals is illustrated. The communication packet in this example has a packet length of 30 milliseconds and two segments per channel. The communication packet is an example of a communication unit in this embodiment.

[0086] 8 stores the data value of the H-level audio signal A2 in one segment of each channel, and the data value of the L-level audio signal A3 in the other segment. In step S6, the encoder 245 of the sound collection device 200 sequentially configures these communication packets according to the communication setting information received in step S4, for example, to encode each of the audio signals A2 and A3. The communication unit 260 of the sound collection device 200 transmits to the digital camera 100 a data signal generated as a result of the processing by the encoder 245 and configured with the above-mentioned communication packets.

[0087] The digital camera 100 receives the data signal from the sound collection device 200 in the communication module 160, and executes decoding of the data signal and float recording processing (S7). First, the control unit 135 of the digital camera 100 functions as, for example, a decoder 165, and decodes the data signal from the sound collection device 200 to obtain an H-level audio signal A2 and an L-level audio signal A3.

[0088] Furthermore, in step S7, the audio processing engine 170 of the digital camera 100 performs the above-mentioned float recording process based on the acquired audio signals A2 and A3 to generate float recorded audio data A10 (see FIG. 4). Such float recording process is performed, for example, using the gain information (Ga, Gb) acquired in step S2. For example, the control unit 135 of the digital camera 100 can generate a video file by associating the generated audio data A10 with image data captured at the same time.

[0089] According to the above communication operation of the present system 10, the wireless communication of data signals between the sound collection device 200 and the digital camera 100 makes it possible to realize floating recording of multiple channels.

[0090] 8 illustrates an example of a communication packet for two channels. In the present system 10, the data signal is not particularly limited to two channels, and may be one channel or three or more channels. For example, in the case of one channel, latency may be reduced by sequentially storing data values ​​of the audio signals A2 and A3 of the same channel in two segments for each channel of the communication packet illustrated in FIG. 8.

[0091] In the above description, an example has been described in which gain information from the sound collection device 200 is transmitted when communication is established (S1 to S3) in the system 10. In the system 10, transmission and reception of gain information is not particularly limited to this, and may be performed after communication is established between the sound collection device 200 and the digital camera 100, for example.

[0092] 3. Summary As described above, in this embodiment, the digital camera 100, which is an example of an imaging device, includes the image sensor 115, which is an example of an imaging unit, the audio processing engine 170, which is an example of an audio processing unit, and the communication module 160, which is an example of a receiving unit. The image sensor 115 captures an image of a subject to generate image data. The audio processing engine 170 generates audio data associated with the image data. The communication module 160 receives an H-level audio signal A2, which is an example of a first audio signal, and an L-level audio signal A3, which is an example of a second audio signal, from an external sound collection device 200. The H-level audio signal A2 indicates a result of processing by the H-level signal processing unit 220 as an example of a first amplification conversion performed on an input sound in the sound collection device 200. The L-level audio signal A3 indicates a result of processing by the L-level signal processing unit 230 as an example of a second amplification conversion performed on an input sound in the sound collection device 200, which is different from the first amplification conversion. The audio processing engine 170 receives the audio signals A2 and A3 from the communication module 160 as input, and performs a process of combining the signals with each other in a float format, which is an example of a predetermined data format, to generate audio data A10 of the pickup result that indicates the input audio in that data format.

[0093] According to the above digital camera 100, the audio processing engine 170 is responsible for processing to generate float format audio data, making it easy to obtain audio data indicating the audio collection results in a specified data format using the sound collection device 200 and digital camera 100.

[0094] In this embodiment, the first amplification conversion amplifies the input sound in an amplifier 222, which is an example of a first amplifier, to which a gain Ga, which is an example of a first gain, is set, and generates an H-level audio signal A2 by A / D conversion. The second amplification conversion amplifies the input sound in an amplifier 232, which is an example of a second amplifier, to which a gain Gb, which is an example of a second gain smaller than the first gain, is set, and generates an L-level audio signal A3 by A / D conversion. In the audio data A10 of the collected sound, the volume of the part corresponding to the first audio signal is equal to or greater than the volume of the part corresponding to the second audio signal. In this way, as the collected sound result of the system 10, audio data A10 of the input sound that is collected with high accuracy in a wide dynamic range using each of the audio signals A2 and A3 is obtained.

[0095] In this embodiment, the sound processing engine 170 acquires gain information indicating gains Ga and Gb in the sound collection device 200, and generates sound data A10 of the sound collection result from each of the sound signals A2 and A3 using the acquired gain information. Using such gain information, the sound processing engine 170 can generate the sound data A10 of the sound collection result with high accuracy. For example, the sound processing engine 170 uses the gain information to generate the sound data A10 of the sound collection result so as to reduce the difference between the gains Ga and Gb in each of the sound signals A2 and A3.

[0096] In this embodiment, the sound processing engine 170 acquires gain information through information communication with the sound collection device 200 in the communication module 160. In this way, the digital camera 100 can easily acquire the gain information of the sound collection device 200, and easily obtain accurate sound data A10.

[0097] In this embodiment, the predetermined data format is a float format having a mantissa part 52 and an exponent part 51. The audio processing engine 170 generates the audio data A10 of the collected sound so that the exponent part 51 increases as the volume of the input sound increases. In this way, the digital camera 100 of this embodiment can generate the audio data A10 of the collected sound with high accuracy using the float format.

[0098] In this embodiment, the communication module 160 receives a data signal conforming to a predetermined communication standard from the sound collection device 200. The data signal includes a segment of an example of a first portion in which an H-level audio signal A2 is stored, and an example of a second portion in which an L-level audio signal A3 is stored, for each communication packet that is an example of a communication unit in the communication standard (see FIG. 8). In this way, data communication between the digital camera 100 and the sound collection device 200 makes it easy for the digital camera 100 to obtain audio data A10 of the collected audio.

[0099] In this embodiment, the digital camera 100 further includes a control unit 135 that generates a moving image file by associating audio data A10 of the collected sound with image data. According to the digital camera 100 of this embodiment, the audio data A10 of the collected sound in the above data format can be obtained in the moving image file resulting from the moving image shooting, making it easier for the user to shoot such moving images.

[0100] In this embodiment, the sound collection device 200 transmits the sound signal to the digital camera 100 by transmitting a data signal including each of the sound signals A2 and A3. The sound collection device 200 includes a sound input unit 210 that acquires an input sound, an H level signal processing unit 220 that is an example of a first signal processing unit that performs a first amplification conversion on the input sound to generate a first sound signal, an H level signal processing unit 230 that is an example of a second signal processing unit that performs a second amplification conversion different from the first amplification conversion on the input sound to generate a second sound signal, and a communication unit 260 that is an example of a transmission unit that transmits the first sound signal and the second sound signal to the imaging device. The sound collection device 200 of this embodiment makes it easy to obtain sound data indicating the sound collection result in a predetermined data format using the sound collection device 200 and the digital camera 100.

[0101] In this embodiment, the imaging system 10 includes the above-mentioned sound collection device 200, and a digital camera 100 that receives first and second audio signals from the sound collection device 200, and performs processing to combine the received first and second audio signals with each other in a predetermined data format as inputs to generate audio data A10 of the sound collection result indicating the input audio in the data format. According to this system 100, it is possible to easily obtain audio data indicating the sound collection result in a predetermined data format using the sound collection device 200 and the digital camera 100.

[0102] (Other embodiments) As described above, the first embodiment has been described as an example of the technology disclosed in this application. However, the technology in this disclosure is not limited to this, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are appropriately performed. In addition, it is also possible to combine the components described in the above embodiment to create a new embodiment. Therefore, other embodiments will be exemplified below.

[0103] In the above-mentioned first embodiment, an operational example in which the digital camera 100 receives gain information from the sound collection device 200 has been described, but the present embodiment is not limited to this. For example, the flash memory 145 of the digital camera 100 of this embodiment may store gain information in advance for each model of the sound collection device 200. The digital camera 100 of this embodiment may receive identification information of the sound collection device 200 instead of the gain information, and may acquire the gain information from the flash memory 145 based on such identification information.

[0104] In the present embodiment, the gain information and the like may be acquired through a user operation, not limited to from the sound collection device 200. Such a modified example will be described with reference to FIG.

[0105] 9 shows a display example of the display monitor 130 in the modified digital camera 100. For example, the digital camera 100 of this embodiment displays an operation screen on the display monitor 130 that allows the gain Ga of the H-level signal processing unit 220 and the gain Gb of the L-level signal processing unit 230 to be changed, and accepts user operations on the operation screen at the operation unit 150.

[0106] 9, the numerical values ​​of the gains Ga and Gb are displayed so as to be changeable. The digital camera 100 may display options for the numerical values ​​of the gains Ga and Gb for each model of the sound collection device 200, for example. Alternatively, the digital camera 100 may display the models of the sound collection device 200 as options and accept a user operation for selection. The digital camera 100 of this embodiment may acquire gain information according to input of various user operations as described above.

[0107] In each of the above embodiments, an example has been described in which the sound collection device 200 and the digital camera 200 perform BLE communication to realize floating recording. In this embodiment, the sound collection device 200 and the digital camera 200 may achieve floating recording by wireless communication or wired communication other than BLE communication. For example, the sound collection device 200 of this embodiment may perform USB standard wired communication with the digital camera 100 to transmit data of each of the audio signals A2 and A3.

[0108] In each of the above embodiments, the sound collecting device 200 is described as having two signal processing units 220, 230 for one channel of input sound. The sound collecting device 200 of this embodiment may have three or more signal processing units for one channel of input sound. In the sound collecting device 200 of this embodiment, the gains of the three or more signal processing units may be set so as to share the dynamic range among them. The digital camera 100 of this embodiment may generate float format sound data based on the sound signals of the sound collection results of each of these three or more signal processing units so as to use the sound that is accurately collected in each sound signal.

[0109] In each of the above embodiments, an example was described in which the sound collection device 200 does not perform float recording processing. In this embodiment, the sound collection device 200 may perform float recording processing, and may further include a configuration such as an arithmetic circuit that realizes such arithmetic processing. Alternatively, the sound collection device 200 may further include a recording function other than float recording. Even in such a case, for example, the sound collection device 200 can transmit each of the audio signals A2 and A3 to the digital camera 100 as in the first embodiment, so that the digital camera 100 can also perform float recording processing, and the arithmetic load of the sound collection device 200 can be reduced, for example. As in the above embodiments, the sound collection device 200 of this embodiment can also easily obtain audio data showing the sound collection result in a float format using the sound collection device 200 and the digital camera 100.

[0110] In each of the above embodiments, an example has been described in which the sound collection device 200 transmits data of the audio signals A2 and A3 to the digital camera 100, and the digital camera 100 performs the floating recording process. In this embodiment, the sound collection device 200 may transmit data to an audio processing device such as various electronic devices other than an imaging device such as the digital camera 100. In this embodiment, such an audio processing device may perform the floating recording process in the same manner as the digital camera 100 in the first embodiment.

[0111] That is, the audio processing device of this embodiment is an imaging device including an audio processing unit that generates audio data, and a receiving unit that receives a first audio signal and a second audio signal from an external audio collection device. The first audio signal indicates the result of a first amplification conversion performed on the input audio in the audio collection device. The second audio signal indicates the result of a second amplification conversion performed on the input audio in the audio collection device, which is different from the first amplification conversion. The audio processing unit generates audio data of the audio collection result that indicates the input audio in a predetermined data format based on the first and second audio signals received from the receiving unit. As a result, in this embodiment, it is possible to easily obtain audio data that indicates the audio collection result in a float format using the audio collection device and the audio processing device that perform data communication.

[0112] In the above embodiments, the float format is exemplified as an example of the predetermined data format. In this embodiment, the predetermined data format is not necessarily limited to the float format, and may be, for example, various data formats that can ensure the resolution of each voice whose volume is significantly different from the range of the voice resolution.

[0113] In addition, in each of the above embodiments, the memory card 142 is exemplified as the recording medium, and the card slot 140 is exemplified as the recording unit of the digital camera 100, but the recording unit is not limited to this. In this embodiment, the recording medium is not limited to a memory card, and may be, for example, an external storage device such as an SSD drive. Furthermore, the digital camera 100 of this embodiment may upload video files, etc. to a cloud server, etc., via the communication module 160, for example.

[0114] In addition, in each of the above embodiments, the digital camera 100 including the optical system 110 and the drive unit 112 has been exemplified. The imaging device of the present embodiment does not need to include the optical system 110 and the drive unit 112, and may be, for example, an interchangeable lens type camera.

[0115] In addition, in each of the above embodiments, a digital camera has been described as an example of an imaging device, but the imaging device of the present disclosure may be any electronic device having an image capturing function (e.g., a video camera, a smartphone, a tablet terminal, etc.).

[0116] (Summary of aspects) Various aspects of the present disclosure are described below.

[0117] A first aspect of the present disclosure is an imaging device including an imaging unit that captures an image of a subject and generates image data, an audio processing unit that generates audio data associated with the image data, and a receiving unit that receives a first audio signal and a second audio signal from an external audio collection device. The first audio signal indicates the result of a first amplification conversion performed on the input audio in the audio collection device. The second audio signal indicates the result of a second amplification conversion performed on the input audio in the audio collection device, the second amplification conversion being different from the first amplification conversion. The audio processing unit performs a process of combining the first and second audio signals received from the receiving unit as inputs, and generates audio data of the audio collection result indicating the input audio in a predetermined data format.

[0118] In a second aspect, in the imaging device according to the first aspect, the first amplification conversion amplifies the input sound in a first amplifier set to a first gain, and generates a first sound signal by A / D (analog / digital) converting the sound. The second amplification conversion amplifies the input sound in a second amplifier set to a second gain smaller than the first gain, and generates a second sound signal by A / D converting the sound. In the sound data of the sound collection result, the volume of the part corresponding to the first sound signal is equal to or greater than the volume of the part corresponding to the second sound signal.

[0119] In a third aspect, in the imaging device described in the second aspect, the audio processing unit acquires gain information set in the first and second amplifiers in the sound collection device, and uses the gain information to generate audio data of the sound collection result from the first and second audio signals.

[0120] In a fourth aspect, in the imaging device according to the third aspect, the sound processing unit acquires gain information through information communication with the sound collection device in the receiving unit.

[0121] In a fifth aspect, in the imaging device according to any one of the first to fourth aspects, the predetermined data format is a float format having a mantissa and an exponent, and the audio processing unit generates audio data of the collected audio result such that the exponent part is increased as the volume of the input audio becomes louder.

[0122] In a sixth aspect, in the imaging device according to any one of the first to fifth aspects, the receiving unit receives a data signal conforming to a predetermined communication standard from the sound collecting device. The data signal includes a first portion in which a first audio signal is stored for each communication unit in the communication standard, and a second portion in which a second audio signal is stored.

[0123] In a seventh aspect, the imaging device according to any one of the first to sixth aspects further includes a control unit that generates a moving image file by associating audio data of a sound collection result with image data.

[0124] An eighth aspect is a sound collection device that transmits a sound signal to an imaging device. The sound collection device includes a sound input unit that acquires an input sound, a first signal processing unit that performs a first amplification conversion on the input sound to generate a first sound signal, a second signal processing unit that performs a second amplification conversion different from the first amplification conversion on the input sound to generate a second sound signal, and a transmission unit that transmits the first sound signal and the second sound signal to the imaging device.

[0125] A ninth aspect is an imaging system comprising a sound collecting device according to the eighth aspect, and an imaging device that receives first and second audio signals from the sound collecting device, and performs a process of combining the received first and second audio signals as inputs to generate audio data of the sound collection result indicative of the input audio in a predetermined data format.

[0126] As described above, the embodiments have been described as examples of the technology in the present disclosure, and the accompanying drawings and detailed description have been provided for this purpose.

[0127] Therefore, among the components described in the attached drawings and the detailed description, not only are there components essential for solving the problem, but there may also be components that are not essential for solving the problem in order to illustrate the above technology. Therefore, the fact that such non-essential components are described in the attached drawings or the detailed description should not be interpreted as immediately indicating that such non-essential components are essential.

[0128] Furthermore, since the above-described embodiments are intended to illustrate the technology in the present disclosure, various modifications, substitutions, additions, omissions, and the like can be made within the scope of the claims or their equivalents. [Industrial Applicability]

[0129] The present disclosure is applicable to an imaging device that receives an audio signal from a sound collecting device, a sound collecting device, and an imaging system including these. [Explanation of symbols]

[0130] 10. Imaging System 100 Digital Camera 115 Image Sensor 135 Control Unit 160 Communication Module 170 Audio Processing Engine 200 Sound recording device 210 Audio input section 220,230 Signal processing section 240 Control Unit 260 Communications Department

Claims

1. an imaging unit that captures an image of a subject and generates image data; an audio processing unit that generates audio data associated with the image data; a receiving unit that receives a first audio signal and a second audio signal from an external sound collecting device, the first audio signal indicates a result of a first amplification conversion performed on an input audio in the sound collection device; the second audio signal indicates a result of a second amplification conversion performed on the input audio in the sound collection device, the second amplification conversion being different from the first amplification conversion; The audio processing unit performs a process of combining the first and second audio signals received from the receiving unit as inputs, and generates audio data of the audio pickup result representing the input audio in a predetermined data format. Imaging device.

2. The first amplification and conversion includes amplifying the input audio in a first amplifier having a first gain set, and performing A / D conversion to generate the first audio signal; The second amplification and conversion includes amplifying the input audio in a second amplifier having a second gain set smaller than the first gain, and performing A / D conversion to generate the second audio signal; In the collected audio data, the volume of the portion corresponding to the first audio signal is equal to or greater than the volume of the portion corresponding to the second audio signal. The imaging device according to claim 1 .

3. The audio processing unit includes: Gain information set in the first and second amplifiers in the sound collection device is obtained, Using the gain information, audio data of the sound collection result is generated from the first and second audio signals. The imaging device according to claim 2 .

4. The sound processing unit acquires the gain information through information communication with the sound collecting device in the receiving unit. The imaging device according to claim 3 .

5. the predetermined data format is a float format having a mantissa and an exponent, The audio processing unit generates the audio data of the collected audio such that the exponent part is increased as the volume of the input audio increases. The imaging device according to claim 1 .

6. The receiving unit receives a data signal conforming to a predetermined communication standard from the sound collecting device, The data signal includes a first portion in which the first audio signal is stored for each communication unit in the communication standard, and a second portion in which the second audio signal is stored. The imaging device according to claim 1 .

7. The apparatus further includes a control unit that generates a video file by associating the audio data of the collected sound with the image data. The imaging device according to claim 1 .

8. A sound collection device that transmits a sound signal to an imaging device, A voice input unit for acquiring an input voice; a first signal processing unit that performs a first amplification conversion on the input voice to generate a first voice signal; a second signal processing unit that performs a second amplification conversion different from the first amplification conversion on the input voice to generate a second voice signal; a transmitting unit that transmits the first audio signal and the second audio signal to the imaging device; A sound recording device comprising:

9. The sound collecting device according to claim 8 , an imaging device that receives the first and second audio signals from the sound collection device, and performs a process of combining the received first and second audio signals as inputs to generate audio data of the sound collection result that indicates the input audio in a predetermined data format; An imaging system comprising:

Citation Information

Patent Citations

  • Data processor

    JP1988282800A

  • Photographing device

    JP2020156076A

  • Multichannel transducer devices and methods of operation thereof

    US20150280648A1