Display method and information processing device

Abstract

A display method according to the present invention comprises: a first acquisition step for acquiring second sound data of a second number of digits larger than a first number of digits from first sound data of a first number of digits, the first sound data of the first number of digits being generated from a sound signal output from a first sound collection device; and an output step for outputting, to a display device, a volume video indicating a volume related to the second sound data.

Description

Technical Field

[0001] The technology of the present invention relates to a display method and an information processing apparatus. Background Art

[0002] Japanese Patent Laid-Open No. 2012-073435 discloses a sound signal conversion device that samples an input analog sound signal of L channel and R channel at a sampling frequency of 192 kHz and a quantization bit number of 24 Bit by an A / D conversion device to generate a digital signal. A signal processing device is connected to the output side of the A / D conversion device. This signal processing device performs a process of downsampling the frequency to 1 / 4 (48 kHz) and a process of converting the downsampled signal into a floating-point format with a quantization bit number of 32 Bit.

[0003] Japanese Patent Laid-Open No. 2002-246913 discloses a data processing device that converts input data from a fixed-point form to a floating-point form by a conversion unit. Summary of the Invention

[0004] Technical Problem to be Solved by the Invention

[0005] An object of one embodiment of the technology of the present invention is to provide a display method and an information processing apparatus that can easily identify whether the volume is appropriate.

[0006] Means for Solving the Technical Problem

[0007] To achieve the above object, the display method of the present invention includes: a first acquisition step of acquiring second sound data of a second number larger than the first number based on first sound data of the first number, the first sound data of the first number being generated based on a sound signal output from a first sound collection device; and an output step of outputting a volume image representing the volume related to the second sound data to a display device.

[0008] Preferably, it further includes a second acquisition step of acquiring a subject image representing an image of a subject, and in the output step, the volume image and the subject image are output to the display device.

[0009] Preferably, the volume image includes any one of a first image representing the level of the volume and a second image representing the change in the volume smaller than the level represented by the first image.

[0010] Preferably, the volume image includes the second image, and the second image is an image representing the time change amount of the volume or an image magnifying a part of the first image.

[0011] Preferably, the volume image includes a first image and a second image. The first sound data includes first modulated sound data obtained by applying a first gain to a sound signal and second modulated sound data obtained by applying a second gain greater than the first gain to the sound signal. The second sound data is sound data generated by synthesizing the first modulated sound data and the second modulated sound data. The first image represents the level of the volume of the first modulated sound data, and the second image represents the level of the volume of the second modulated sound data.

[0012] Preferably, the volume image includes a first image. The first sound data includes first modulated sound data obtained by applying a first gain to a sound signal and second modulated sound data obtained by applying a second gain greater than the first gain to the sound signal. The second sound data is sound data generated by synthesizing the first modulated sound data and the second modulated sound data. The first image is an image representing the volume of the second sound data in a second digit.

[0013] Preferably, it further includes a third acquisition step of acquiring third sound data, which is generated by extracting data in a volume range having a width of a third digit smaller than the second digit from the second sound data. The volume image includes a third image related to the third sound data.

[0014] Preferably, the volume image includes the first image and the third image, and the third image is an image related to the volume of the third sound data.

[0015] Preferably, it further includes a fourth acquisition step of acquiring the volume of ambient sound based on the sound signal output from the second sound collection device. The volume image includes information related to the volume of the ambient sound.

[0016] Preferably, the volume image includes information about the ratio of the volume related to the second sound data to the volume of the ambient sound.

[0017] The display method of the present invention includes: a first acquisition step of acquiring second sound data with a second digit greater than the first digit based on the first sound data with the first digit, where the first sound data with the first digit is generated based on the sound signal output from the first sound collection device; a third acquisition step of acquiring third sound data, which is generated by extracting data in a volume range having a width of a third digit smaller than the second digit from the second sound data; and an output step of outputting an auxiliary image related to the third sound data to a display device.

[0018] Preferably, it further includes a second acquisition step of acquiring a subject image representing the subject of the second sound data. In the output step, in addition to the auxiliary image, the subject image is also output to the display device.

[0019] Preferably, the auxiliary image is an image related to the volume range.

[0020] Preferably, the auxiliary image is an image related to the degradation information, which is the information of the sound degraded from the second sound data in the third sound data.

[0021] Preferably, the degradation information is information related to the sound source of the degraded sound.

[0022] Preferably, it further includes an extraction step of extracting subject information corresponding to the sound source from the subject image, and the degradation information is the subject information.

[0023] The display method of the present invention includes: a first file creation step of creating a first file containing third sound data, where the third sound data is generated as follows: according to the first sound data of the first number of digits, create second sound data of a second number of digits larger than the first number of digits, and extract data of a volume range having a width of a third number of digits smaller than the second number of digits according to the second sound data, and the first sound data of the first number of digits is generated according to the sound signal output from the first sound collection device; a second file creation step of creating a second file containing the first sound data; and an output step of outputting a first icon representing the first file and a second icon representing the second file to the display device in different display manners.

[0024] Preferably, in the first file creation step, the volume range can be changed for the second sound data at a specified time, and in the output step, the display manner of the first icon is changed according to whether the volume range is changed on the time axis.

[0025] The information processing device of the present invention includes a processor, and the processor executes: a first acquisition step of acquiring second sound data of a second number of digits larger than the first number of digits according to the first sound data of the first number of digits, and the first sound data of the first number of digits is generated according to the sound signal output from the first sound collection device; and an output step of outputting a volume image representing the volume related to the second sound data to the display device.

[0026] The information processing device of the present invention includes a processor, and the information processing device executes: a first acquisition step of acquiring second sound data of a second number of digits larger than the first number of digits according to the first sound data of the first number of digits, and the first sound data of the first number of digits is generated according to the sound signal output from the first sound collection device; a third acquisition step of acquiring third sound data, where the third sound data is generated by extracting data of a volume range having a width of a third number of digits smaller than the second number of digits according to the second sound data; and an output step of outputting an auxiliary image related to the third sound data to the display device.

[0027] The information processing apparatus of the present invention includes a processor, and the information processing apparatus performs: a first file creation process of creating a first file including third sound data, where the third sound data is generated as follows: based on first sound data of a first number of bits, second sound data of a second number of bits larger than the first number of bits is created, and data within a volume range having a width of a third number of bits smaller than the second number of bits is extracted based on the second sound data, and the first sound data of the first number of bits is generated based on a sound signal output from a first sound collection device; a second file creation process of creating a second file including the first sound data; and an output process of outputting a first icon representing the first file and a second icon representing the second file to a display device in different display manners. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 FIG. is an example of a configuration diagram of a camera device according to the first embodiment.

[0029] Figure 2 FIG. is an example of a configuration diagram of a sound signal processing circuit.

[0030] Figure 3 FIG. conceptually shows sound signal processing based on a sound signal processing circuit.

[0031] Figure 4 FIG. is an example of a functional configuration diagram of a processor.

[0032] Figure 5 FIG. conceptually shows a synthesis process and a data format conversion process.

[0033] Figure 6 FIG. conceptually shows a data extraction process.

[0034] Figure 7 FIG. is an example of a volume image.

[0035] Figure 8 FIG. is an example of a volume image and a subject image displayed on a display.

[0036] Figure 9 FIG. is an example of a flowchart of the operation of a camera device.

[0037] Figure 10 FIG. shows a comparison between a conventional volume display and a volume display based on the present embodiment.

[0038] Figure 11 FIG. is a diagram showing a first modification example of a volume image.

[0039] Figure 12 FIG. is a diagram showing a second modification example of a volume image.

[0040] Figure 13 This is a diagram showing a third modified example of the volume image.

[0041] Figure 14 This is a diagram showing a fourth modified example of the volume image.

[0042] Figure 15 This is a diagram showing a fifth modified example of the volume image.

[0043] Figure 16 This is a diagram showing the functional structure of the processor according to the second embodiment.

[0044] Figure 17 This is a diagram showing an example of the volume image generated by the volume image generation unit according to the second embodiment.

[0045] Figure 18 This is a diagram showing a modified example of the volume image.

[0046] Figure 19 This is a flowchart showing the operation of the imaging device according to the second embodiment.

[0047] Figure 20 This is a diagram showing an example of the functional structure of the processor according to the third embodiment.

[0048] Figure 21 This is a diagram showing an example of the functional structure of the processor according to the fourth embodiment.

[0049] Figure 22 This is a flowchart showing the operation of the imaging device according to the fourth embodiment.

[0050] Figure 23 This is a diagram showing an example of the first icon and the second icon.

[0051] Figure 24 This is a diagram for explaining the operation of changing the volume range with respect to the time axis.

[0052] Figure 25 Shows the first icon when changing the volume range and the first icon when not changing the volume range.

[0053] Figure 26 This is a diagram showing an example of a moving image playback screen. Detailed Embodiments

[0054] An example of an embodiment related to the technology of the present invention will be described with reference to the accompanying drawings.

[0055] First, the terms used in the following description will be explained.

[0056] In the following description, "AF" is an abbreviation for "Auto Focus". "MF" is an abbreviation for "Manual Focus". "IC" is an abbreviation for "Integrated Circuit". "CPU" is an abbreviation for "Central Processing Unit". "RAM" is an abbreviation for "Random Access Memory". "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor".

[0057] "FPGA" is an abbreviation for "Field Programmable Gate Array". "PLD" is an abbreviation for "Programmable Logic Device". "ASIC" is an abbreviation for "Application Specific Integrated Circuit". "OVF" is an abbreviation for "Optical View Finder". "EVF" is an abbreviation for "Electronic View Finder". "ADC" is an abbreviation for "Analog to Digital Converter". "LPCM" is an abbreviation for "Linear Pulse Code Modulation".

[0058] As an embodiment of the imaging device, a lens interchangeable digital camera is exemplified to describe the technology of the present invention. In addition, the technology of the present invention is not limited to lens interchangeable type, and can also be applied to a lens integrated type digital camera.

[0059] [First Embodiment]

[0060] Figure 1 An example of the structure of the imaging device 10 according to the first embodiment is shown. The imaging device 10 is a lens interchangeable digital camera. The imaging device 10 is composed of a housing 11 and an imaging lens 12 that is detachably mounted on the housing 11 and includes a focusing lens 31. The imaging lens 12 is mounted on the front surface side of the housing 11 via a bayonet 11A. In addition, the imaging device 10 is an example of the "information processing device" related to the technology of the present invention.

[0061] Moreover, an external microphone 13 can be detachably mounted on the housing 11. The external microphone 13 is mounted on the housing 11 via a connection portion 11B provided on the upper surface of the housing 11. The external microphone 13 is, for example, a shotgun microphone, a zoom microphone, or the like. The connection portion 11B is, for example, a hot shoe.

[0062] An operation unit 16 including a dial, a release button, etc. is provided on the housing 11. As operation modes of the imaging device 10, for example, a still image shooting mode, a moving image shooting mode, and an image display mode are included. The operation unit 16 is operated by the user when setting the operation mode. Moreover, the operation unit 16 is operated by the user when starting to execute still image shooting or moving image shooting.

[0063] Moreover, a viewfinder 14 is provided on the housing 11. For example, the viewfinder 14 is a hybrid viewfinder (registered trademark). The hybrid viewfinder is, for example, a viewfinder that selectively uses an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF"). The user can observe an optical image or an instant preview image of the subject projected through the viewfinder 14 via a viewfinder eyepiece portion (not shown).

[0064] Moreover, a display 15 is provided on the back side of the housing 11. An image of the subject obtained by shooting and various menu screens are displayed on the display 15. The user can also observe the instant preview image projected through the display 15 instead of observing the viewfinder 14. In addition, the display 15 is an example of the "display device" related to the technology of the present invention.

[0065] Moreover, a speaker 17 is provided on the housing 11. The speaker 17 outputs sound according to the sound data included in a moving image file 28 described later.

[0066] The housing 11 and the imaging lens 12 are electrically connected via electrical contacts 11C provided on a bayonet mount 11A.

[0067] The imaging lens 12 includes a focusing lens 31, an aperture 32, and a lens drive control unit 33. The lens drive control unit 33 is electrically connected to a processor 25 accommodated in the housing 11 via the electrical contacts 11C.

[0068] The lens drive control unit 33 drives the focusing lens 31 and the aperture 32 according to a control signal transmitted from the processor 25. In order to adjust the position of the focusing lens 31, the lens drive control unit 33 performs drive control of the focusing lens 31 according to a focus control signal transmitted from the processor 25.

[0069] The aperture 32 has an aperture with a variable opening diameter. In order to adjust the amount of incident light incident on the imaging sensor 20, the lens drive control unit 33 performs drive control of the aperture 32 according to an aperture adjustment signal transmitted from the processor 25.

[0070] In addition, an imaging sensor 20, an image processing circuit 21, a built-in microphone 22, a sound signal processing circuit 23, a processor 25, and a storage device 26 are provided inside the housing 11. The imaging sensor 20, the image processing circuit 21, the built-in microphone 22, the sound signal processing circuit 23, the storage device 26, the display 15, and the speaker 17 are controlled by the processor 25 to operate.

[0071] The processor 25 is constituted by a CPU, for example. A RAM 25A serving as a primary storage memory is connected to the processor 25. The storage device 26 is constituted by a non-volatile memory such as a flash memory, for example. The processor 25 executes various processes according to a program 27 stored in the storage device 26. In addition, the processor 25 may be constituted by an aggregate of a plurality of IC chips. Further, for example, a moving image file 28 generated as a result of the imaging device 10 performing an image imaging operation is stored in the storage device 26.

[0072] The imaging sensor 20 is a CMOS image sensor, for example. The light (subject image) that has passed through the imaging lens 12 is incident on the light receiving surface 20A of the imaging sensor 20. A plurality of pixels that generate a captured image signal by performing photoelectric conversion are formed on the light receiving surface 20A. The imaging sensor 20 generates and outputs data representing the subject image by performing photoelectric conversion on the light incident on each pixel. Hereinafter, the data representing the subject image will be simply referred to as the subject image PD.

[0073] The image processing circuit 21 performs image processing including white balance correction, gamma correction processing, etc. on the subject image PD output from the imaging sensor 20.

[0074] The built-in microphone 22 is a stereo microphone having a pair of sound collecting elements 22A and 22B. The sound collecting elements 22A and 22B are a sound sensor for the left channel (hereinafter referred to as the L channel) and a sound sensor for the right channel (hereinafter referred to as the R channel). The sound collecting elements 22A and 22B are sound sensors such as electrostatic type, piezoelectric type, and electrodynamic type, and output the collected sound as sound signals AL and AR. The sound signal processing circuit 23 performs sound signal processing including gain processing, A / D conversion processing, etc. on the sound signals AL and AR output from the sound collecting elements 22A and 22B. In addition, the built-in microphone 22 is an example of the "second sound collecting device" related to the technology of the present invention.

[0075] The external microphone 13 includes a sound collecting element 41, an amplifier 42, and a microphone control unit 43. In the present embodiment, the external microphone 13 is a monaural microphone having one sound collecting element 41. The sound collecting element 41 is a sound sensor such as an electrostatic type, a piezoelectric type, or an electrodynamic type, and outputs the collected sound as a sound signal. The amplifier 42 performs a gain process on the sound signal output from the sound collecting element 41. The microphone control unit 43 controls the gain amount based on the gain process of the amplifier 42. In addition, the external microphone 13 is an example of the "first sound collecting device" related to the technology of the present invention.

[0076] Moreover, the microphone control unit 43 supplies the sound signal that has undergone the gain process by the amplifier 42 to the sound signal processing circuit 23 within the housing 11 via the connection unit 11B. A monaural and analog sound signal AS is supplied from the external microphone 13 to the sound signal processing circuit 23. In addition, the operation of the microphone control unit 43 is controlled by the processor 25.

[0077] Figure 2 An example of the structure of the sound signal processing circuit 23 is shown. The sound signal processing circuit 23 includes a first preamplifier 51A, a first ADC 52A, a second preamplifier 51B, and a second ADC 52B.

[0078] The first preamplifier 51A and the first ADC 52A are processing units for the L channel that perform a gain process and an A / D conversion process on the sound signal AL output from the sound collecting element 22A included in the built-in microphone 22. The second preamplifier 51B and the second ADC 52B are processing units for the R channel that perform a gain process and an A / D conversion process on the sound signal AR output from the sound collecting element 22B included in the built-in microphone 22.

[0079] The gain amount G1 of the first preamplifier 51A is controlled by the processor 25. The gain amount G2 of the second preamplifier 51B is controlled by the processor 25. When performing a gain process on the sound signals AL and AR output from the built-in microphone 22, the gain amount G1 and the gain amount G2 are set to the same value by the processor 25. The first ADC 52A and the second ADC 52B convert the analog sound signal into a digital signal in the form of 24-bit LPCM, for example, by sampling with a quantization bit number of 24 bits. In addition, it is an example of the pulse code modulation form.

[0080] When the external microphone 13 is connected to the connection unit 11B, the sound signal AS output from the external microphone 13 is input to the sound signal processing circuit 23. In this case, the operation of the built-in microphone 22 is invalidated, and the sound signal is not input from the built-in microphone 22 to the sound signal processing circuit 23.

[0081] The sound signal AS output from the external microphone 13 is input to the first preamplifier 51A and the second preamplifier 51B. The first preamplifier 51A performs a gain process on the sound signal AS with a gain amount G1. The second preamplifier 51B performs a gain process on the sound signal AS with a gain amount G2. When performing a gain process on the sound signal AS output from the external microphone 13, the gain amount G1 and the gain amount G2 are set to different values by the processor 25. Hereinafter, the gain process performed by the first preamplifier 51A is referred to as the first gain process, and the gain process performed by the second preamplifier 51B is referred to as the second gain process.

[0082] The first ADC 52A converts the sound signal AS that has undergone the first gain process by the first preamplifier 51A into a digital signal. The second ADC 52B converts the sound signal AS that has undergone the second gain process by the second preamplifier 51B into a digital signal. Hereinafter, the sound signal AS digitized by the first ADC 52A is referred to as the first modulated tone data ASM1, and the sound signal AS digitized by the second ADC 52B is referred to as the second modulated tone data ASM2. Also, the sound data generated based on the sound signal AS output from the external microphone 13 is referred to as the first sound data AS1. The first sound data AS1 includes the first modulated tone data ASM1 and the second modulated tone data ASM2. The first sound data AS1 is output from the sound signal processing circuit 23 to the processor 25.

[0083] Figure 3 Conceptually shows the sound signal processing based on the sound signal processing circuit 23. The sound signal AS output from the external microphone 13 is input to the L-channel processing unit and the R-channel processing unit. The sound signal AS input to the L-channel processing unit is converted into a digital signal after undergoing the first gain process with the gain amount G1, and thus is output from the sound signal processing circuit 23 as the first modulated tone data ASM1. The sound signal AS input to the R-channel processing unit is converted into a digital signal after undergoing the second gain process with the gain amount G2, and thus is output from the sound signal processing circuit 23 as the second modulated tone data ASM2. In the present embodiment, the number of bits (hereinafter referred to as the first number of bits) of the first sound data AS1 is 24 bits.

[0084] For example, the gain amount G1 is set to +48 dB, and the gain amount G2 is set to -48 dB. Since 48 dB corresponds to a volume width of 8 bits, as Figure 3 shown, there is a deviation of 16 bits between the high-gain first modulated tone data ASM1 and the low-gain second modulated tone data ASM2. In other words, there is an overlap of 8 bits between the first modulated tone data ASM1 and the second modulated tone data ASM2.

[0085] Figure 4An example of the functional structure of the processor 25 is shown. The processor 25 implements various functional units by executing processes according to the program 27 stored in the storage device 26. Figure 4 The various functional units shown are implemented in the moving image shooting mode. As Figure 4 shown, for example, the processor 25 implements a main control unit 60, a synthesis processing unit 61, a data format conversion unit 62, a sound data file creation unit 64, an editing unit 65, a file creation unit 66, and a volume image generation unit 67. The editing unit 65 includes a volume range setting unit 65A and a data extraction unit 65B.

[0086] The main control unit 60 uniformly controls each unit of the imaging device 10. The main control unit 60 controls the operation of the imaging device 10 according to the instruction signal input from the operation unit 16. The main control unit 60 controls the imaging sensor 20 to cause the imaging sensor 20 to perform a shooting operation. The imaging sensor 20 outputs a subject image PD generated by shooting through the imaging lens 12. In the moving image shooting mode, the imaging sensor 20 outputs the subject image PD in a frame period. The subject image PD output from the imaging sensor 20 is input to the processor 25 after being subjected to image processing by the image processing circuit 21.

[0087] Moreover, in the moving image shooting mode, when the external microphone 13 is connected to the connection unit 11B, the main control unit 60 controls the external microphone 13 to perform a sound collection operation. The external microphone 13 outputs a sound signal AS to the sound signal processing circuit 23 via the connection unit 11B during the shooting operation of the imaging sensor 20. The sound signal processing circuit 23 generates and outputs first-digit first sound data AS1 including first modulation sound data ASM1 and second modulation sound data ASM2 by performing the above-described sound signal processing. That is, the first sound data AS1 is sound data corresponding to the subject image PD obtained by shooting a subject with the imaging sensor 20.

[0088] The synthesis processing unit 61 acquires the first sound data AS1 output from the sound signal processing circuit 23, and synthesizes the first modulation sound data ASM1 and the second modulation sound data ASM2 included in the first sound data AS1, thereby creating second sound data AS2. The second sound data AS2 is digital data in the LPCM format.

[0089] The data format conversion unit 62 converts the data format of the second sound data AS2 into a floating-point format. Hereinafter, the second sound data AS2 after being converted into the floating-point format is referred to as second sound data AS2F. The second sound data AS2F is sound data having a second digit larger than the first digit. In the present embodiment, the second digit is 32 bits.

[0090] The sound data file creation unit 64 creates a sound data file 68 that includes the second sound data AS2F created by the data format conversion unit 62. The sound data file creation unit 64 records the created sound data file 68 in the storage device 26.

[0091] The editing unit 65 refers to the sound data file 68 recorded in the storage device 26 and creates a third sound data AS3 having a third number of bits smaller than the second number based on the second sound data AS2F. For example, the third number of bits is 24 bits.

[0092] Specifically, the volume range setting unit 65A sets a volume range VR having a width of the third number of bits for the dynamic range of the second sound data AS2F. In the present embodiment, the volume range setting unit 65A sets the volume range VR according to the specified information input from the main control unit 60. The data extraction unit 65B extracts the data of the volume range VR set by the volume range setting unit 65A from the second sound data AS2F, thereby creating the third sound data AS3. The third sound data AS3 is digital data in the LPCM format.

[0093] The file creation unit 66 creates a moving image file 28 that includes the subject image PD output from the image processing circuit 21 and the third sound data AS3 output from the data extraction unit 65B, and stores it in the storage device 26.

[0094] The volume image generation unit 67 generates a volume image PA representing the volume related to the second sound data AS2F. In the present embodiment, the volume image generation unit 67 generates the volume image PA based on the first modulation sound data ASM1 and the second modulation sound data ASM2 included in the first sound data AS1.

[0095] The main control unit 60 outputs the volume image PA generated by the volume image generation unit 67 and the subject image PD generated by the image processing circuit 21 to the display 15. The display 15 displays the volume image PA and the subject image PD input from the main control unit 60.

[0096] Figure 5 Conceptually shows the synthesis process based on the synthesis processing unit 61 and the data format conversion process based on the data format conversion unit 62. The synthesis processing unit 61 synthesizes the first modulation sound data ASM1 and the second modulation sound data ASM2 by performing a mixing process on the overlapping portion of the 8-bit amounts of the first modulation sound data ASM1 and the second modulation sound data ASM2. The number of bits of the second sound data AS2 generated by this synthesis process is 40 bits. In this way, the second sound data AS2 with an expanded dynamic range of volume is obtained by synthesizing the first modulation sound data ASM1 and the second modulation sound data ASM2 with different synthesis gain amounts.

[0097] The data format conversion unit 62 converts the second audio data AS2 in 40-bit fixed-point format into the second audio data AS2F in 32-bit floating-point format (so-called 32-bit floating-point number). The 32-bit floating-point number consists of 1-bit sign, 8-bit exponent part, and 23-bit mantissa part. The conversion from fixed-point format to floating-point format can use a well-known method. With the floating-point format, a wide range of numerical expressions can be performed.

[0098] Figure 6 Conceptually shows the data extraction process based on the data extraction unit 65B. The data extraction unit 65B extracts the data within the volume range VR set by the volume range setting unit 65A from the second audio data AS2F, thereby creating the third audio data AS3 in 24-bit fixed-point format. Specifically, the data extraction unit 65B selects the values of the sign and exponent part of the 32-bit floating-point number according to the volume range VR, thereby creating the 24-bit third audio data AS3 represented by the mantissa part.

[0099] Usually, the audio data included in the moving image file is digital data in 24-bit LPCM format. Therefore, in this embodiment, the 24-bit third audio data AS3 is created.

[0100] For example, the volume range VR is set according to the operation of the user using the operation unit 16. In this case, the main control unit 60 supplies the operation signal of the user input from the operation unit 16 to the volume range setting unit 65A. The volume range setting unit 65A sets the volume range VR according to the operation signal supplied from the main control unit 60. The user can use the operation unit 16 to set the volume range VR at a desired position within the dynamic range of the second audio data AS2F. Also, the user can change the setting position of the volume range VR for the second audio data AS2F at a specified time by operating the operation unit 16.

[0101] Figure 7 Shows an example of the volume image PA. In this embodiment, the volume image PA includes the first image PA1 and the second image PA2. The volume image generation unit 67 generates the first image PA1 according to the first modulated audio data ASM1, and generates the second image PA2 according to the second modulated audio data ASM2. In this embodiment, the first image PA1 and the second image PA2 are respectively indicators representing the levels of the volume indicated by the volume bar B.

[0102] The second image PA2 is generated according to the second modulated audio data ASM2 to which a gain lower than that of the first modulated audio data ASM1 is applied, so it represents a change in volume with a smaller level than the volume represented by the first image PA1.

[0103] Further, the first image PA1 and the second image PA2 each have a marker M indicating the maximum value of the level within a specified past period. The volume image generation unit 67 can change the color of the marker M or the like according to whether the maximum value is within an appropriate range.

[0104] Also, scales indicating the level of the volume may be displayed in units such as decibels (dB) in the first image PA1 and the second image PA2. The scales may be equally spaced or non-equally spaced.

[0105] Figure 8 An example of the volume image PA and the subject image PD displayed on the display 15 is shown. As Figure 8 shown, for example, the volume image PA is displayed so as to overlap a part of the subject image PD. The user can easily recognize whether the volume of the second sound data AS2F is appropriate through the volume image PA displayed on the display 15. By recognizing the volume of the second sound data AS2F, the user can appropriately set the volume range VR when creating the third sound data AS3.

[0106] Figure 9 It is a flowchart showing an example of the operation of the imaging device 10. Figure 9 It shows the operation when the moving image shooting mode is selected as the operation mode and the external microphone 13 is connected to the connection part 11B.

[0107] First, the main control unit 60 determines whether there is an instruction to start moving image shooting by the user (step S10). When it is determined that there is a start instruction (step S10: YES), the first acquisition process (step S11) and the second acquisition process (step S12) are executed simultaneously.

[0108] In the first acquisition process, the second-digit second sound data AS2F larger than the first digit is acquired based on the first sound data AS1 of the first digit, and the first sound data AS1 of the first digit is generated based on the sound signal AS output from the external microphone 13. In the present embodiment, the first sound data AS1 includes the first modulation sound data ASM1 and the second modulation sound data ASM2. The first modulation sound data ASM1 and the second modulation sound data ASM2 are generated by performing gain processing on the sound signal AS with different gain amounts in the sound signal processing circuit 23 and then performing A / D conversion. The second sound data AS2F is generated by the data form conversion unit 62 converting the data form of the second sound data AS2, and the second sound data AS2 is generated by the synthesis processing unit 61 synthesizing the first modulation sound data ASM1 and the second modulation sound data ASM2. Further, in the first acquisition process, the sound data file creation unit 64 creates a sound data file 68 including the second sound data AS2F and records it in the storage device 26.

[0109] In the second acquisition process, a subject image PD is acquired, and the subject image PD represents an image of a subject generated by photographing the subject with the imaging sensor 20. In the present embodiment, the subject image PD is subjected to image processing by the image processing circuit 21.

[0110] After the first acquisition process and the second acquisition process, an output process (step S13) is executed. In the output process, a volume image PA representing the volume associated with the second sound data AS2F is generated by the volume image generation unit 67. In the present embodiment, the volume image PA including the first image PA1 and the second image PA2 is generated based on the first modulation sound data ASM1 and the second modulation sound data ASM2. Further, in the output process, the main control unit 60 outputs the volume image PA and the subject image PD to the display 15 and displays them on the display 15.

[0111] After the output process, the main control unit 60 determines whether there is an end instruction for moving image shooting by the user (step S14). When it is determined that there is no end instruction (step S14: No), the process returns to steps S11 and S12. Steps S11 and S12 are repeatedly executed until it is determined in step S14 that there is an end instruction.

[0112] When it is determined that there is an end instruction (step S14: Yes), a third acquisition process (step S15) is executed. In the third acquisition process, third sound data AS3 is acquired, and the third sound data is generated as follows: The sound data file 68 recorded in the storage device 26 is read, and data within a volume range VR having a third digit smaller than the second digit is extracted based on the second sound data AS2F. For example, the volume range VR is set according to an operation by the user using the operation unit 16.

[0113] After the third acquisition process, a moving image file creation process (step S16) is executed. In the moving image file creation process, a moving image file 28 including the subject image PD generated for a plurality of frames and the third sound data AS3 is created and recorded in the storage device 26. Thus, the operation of the imaging device 10 ends.

[0114] In the operation of the imaging device 10 described above, the user can confirm whether the volume level of the sound emitted by the subject is appropriate through the volume image PA displayed on the display 15.

[0115] Next, the effects exhibited by the imaging device 10 of the present embodiment will be described. In the format of a general moving image file, for example, it is required that the number of bits of the audio data incorporated into the moving image file be 24 bits or less. In the present embodiment, when moving image imaging is performed, the second audio data AS2F with an expanded bit depth of 32 bits is generated and recorded. However, due to the above format, the 32-bit second audio data AS2F is downsampled to 24-bit third audio data AS3. Further, in the present embodiment, a volume range VR is set in order to extract the sound emitted from a desired subject from the second audio data AS2F during downsampling.

[0116] Figure 10 FIG. 4 shows a comparison between the conventional volume display and the volume display based on the present embodiment. The conventional volume display, for example, converts the sound signals AL and AR collected by the built-in microphone 22 into 24-bit audio data to display the volume level. When the volume of the subject is large and the volume level of the 24-bit audio data reaches the maximum value (Max), it is not clear whether the volume level of the 32-bit second audio data AS2F reaches the maximum value with the conventional volume display. Conversely, when the volume of the subject is small and the volume level of the 24-bit audio data is at the minimum value (Min), it is not clear whether the volume level of the 32-bit second audio data AS2F is at the minimum value with the conventional volume display. Thus, with the conventional volume display, it is not clear whether there are defects such as sound distortion caused by a large sound or omission of picking up a small sound in the second audio data AS2F recorded during moving image imaging. If the second audio data AS2F has a defect, no matter how the volume range VR is set, the sound emitted from the desired subject cannot be extracted, and sometimes high-quality third audio data AS3 cannot be obtained.

[0117] In contrast, in the present embodiment, when moving image imaging is performed, a volume image PA indicating the volume related to the 32-bit second audio data AS2F is displayed, so that the user can identify whether there are defects such as sound distortion caused by a large sound or omission of picking up a small sound in the second audio data AS2F. For example, as Figure 10As shown, even when the level of the volume reaches the maximum value in the conventional volume display, according to the volume image PA of the present embodiment, it is sometimes possible to confirm that "sound distortion" has not occurred. On the contrary, even when the level of the volume is at the minimum value in the conventional volume display, according to the volume image PA of the present embodiment, it is sometimes possible to confirm that "sound pickup omission" has not occurred. Thus, according to the present embodiment, by referring to the volume image PA displayed when recording the second sound data AS2F during dynamic image shooting, the user can identify in advance whether high-quality third sound data AS3 can be obtained through downsampling after the dynamic image shooting is completed. Therefore, the user can detect in advance whether there is a recording failure of the second sound data AS2F during dynamic image shooting.

[0118] [Modification Example]

[0119] Next, various modification examples of the first embodiment will be described. In the above embodiment, the third sound data AS3 is generated by downsampling the 32-bit second sound data AS2F, but the third sound data AS3 can also be generated by downsampling the 40-bit second sound data AS2. In this case, the "second sound data of the second number" related to the technology of the present invention is the second sound data AS2, and the second number is 40 bits.

[0120] Moreover, regarding the volume image PA generated by the volume image generation unit 67, various modifications can be made as described below.

[0121] In the above embodiment, the volume image PA includes the first image PA1 and the second image PA2, but it may include only the first image PA1. For example, as Figure 11 shown, the volume image PA may include only the first image PA1, which is an image representing the volume of the second sound data AS2F in the second number. Similar to the above embodiment, the first image PA1 is an indicator representing the level of the volume by the volume bar B. And the mark M represents the maximum value of the level within a past specified period.

[0122] Moreover, as Figure 12 shown, in addition to the first image PA1, the volume image PA may include a second image PA2 that magnifies a part of the first image PA1, where the first image PA1 is an image representing the volume of the second sound data AS2F in the second number. And the second image PA2 may also be an image representing the amount of change in the volume of the second sound data AS2F over time. For example, the second image PA2 may represent the difference in the volume level between the current frame and the previous frame. And the second image PA2 may also represent the absolute value of this difference.

[0123] Moreover, as Figure 13As shown, the volume image PA may also include a third image PA3 related to the third sound data AS3. Figure 13 The first image PA1 shown is an image representing the volume of the second sound data AS2F in the second digit. The third image PA3 is an image representing a volume range VR having a width of the third digit for generating the third sound data AS3. Further, the volume image generation unit 67 may determine the position of the third image PA3 relative to the first image PA1 based on the level of the volume of the second sound data AS2F within a past specified period. For example, the third image PA3 may be arranged such that the central value or average value of the level of the volume of the second sound data AS2F within the past specified period is at the center. In this way, by displaying the third image PA3, the user can easily perform the setting operation of the volume range VR during the editing after the moving image shooting.

[0124] Also, as Figure 14 shown, the volume image generation unit 67 may calculate the difference ΔV between the maximum value and the maximum value of the level of the volume of the second sound data AS2F during the period from the start of the moving image shooting to the generation of the volume image PA. The volume image generation unit 67 may display a warning message MS1 within the volume image PA when the calculated difference ΔV exceeds the width of the volume range VR. The warning message MS1 indicates that sound distortion, sound pickup omission, etc. will occur when the third sound data AS3 is generated by downsampling.

[0125] Also, as Figure 15 shown, the volume image generation unit 67 may display a warning message MS2 within the volume image PA when the level of the volume of the second sound data AS2F is lower than the lower limit of the volume range VR. Similarly, the volume image generation unit 67 may also display a warning message within the volume image PA when the level of the volume of the second sound data AS2F exceeds the upper limit of the volume range VR.

[0126] [Second Embodiment]

[0127] Next, the imaging device according to the second embodiment will be described. The imaging device according to this embodiment differs only in the functional structure of the processor 25 from the above-described embodiment.

[0128] Figure 16FIG. 0 shows an example of the functional structure of the processor 25 according to the second embodiment. In the present embodiment, the volume image generation unit 67 further obtains information related to the volume of the ambient sound based on the sound signals AL and AR output from the built-in microphone 22. The external microphone 13 is a directional microphone such as a shotgun microphone, and thus mainly collects the sound of the main subject desired by the user (hereinafter referred to as the desired sound). In contrast, the built-in microphone 22 is an omnidirectional microphone without directionality, and thus widely collects sounds including ambient sounds other than the sound of the main subject. The volume image generation unit 67 can obtain the volume of the ambient sound based on the difference between the sound collected by the external microphone 13 and the sound collected by the built-in microphone 22.

[0129] Figure 17 FIG. 4 shows an example of the volume image PA generated by the volume image generation unit 67 according to the second embodiment. In the present embodiment, the volume image generation unit 67 generates a volume image PA including information related to the volume of the ambient sound. For example, the volume image PA includes information about the ratio of the volume related to the second sound data AS2F to the volume of the ambient sound. Figure 17 The shown volume image PA represents, in a comparable manner, the ratio of the volume level of the sound of the second sound data AS2F (i.e., the desired sound) to the volume level of the ambient sound. The user can identify whether the desired sound is drowned out by the ambient sound during dynamic image shooting through the volume image PA of the present embodiment.

[0130] When the volume of the desired sound is less than the volume of the ambient sound, the desired sound may be drowned out and disappear. Therefore, as Figure 18 shown, the volume image generation unit 67 can display a warning message MS3 in the volume image PA when the volume of the desired sound is less than the volume of the ambient sound.

[0131] Figure 19 FIG. 15 is a flowchart showing the operation of the imaging device according to the second embodiment. In the present embodiment, when the main control unit 60 determines that there is a start instruction, in addition to the first acquisition process (step S11) and the second acquisition process (step S12), a fourth acquisition process (step S20) is also executed. In the fourth acquisition process, information related to the volume of the ambient sound is obtained based on the sound signals AL and AR output from the built-in microphone 22. And, in the output process (step S13), the volume image generation unit 67 generates a volume image PA including information related to the volume of the ambient sound. Other operations are the same as those in the first embodiment.

[0132] In addition, the same modifications as those in the first embodiment can also be applied to the present embodiment.

[0133] [Third Embodiment]

[0134] Next, the imaging device according to the third embodiment will be described. The imaging device according to this embodiment differs only in the functional structure of the processor 25 from the above-described embodiments.

[0135] Figure 20 An example of the functional structure of the processor 25 according to the third embodiment is shown. In this embodiment, the volume image generation unit 67 creates an auxiliary image based on the information EI obtained from the editorial department 65, and generates a volume image PA including the auxiliary image. For example, the auxiliary image is an image related to the volume range VR having the third number of digits used for generating the third sound data AS3. The auxiliary image can be the same image as Figure 13 the third image PA3 shown.

[0136] The auxiliary image can also be an image related to the degradation information, which is information about the sound degraded from the second sound data AS2F when generating the third sound data AS3. Degradation means that at least a part of the sound of the sound source included in the second sound data AS2F is not included in the volume range of the third sound data AS3. The degradation information is the missing sound information when creating the third sound data AS3 with less information amount than the second sound data AS2F from the second sound data AS2F. That is, the degradation information is information included in a range outside the volume range VR of the second sound data AS2F.

[0137] In addition, the degradation information can also be information related to the sound source of the degraded sound. The sound source of the degraded sound is, for example, the type of the sound source of the sound degraded from the second sound data AS2F. For example, as Figure 8 shown, when there are a person and a bird in the subject image PD and the sound emitted by the bird is missing, it is determined that the "bird" is the sound source of the degraded sound. In this way, the volume image generation unit 67 can perform an extraction process of extracting subject information corresponding to the sound source from the subject image PD. In this case, the degradation information is the subject information. The subject information is information such as the type of the subject corresponding to the sound source of the degraded sound.

[0138] In addition, the same modification as the first embodiment can also be applied to this embodiment.

[0139] [Fourth Embodiment]

[0140] Next, the imaging device according to the fourth embodiment will be described. The imaging device according to this embodiment differs only in the functional structure of the processor 25 from the above-described embodiments.

[0141] Figure 21An example of the functional structure of the processor 25 according to the fourth embodiment is shown. In the present embodiment, the file creation unit 66 can create, in addition to the first moving image file 28A including the subject image PD and the third sound data AS3, a second moving image file 28B including the subject image PD and the first sound data AS1. The first sound data AS1 included in the second moving image file 28B is the first-digit sound data obtained by digitizing the sound signal AS output from the external microphone 13. The first sound data AS1 can be either the first modulated sound data ASM1 or the second modulated sound data ASM2. In addition, the first moving image file 28A is an example of the "first file" related to the technology of the present invention. The second moving image file 28B is an example of the "second file" related to the technology of the present invention.

[0142] Moreover, in the present embodiment, the main control unit 60 generates a first icon representing the first moving image file 28A and a second icon representing the second moving image file 28B, and outputs both of them to the display 15 in different display modes. The display 15 displays the first icon and the second icon. By using the operation unit 16 to click on either the first icon or the second icon, the user can selectively play the first moving image file 28A and the second moving image file 28B.

[0143] In addition, the main control unit 60 may cause the file creation unit 66 to create only either the first moving image file 28A or the second moving image file 28B according to the operation of the user on the operation unit 16.

[0144] Figure 22 It is a flowchart showing the operation of the imaging device according to the fourth embodiment. In the present embodiment, after the third acquisition step (step S15), the first file creation step (step S16A), the second file creation step (step S16B), and the icon output step (step S16C) are executed by the file creation unit 66.

[0145] In the first file creation step, the first moving image file 28A is created by the file creation unit 66. In the second file creation step, the second moving image file 28B is created by the file creation unit 66. In the icon output step, the first icon and the second icon are output to the display 15 by the main control unit 60.

[0146] Figure 23 An example of the first icon and the second icon displayed on the display 15 is shown. As Figure 23 shown, the number of bits (bit depth) of the sound data included in the moving image file is displayed in the first icon 71 and the second icon 72.

[0147] The third audio data AS3 included in the first moving image file 28A and the first audio data AS1 included in the second moving image file 28B are both 24-bit audio data. Therefore, to distinguish between the two, the characters "DC" are added to the first icon 71 to indicate that the third audio data AS3 included in the first moving image file 28A was generated by down-converting the second audio data AS2F, which has a larger bit count. In this embodiment, the display formats of the first icon 71 and the second icon 72 are differentiated by the addition of these characters. However, the display formats may also be differentiated by color, shape, or other means.

[0148] And, as Figure 24 As shown, the user can change the volume range VR at a predetermined time on the time axis of the second sound data AS2F by operating the operation unit 16. Furthermore, the user can set the volume range VR to be constant without changing it. Therefore, the display mode of the first icon 71 can be changed depending on whether the volume range VR of the third sound data AS3 included in the first moving image file 28A has been changed on the time axis.

[0149] Figure 25 The first icon 71A when the volume range VR is changed and the first icon 71B when the volume range VR is not changed are shown. Figure 25 In the example shown, a pattern is added to the first icon 71B in order to change the display form of the first icon 71A and the first icon 71B. Alternatively, the display form may be changed by characters, colors, shapes, etc.

[0150] Figure 26 An example of a dynamic image playback screen is shown. When the main control unit 60 plays the first dynamic image file 28A, it will be the same as that described in the modified example of the first embodiment. Figure 13 The same volume image PB as the volume image PA is displayed on the display 15. Furthermore, when the main control unit 60 plays the first moving image file 28A containing the third sound data AS3 whose volume range VR has been changed on the time axis, it changes the position of the frame F representing the volume range VR set when the third sound data AS3 was created, depending on the playback time. In this embodiment, the volume range VR is represented by the frame F, but it may also be represented by other display formats such as bars.

[0151] Furthermore, the same modifications as those in the first embodiment can also be applied to this embodiment.

[0152] Furthermore, the technology of the present invention is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablet terminals that have a camera function.

[0153] In the above-described embodiment, as the hardware structure of the control unit exemplified by the processor 25, various processors shown below can be used. The above-mentioned various processors include a CPU, which is a common processor that functions as software (program) execution, and in addition, processors such as FPGAs that can change the circuit structure after manufacturing. The FPGA includes a dedicated circuit, etc., and the dedicated circuit is a processor such as a PLD or an ASIC having a circuit structure specifically designed to execute a specific process.

[0154] The control unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple control units may be composed of one processor.

[0155] Examples of a single processor constituting multiple control units can be thought of in many ways. The first example has the following method: Represented by computers such as clients and servers, a processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple control units. The second example has the following method: Represented by a System On Chip (SOC), etc., a processor that uses one IC chip to implement the functions of the entire system including multiple control units is used. Thus, as the hardware structure, the control unit can be composed of one or more of the above-mentioned various processors.

[0156] Furthermore, as the hardware structure of these various processors, more specifically, a circuit formed by combining circuit elements such as semiconductor elements can be used.

[0157] As long as there is no contradiction, the above-described embodiment and each modification example can be appropriately combined.

[0158] The content described and illustrated above is a detailed description of a part related to the technology of the present invention, and is only an example of the technology of the present invention. For example, the description related to the above structure, function, action, and effect is an example of the description related to the structure, function, action, and effect of a part related to the technology of the present invention. Therefore, of course, it is also possible to delete unnecessary parts, add new elements, or make replacements to the content described and illustrated above without departing from the gist of the technology of the present invention. Also, in order to avoid trouble and facilitate understanding of the part related to the technology of the present invention, descriptions related to common technical knowledge that do not require special explanation when implementing the technology of the present invention are omitted from the content described and illustrated above.

[0159] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each document, patent application, and technical standard were specifically and individually incorporated by reference.

[0160] Through the above description, the following techniques can be mastered.

[0161] [Supplementary Note Item 1]

[0162] A display method, comprising:

[0163] A first acquisition step of acquiring second sound data of a second digit greater than the first digit based on first sound data of the first digit, the first sound data of the first digit being generated based on a sound signal output from a first sound collection device; and

[0164] An output step of outputting a volume image representing the volume associated with the second sound data to a display device.

[0165] [Supplementary Note Item 2]

[0166] The display method according to Supplementary Note Item 1 further includes a second acquisition step of acquiring a subject image representing an image of a subject,

[0167] In the output step, the volume image and the subject image are output to the display device.

[0168] [Supplementary Note Item 3]

[0169] The display method according to Supplementary Note Item 1 or 2, wherein

[0170] The volume image includes either a first image representing the level of the volume or a second image representing the change in the volume smaller than the level represented by the first image.

[0171] [Supplementary Note Item 4]

[0172] The display method according to Supplementary Note Item 3, wherein

[0173] The volume image includes the second image,

[0174] The second image is an image representing the time variation amount of the volume or an image magnifying a part of the first image.

[0175] [Supplementary Note Item 5]

[0176] The display method according to Supplementary Note Item 3, wherein

[0177] The volume image includes the first image and the second image,

[0178] The first sound data includes first modulated sound data to which a first gain is applied to the sound signal and second modulated sound data to which a second gain greater than the first gain is applied to the sound signal,

[0179] The second sound data is sound data generated by synthesizing the first modulation sound data and the second modulation sound data.

[0180] The first image represents the level of the volume of the first modulation sound data, and the second image represents the level of the volume of the second modulation sound data.

[0181] [Supplementary Note Item 6]

[0182] According to the display method described in Supplementary Note Item 3, wherein

[0183] The volume image includes the first image.

[0184] The first sound data includes first modulation sound data to which a first gain is applied to the sound signal and second modulation sound data to which a second gain larger than the first gain is applied to the sound signal.

[0185] The second sound data is sound data generated by synthesizing the first modulation sound data and the second modulation sound data.

[0186] The first image is an image representing the volume of the second sound data in the second digit.

[0187] [Supplementary Note Item 7]

[0188] According to the display method described in Supplementary Note Item 3 or 6, it further includes a third acquisition step of acquiring third sound data.

[0189] The third sound data is generated by extracting data in a volume range having a width of a third digit smaller than the second digit according to the second sound data.

[0190] The volume image includes a third image related to the third sound data.

[0191] [Supplementary Note Item 8]

[0192] According to the display method described in Supplementary Note Item 7, wherein

[0193] The volume image includes the first image and the third image.

[0194] The third image is an image related to the volume of the third sound data.

[0195] [Supplementary Note Item 9]

[0196] According to the display method described in any one of Supplementary Note Items 1 to 8, it further includes a fourth acquisition step of acquiring the volume of ambient sound according to the sound signal output from the second sound collection device.

[0197] The volume image includes information related to the volume of the ambient sound.

[0198] [Supplementary Note Item 10]

[0199] According to the display method described in Supplementary Note Item 9, wherein,

[0200] The volume image includes information on the ratio of the volume related to the second sound data to the volume of the ambient sound.

[0201] [Supplementary Note Item 11]

[0202] A display method, comprising:

[0203] A first acquisition step of acquiring second sound data of a second number larger than the first number based on the first sound data of the first number, the first sound data of the first number being generated based on a sound signal output from a first sound collecting device;

[0204] A third acquisition step of acquiring third sound data, the third sound data being generated by extracting data of a volume range having a width of a third number smaller than the second number based on the second sound data; and

[0205] An output step of outputting an auxiliary image related to the third sound data to a display device.

[0206] [Supplementary Note Item 12]

[0207] According to the display method described in Supplementary Note Item 11, it further includes a second acquisition step of acquiring a subject image representing an image of a subject of the second sound data,

[0208] In the output step, in addition to the auxiliary image, the subject image is also output to the display device.

[0209] [Supplementary Note Item 13]

[0210] According to the display method described in Supplementary Note Item 11 or 12, wherein,

[0211] The auxiliary image is an image related to the volume range.

[0212] [Supplementary Note Item 14]

[0213] According to the display method described in Supplementary Note Item 12, wherein,

[0214] The auxiliary image is an image related to information on sound degraded from the second sound data in the third sound data, that is, degraded information.

[0215] [Supplementary Note Item 15]

[0216] The display method according to appended note item 14, wherein,

[0217] The deterioration information is information related to the sound source of the deteriorated sound.

[0218] [Appended note item 16]

[0219] The display method according to appended note item 15, further comprising an extraction step of extracting subject information corresponding to the sound source from the subject image,

[0220] The deterioration information is the subject information.

[0221] [Appended note item 17]

[0222] A display method, comprising:

[0223] A first file creation step of creating a first file including third sound data, the third sound data being generated as follows: creating second sound data having a second number larger than the first number based on first sound data of the first number, and extracting data of a volume range having a width of a third number smaller than the second number based on the second sound data, the first sound data of the first number being generated based on a sound signal output from a first sound collecting device;

[0224] A second file creation step of creating a second file including the first sound data; and

[0225] An output step of outputting a first icon representing the first file and a second icon representing the second file to a display device in different display manners.

[0226] [Appended note item 18]

[0227] The display method according to appended note item 17, wherein,

[0228] In the first file creation step, the volume range can be changed for the second sound data at a prescribed time,

[0229] In the output step, the display manner of the first icon is changed according to whether the volume range is changed on a time axis.

Claims

1. A display method, comprising: a first acquisition step of acquiring second sound data of a second digit larger than the first digit based on first sound data of the first digit, the first sound data of the first digit being generated based on a sound signal output from a first sound collection device; and an output step of outputting a volume image representing a volume associated with the second sound data to a display device.

2. The display method according to claim 1, further comprising a second acquisition step of acquiring a subject image representing an image of a subject, wherein, in the output step, the volume image and the subject image are output to the display device.

3. The display method according to claim 1, wherein the volume image includes either a first image representing a level of the volume or a second image representing a change in the volume smaller than the level represented by the first image.

4. The display method according to claim 3, wherein the volume image includes the second image, and the second image is an image representing a time change amount of the volume or an image magnifying a part of the first image.

5. The display method according to claim 3, wherein the volume image includes the first image and the second image, the first sound data includes first modulation sound data to which a first gain is applied to the sound signal and second modulation sound data to which a second gain larger than the first gain is applied to the sound signal, the second sound data is sound data generated by synthesizing the first modulation sound data and the second modulation sound data, the first image represents a level of the volume of the first modulation sound data, and the second image represents a level of the volume of the second modulation sound data.

6. The display method according to claim 3, wherein the volume image includes the first image, the first sound data includes first modulation sound data to which a first gain is applied to the sound signal and second modulation sound data to which a second gain larger than the first gain is applied to the sound signal, the second sound data is sound data generated by synthesizing the first modulation sound data and the second modulation sound data, and the first image is an image representing the volume of the second sound data in the second digit.

7. The display method according to claim 3, further comprising a third acquisition step of acquiring third sound data, the third sound data being generated by extracting data of a volume range having a width of a third digit smaller than the second digit based on the second sound data, wherein the volume image includes a third image associated with the third sound data.

8. The display method according to claim 7, wherein the volume image includes the first image and the third image, and the third image is an image associated with the volume of the third sound data.

9. The display method according to claim 1, further comprising a fourth acquisition step of acquiring a volume of ambient sound based on a sound signal output from a second sound collection device, wherein the volume image includes information associated with the volume of the ambient sound.

10. The display method according to claim 9, wherein, the volume image includes information on the ratio of the volume related to the second sound data and the volume of the ambient sound.

11. A display method, comprising: a first acquisition step of acquiring second sound data of a second number larger than the first number based on first sound data of the first number, the first sound data of the first number being generated based on a sound signal output from a first sound collection device; a third acquisition step of acquiring third sound data, the third sound data being generated by extracting data of a volume range having a width of a third number smaller than the second number based on the second sound data; and an output step of outputting an auxiliary image related to the third sound data to a display device.

12. The display method according to claim 11, further comprising a second acquisition step of acquiring a subject image representing an image of a subject of the second sound data, in the output step, in addition to the auxiliary image, the subject image is also output to the display device.

13. The display method according to claim 11, wherein, the auxiliary image is an image related to the volume range.

14. The display method according to claim 12, wherein, the auxiliary image is an image related to information on sound deteriorated from the second sound data in the third sound data, i.e., deterioration information.

15. The display method according to claim 14, wherein, the deterioration information is information related to the sound source of the deteriorated sound.

16. The display method according to claim 15, further comprising an extraction step of extracting subject information corresponding to the sound source from the subject image, the deterioration information is the subject information.

17. A display method, comprising: a first file creation step of creating a first file including third sound data, the third sound data being generated by: creating second sound data of a second number larger than the first number based on first sound data of the first number, and extracting data of a volume range having a width of a third number smaller than the second number based on the second sound data, the first sound data of the first number being generated based on a sound signal output from a first sound collection device; a second file creation step of creating a second file including the first sound data; and an output step of outputting a first icon representing the first file and a second icon representing the second file to a display device in different display manners.

18. The display method according to claim 17, wherein, in the first file creation step, the volume range can be changed for the second sound data at a specified time, in the output step, the display manner of the first icon is changed according to whether the volume range is changed on the time axis.

19. An information processing device, comprising a processor, the processor executes: The first acquisition step, based on the first sound data of the first digit, acquires the second sound data of the second digit that is larger than the first digit, and the first sound data of the first digit is generated based on the sound signal output from the first sound collection device; and The output step outputs a volume image representing the volume related to the second sound data to a display device.

20. An information processing device, which includes a processor, The processor performs the following steps: The first acquisition step, based on the first sound data of the first digit, acquires the second sound data of the second digit that is larger than the first digit, and the first sound data of the first digit is generated based on the sound signal output from the first sound collection device; The third acquisition step acquires third sound data, and the third sound data is generated by extracting data of a volume range having a width of a third digit smaller than the second digit based on the second sound data; and The output step outputs an auxiliary image related to the third sound data to a display device.

21. An information processing device, which includes a processor, The processor performs the following steps: The first file creation step creates a first file including third sound data, and the third sound data is generated by: creating the second sound data of the second digit that is larger than the first digit based on the first sound data of the first digit, and extracting data of a volume range having a width of a third digit smaller than the second digit based on the second sound data, and the first sound data of the first digit is generated based on the sound signal output from the first sound collection device; The second file creation step creates a second file including the first sound data; and The output step outputs a first icon representing the first file and a second icon representing the second file to a display device in different display manners.

Images