electronic machines

Abstract

This technology provides the ability to present a suitable tactile sensation even when video and audio signals are of high quality. [Solution] The electronic device of the present invention is characterized by comprising: an acquisition means for acquiring at least one of a video signal and an audio signal; an improvement means for performing processing to improve the quality of the signal acquired by the acquisition means; and a control means for recording, in association with the signal after its quality has been improved by the improvement means and information regarding the amount of processing performed by the improvement means.

Description

Technical Field

[0001] The present invention relates to electronic devices, and particularly to a technology for presenting a pseudo tactile sensation.

Background Art

[0002] There is known a tactile presentation device (haptic device) that presents a pseudo tactile sensation to a user by vibrating a vibration device with a predetermined drive waveform or controlling the temperature of the device surface. When viewing content such as games and movies, the sense of immersion in the content can be enhanced by presenting a pseudo tactile sensation according to the viewing scene of the content using the tactile presentation device provided on the display device or the controller.

[0003] In addition, the performance of imaging devices has been remarkably improved, and the quality of video and audio can be improved by, for example, correcting camera shake in video and removing noise in audio.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When enhancing the sense of immersion in content by tactile presentation, it is important to present the characteristics of the shooting environment tactilely. However, when high-quality processing such as camera shake correction and noise removal is performed, the characteristics of the shooting environment are removed from the video and audio. Therefore, it is difficult to present a suitable tactile sensation from the high-quality video signal and audio signal after the high-quality processing. In the technology disclosed in Patent Document 1, tactile information is generated based on an audio signal, but it is impossible to generate tactile information that makes the user feel wind from the audio signal after the wind noise has been removed by noise removal.

[0006] The present invention aims to provide a technology that enables the presentation of suitable tactile sensations even when video and audio signals are of high quality. [Means for solving the problem]

[0007] A first aspect of the present invention is an electronic device characterized by comprising: an acquisition means for acquiring at least one of a video signal and an audio signal; an improvement means for performing processing to improve the quality of the signal acquired by the acquisition means; and a control means for recording, in association with the signal after its quality has been improved by the improvement means and information regarding the amount of processing performed by the improvement means.

[0008] A second aspect of the present invention is a control method for electronic equipment, characterized by comprising: an acquisition step of acquiring at least one of a video signal and an audio signal; an improvement step of performing processing to improve the quality of the signal acquired in the acquisition step; and a control step of recording the signal after its quality has been improved in the improvement step and information relating to the amount of processing performed in the improvement step.

[0009] A third aspect of the present invention is a program for causing a computer to function as each of the means of the electronic device. A fourth aspect of the present invention is a computer-readable storage medium that stores a program for causing a computer to function as each of the means of the electronic device. [Effects of the Invention]

[0010] According to the present invention, even when the quality of video signals and audio signals is improved, a suitable tactile sensation can be presented. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of an imaging device. [Figure 2] This is a block diagram showing the configuration of the control unit of the imaging device. [Figure 3] This is a flowchart showing the processing of the vibration-damping tactile information generation unit. [Figure 4] This flowchart shows the processing of the audio haptic information generation unit. [Figure 5] This is a block diagram showing the configuration of the image processing unit and control unit of an imaging device. [Figure 6] This is a diagram showing the external appearance of the imaging system. [Figure 7] This is a flowchart showing the processing of the image processing unit in the sub-information acquisition section. [Modes for carrying out the invention]

[0012] <Embodiment 1> Embodiment 1 of the present invention will now be described. Figure 1 is a block diagram showing the configuration of an imaging device 100 according to Embodiment 1. The imaging device 100 includes an optical system 101, an image sensor 102, an A / D conversion unit 103, an image processing unit 104, a recording unit 105, a control unit 106, a memory 107, an operation unit 108, a display unit 109, a motion sensor 110, a vibration damping unit 111, and a microphone 112.

[0013] The optical system 101 includes a focusing lens, an aperture, and a shutter. By driving the focusing lens, the system can focus on the subject, and by controlling the aperture and shutter, the amount of exposure can be adjusted.

[0014] The image sensor 102 is a photoelectric conversion element such as a CCD or CMOS that converts light incident on the image sensor 102 from a subject via the optical system 101 (the optical image formed on the image sensor 102 by the optical system 101) into an electrical signal through photoelectric conversion.

[0015] The A / D conversion unit 103 acquires a digital video signal by digitizing the electrical signal (analog signal) obtained by the image sensor 102.

[0016] The image processing unit 104 performs image processing such as simulcasting, white balance correction, and gamma processing on the video signal obtained by the A / D conversion unit 103, and outputs the processed video signal to the recording unit 105.

[0017] The recording unit 105 records the video signal output from the image processing unit 104 on a recording medium not shown. When recording a still image, the recording unit 105 converts the image format of the video signal output from the image processing unit 104 into JPEG or the like. When recording a moving image, the recording unit 105 converts the image format of the video signal output from the image processing unit 104 into mp4 or the like. The recording unit 105 can also record the audio signal (described later) output from the control unit 106 on the recording medium. The recording unit 105 can also record various information such as tactile information in association with the signal to be recorded (video signal or audio signal) as meta information.

[0018] The control unit 106 controls the operation of the imaging device 100. In Embodiment 1, the control unit 106 determines the processing amount (anti-shake amount) of the anti-shake processing performed by the anti-shake unit 111 and generates tactile information. The control unit 106 can also correct the audio signal output from the microphone 112 and output the corrected audio signal to the recording unit 105.

[0019] The memory 107 stores various information such as information used in the image processing unit 104 (for example, illumination information used for white balance correction processing) and settings related to shooting.

[0020] The operation unit 108 receives operations (instructions) for the imaging device 100 from the user. The operation unit 108 may have operation members such as buttons, or may acquire information on operations performed by the user using the operation members of an external device for the imaging device 100 from the external device by wire or wirelessly.

[0021] The display unit 109 is, for example, a liquid crystal display, and displays various images such as a screen for assisting the user's operation, a live video captured by the imaging element 102, and a video recorded by the recording unit 105. A touch panel may be integrally provided on the display unit 109.

[0022] The motion sensor 110 detects the movement of the imaging device 100 and outputs the detection result. The motion sensor 110 may include, for example, an accelerometer or a gyroscope.

[0023] The vibration isolation unit 111 performs vibration isolation processing based on the output of the motion sensor 110 (motion detection result) by moving at least one of the optical system 101 and the image sensor 102 to cancel out the movement of the imaging device 100. The vibration isolation processing can also be interpreted as a process to remove vibrations caused by vibrations of the imaging device 100 from the image represented by the output of the image processing unit 104 (video signal). The vibration isolation processing improves the quality of the video signal input to the image processing unit 104 and the recording unit 105.

[0024] The microphone 112 detects sound in the vicinity of the imaging device 100 and outputs an audio signal representing that sound.

[0025] Figure 2 is a block diagram showing the configuration of the control unit 106. The control unit 106 includes a vibration isolation amount determination unit 201, a vibration isolation tactile information generation unit 202, an audio noise amount estimation unit 203, an audio tactile information generation unit 204, and an audio processing unit 205.

[0026] The vibration isolation amount determination unit 201 determines the processing amount (vibration isolation amount) for vibration isolation based on the output (motion detection result) of the motion sensor 110. The vibration isolation unit 111 performs vibration isolation processing with the vibration isolation amount determined by the vibration isolation amount determination unit 201.

[0027] The vibration-damping tactile information generation unit 202 generates tactile information (information indicating touch) based on the vibration damping amount determined by the vibration damping amount determination unit 201. For example, the vibration-damping tactile information generation unit 202 generates tactile information to present vibrations removed from the image by vibration damping processing (vibrations of the imaging device 100 that are no longer represented in the image by vibration damping processing) through touch. In Embodiment 1, it is assumed that tactile information indicating the frequency and intensity of vibrations that reproduce touch is generated.

[0028] The audio noise level estimation unit 203 extracts noise signals (noise components) from the output (audio signal) of the microphone 112 and outputs the extracted noise signals to the audio tactile information generation unit 204 and the audio processing unit 205. The method for extracting noise signals is not particularly limited; noise information may be extracted by analyzing the waveform of the audio signal, or a preset type of noise signal may be extracted. For example, the user specifies the type of noise sound to be removed, such as wind noise or machine vibration noise. The audio noise level estimation unit 203 extracts frequency components corresponding to the specified type from the audio signal as noise signals.

[0029] The audio processing unit 205 performs noise reduction processing on the output (audio signal) of the microphone 112, removing the noise signal extracted by the audio noise amount estimation unit 203, and outputs the audio signal after noise reduction processing to the recording unit 105. The quality of the input audio signal is improved. The noise signal may be interpreted as the processing amount of the noise reduction process. Similar to the image processing unit 104, the audio processing unit may be an external element of the control unit 106. The image processing unit 104 may be an internal element of the control unit 106.

[0030] The audio tactile information generation unit 204 generates tactile information based on the noise signal extracted by the audio noise amount estimation unit 203. For example, it generates tactile information to present features removed from the audio signal by noise reduction processing. In Embodiment 1, it is assumed that tactile information indicating the frequency and intensity of vibrations that reproduce the tactile sensation is generated.

[0031] Figure 3 is a flowchart showing the processing of the vibration-damping tactile information generation unit 202.

[0032] In S301, the vibration isolation tactile information generation unit 202 obtains the vibration isolation amount from the vibration isolation amount determination unit 201.

[0033] In S302, the vibration-damping tactile information generation unit 202 estimates the type (cause) of vibration occurring in the imaging device 100 by analyzing the vibration damping amount acquired in S301. Here, vibrations occurring in the imaging device 100 may be interpreted as vibrations that are eliminated by vibration damping processing. For example, vibrations of an intensity weaker than a predetermined intensity are estimated to be vibrations caused by slight hand tremors of a stationary user holding the imaging device 100. If vibrations of an intensity stronger than a predetermined intensity occur at a predetermined frequency (period), these vibrations are estimated to be vibrations caused by the user walking while holding the imaging device 100. If vibrations of a predetermined intensity continue, these vibrations are estimated to be vibrations caused by the user moving on a mobile device while holding the imaging device 100.

[0034] In S303, the vibration-damping tactile information generation unit 202 determines the frequency of vibrations that reproduce tactile sensations based on the type estimated in S302. For example, the correspondence between the type of vibration of the imaging device 100 and the frequency of vibrations that reproduce tactile sensations is predetermined and stored in the memory 107. The vibration-damping tactile information generation unit 202 then refers to the memory 107 to determine the frequency corresponding to the type estimated in S302.

[0035] In S304, the vibration-damping tactile information generation unit 202 determines the intensity (amplitude) of the vibration to reproduce the tactile sensation based on the vibration damping amount (magnitude of vibration of the imaging device 100) acquired in S301. For example, a relationship formula between the vibration damping amount and the intensity of the vibration to reproduce the tactile sensation is predetermined, and the vibration-damping tactile information generation unit 202 uses this relationship formula to calculate the intensity from the vibration damping amount acquired in S301. For example, the vibration-damping tactile information generation unit 202 determines (calculates) a higher intensity the larger the vibration damping amount.

[0036] The vibration-damping tactile information generation unit 202 generates tactile information indicating the frequency determined in S303 and the intensity determined in S304.

[0037] Figure 4 is a flowchart showing the processing of the audio haptic information generation unit 204.

[0038] In S401, the voice tactile information generation unit 204 acquires a noise signal from the voice noise amount estimation unit 203.

[0039] In S402, the audio haptic information generation unit 204 acquires information on the type of noise signal (a type specified by the user) acquired in S401.

[0040] In S403, the audio tactile information generation unit 204 determines the frequency of vibration to reproduce the tactile sensation based on the type acquired in S402. For example, the correspondence between the type of noise signal and the frequency of vibration to reproduce the tactile sensation is predetermined and stored in the memory 107. The audio haptic information generation unit 204 refers to the memory 107 and determines the frequency corresponding to the type acquired in S402.

[0041] In S404, the voice-tactile information generation unit 204 determines the intensity (amplitude) of the vibration that reproduces the tactile sensation based on the noise signal acquired in S401. For example, a relationship formula between the magnitude of the noise signal and the intensity of the vibration that reproduces the tactile sensation is predetermined, and the voice-tactile information generation unit 204 uses this relationship formula to calculate the vibration intensity from the magnitude of the noise signal acquired in S401. For example, the voice-tactile information generation unit 204 determines (calculates) a higher intensity the larger the noise signal.

[0042] The audio tactile information generation unit 204 generates tactile information indicating the frequency determined in S403 and the intensity determined in S404.

[0043] As described above, according to Embodiment 1, video signals and audio signals are acquired, and processing (quality enhancement) is performed to improve the quality of the acquired signals. Then, the enhanced signals and information regarding the amount of processing (tactile information based on the amount of processing) are associated with each other and recorded. In this way, even if information is removed from the video or audio signals due to quality enhancement, a suitable tactile sensation (tactile sensation related to the removed information) can be presented to the user based on the recorded information (tactile information).

[0044] Although an example of detecting the movement of the imaging device 100 using a motion sensor 110 has been described, the method of motion detection is not limited to this. For example, the movement of the imaging device 100 may be detected by detecting the time change of the video signal. For example, optical flow, which detects motion vectors from two consecutive image frames, can be used for the process of detecting the time change of the video signal. Furthermore, although an example of mechanical vibration isolation processing that moves at least one of the optical system 101 and the image sensor 102 has been described, the vibration isolation processing is not limited to this. For example, the vibration isolation processing may be electronic vibration isolation processing that deforms the image based on the motion detection result. Either mechanical vibration isolation processing or electronic vibration isolation processing may be performed, or both may be performed.

[0045] Figure 5 is a block diagram showing the configuration of the image processing unit 104 and the control unit 106 when using optical flow. The image processing unit 104 has an optical flow calculation unit 501 and an image correction unit 502, and the control unit 106 has an electronic vibration isolation tactile information generation unit 503. Elements related to audio signals are omitted in Figure 5.

[0046] The optical flow calculation unit 501 acquires the video signal of the current frame from the image sensor 102 and the video signal of the previous frame (the frame immediately preceding the current frame) from the recording unit 105. The optical flow calculation unit 501 extracts (detects) multiple feature points from each of the two acquired video signals and calculates the motion vector of each feature point between the two frames. The optical flow calculation unit 501 then outputs the group of motion vectors for the multiple feature points to the video correction unit 502 and the electronic vibration isolation tactile information generation unit 503 of the control unit 106.

[0047] The video correction unit 502 performs geometric deformation on the video signal of the current frame based on the motion vector group output from the optical flow calculation unit 501, so that the movement of feature points from the previous frame is reduced. The video correction unit 502 then outputs the geometrically deformed video signal to the recording unit 105. This improves the quality of the video signal input to the recording unit 105.

[0048] The electronic vibration isolation tactile information generation unit 503 determines a representative motion vector that represents the group of motion vectors output from the optical flow calculation unit 501. Then, the electronic vibration isolation tactile information generation unit 503 generates tactile information by using the representative motion vector as the vibration isolation amount described above and performing the same processing as the vibration isolation tactile information generation unit 202.

[0049] The imaging device 100 is not limited to a digital camera, but may be, for example, a smartphone or a tablet device. Furthermore, although an example of applying the present invention to an imaging device has been described, the present invention may also be applied to electronic devices other than imaging devices. For example, the present invention can be applied to controllers, smartphones, tablet devices, etc., connected to an imaging device. The present invention may also be applied to an editing device for editing content such as video and music.

[0050] While examples were described where tactile information indicates both the frequency and intensity of vibration, tactile information may indicate only one of these. Furthermore, while examples were described where tactile sensation is presented through the vibration of a device, the method of presenting tactile sensation is not particularly limited. For example, tactile sensation may be presented using temperature or ultrasound. Similarly, while examples were described where both video and audio signals are acquired (and recorded with enhanced quality), only one of these signals may be acquired (and recorded with enhanced quality).

[0051] While examples of associating tactile information with video and audio signals have been described, the information associated with video and audio signals is not limited to tactile information; it can be any information related to the amount of processing required for quality enhancement. For example, the information associated with video and audio signals may include information indicating the type and magnitude of vibrations removed by vibration isolation processing, information indicating the time variation of the output of a motion sensor or video signal, and information indicating the type and magnitude of noise signals removed by noise reduction processing.

[0052] <Embodiment 2> Embodiment 2 of the present invention will now be described. Note that the same configurations and processes as in Embodiment 1 will not be described below, and only configurations and processes different from those in Embodiment 1 will be described.

[0053] In Embodiment 1, tactile information (tactile information based on the amount of processing for quality enhancement) is generated and recorded based on information removed from the signal (video signal or audio signal) due to quality enhancement. In Embodiment 2, information that cannot be obtained from either the pre-enhancement video signal or the post-enhancement video signal (information that does not appear in either the pre-enhancement video signal or the post-enhancement video signal) is acquired regarding the video signal before quality enhancement. Then, the acquired information (tactile information that cannot be obtained from either the pre-enhancement video signal or the post-enhancement video signal) is recorded in association with the signal (video signal or audio signal).

[0054] Figure 6(A) is an external view of the imaging system according to Embodiment 2. As shown in Figure 6(A), in Embodiment 2, a sub-information acquisition unit 600 is attached to the imaging device 100. The sub-information acquisition unit 600 is, for example, an imaging device that images a range different from the imaging range of the imaging device 100 (such as the range of the imaging device 100 that includes the photographer).

[0055] Figure 6(B) is a block diagram showing the configuration of the sub-information acquisition unit 600. The sub-information acquisition unit 600 includes an optical system 601, an image sensor 602, an A / D conversion unit 603, an image processing unit 604, a recording unit 605, a control unit 606, a memory 607, and a communication unit 608. The optical system 601, image sensor 602, A / D conversion unit 603, recording unit 605, control unit 606, and memory 607 each have the same functions as the optical system 101, image sensor 102, A / D conversion unit 103, recording unit 105, control unit 106, and memory 107 of the imaging device 100.

[0056] The image processing unit 604, like the image processing unit 104, performs image processing such as simultaneous processing, white balance correction processing, and gamma processing on the video signal obtained by the A / D conversion unit 603. Furthermore, the image processing unit 604 detects the main subject (for example, the photographer's face) from the image represented by the video signal and estimates the main subject's facial expression and emotions from the image of the main subject. Various existing methods can be applied to body detection. Various existing methods can also be applied to estimate the facial expressions and emotions of the main subject. The image processing unit 604 generates tactile information based on the estimated facial expressions and emotions, and outputs the video signal and tactile information to the recording unit 605.

[0057] The communication unit 608 communicates with the imaging device 100 (a communication unit, not shown, that the imaging device 100 has). The communication unit 608 receives control signals from the imaging device 100 in response to user operations and outputs video signals and tactile information output from the image processing unit 604 to the recording unit 605 to the imaging device 100. The control unit 106 of the imaging device 100 controls the system to record the high-quality signal, the tactile information based on the amount of processing for high quality, the video signal acquired from the sub-information acquisition unit 600, and the tactile information acquired from the sub-information acquisition unit 600 in association with each other.

[0058] Figure 7 is a flowchart showing the processing performed by the image processing unit 604.

[0059] In S701, the image processing unit 604 acquires the video signal captured by the image sensor 602 and converted into a digital signal by the A / D conversion unit 603.

[0060] In S702, the image processing unit 604 detects the face of the main subject from the video (for example, a single frame image) represented by the video signal acquired in S701. For example, the image processing unit 604 detects one or more faces from the video and determines the largest detected face to be the face of the main subject.

[0061] In S703, the image processing unit 604 estimates the emotion of the main subject detected in S702. For example, the image processing unit 604 estimates the emotion of the main subject based on facial features such as the eyes, mouth, and nose. Facial features can be detected from the main subject's video (face image). The facial features of the main subject may be detected during the process of detecting the main subject's face, or they may be detected after detecting the main subject's face. The size of the facial features and the position of the feature points of the facial features (e.g., the position of the corners of the mouth) depend on the facial expression. For example, the size of the eyes and the position of the eye feature points differ when the eyes are open and when they are closed. Similarly, the size of the mouth and the position of the mouth feature points differ when the mouth is open and when it is closed. Assume that dictionary data is prepared in advance, which associates emotions with a set of features (feature vectors) of the sizes of multiple facial features and the positions (coordinates) of the feature points of multiple facial features. In that case, the emotion corresponding to the main subject's feature vector can be determined as the emotion of the main subject by referring to the dictionary data.

[0062] In S704, the image processing unit 604 generates tactile information based on the emotion estimated in S703. For example, it generates tactile information to present physical changes (e.g., changes in heart rate) associated with the emotion estimated in S703. For example, if the estimated emotion is "anxiety," it is presumed that the subject's heart is beating faster than usual. Therefore, if the estimated emotion is "anxiety," tactile information showing high-frequency vibrations (e.g., 120Hz) is generated.

[0063] As described above, according to Embodiment 2, information (second information) that cannot be obtained from the video signal of the imaging device 100 is acquired. The signal after quality enhancement, information regarding the amount of processing required for quality enhancement, and the second information are then recorded in association with each other. In this way, in addition to the effects of Embodiment 1, the effect of presenting the user with tactile sensations based on information that cannot be obtained from the video signal of the imaging device 100 is obtained.

[0064] In addition, while we have explained an example of estimating the emotions of the main subject using S703, it is also possible to estimate the heart rate of the main subject based on the time changes (video) of the main subject's face using S703. In step 4, tactile information may be generated based on the estimated heart rate.

[0065] Although an example has been described in which the sub-information acquisition unit 600 is an imaging device, the sub-information acquisition unit 600 may also be a position sensor that detects the imaging position, or a temperature sensor that detects the temperature at the imaging position (around the imaging device 100).

[0066] An example where the sub-information acquisition unit 600 is a temperature sensor will be described. When the sub-information acquisition unit 600 is a temperature sensor, the sub-information acquisition unit 600 estimates changes in the main subject's body based on the detected temperature and generates tactile information. For example, if the temperature is low enough that a person is shivering, it generates tactile information showing vibrations with a period (frequency) similar to that of shivering. If the temperature is high enough that a person is out of breath, it generates tactile information showing vibrations with a period similar to that of breathing when out of breath. Tactile information may also be generated so that the tactile sensation is presented at a temperature corresponding to the detected temperature. Tactile information may also be generated so that if the detected temperature is low, the tactile sensation is presented at a low temperature, and if the detected temperature is high, the tactile sensation is presented at a high temperature.

[0067] An example where the sub-information acquisition unit 600 is a position sensor will be described. When the sub-information acquisition unit 600 is a position sensor, the sub-information acquisition unit 600 estimates changes in the main subject's body based on the detected position and generates tactile information. For example, if the detected position is at a high altitude and shortness of breath is likely to occur, the sub-information acquisition unit 600 generates tactile information that shows vibrations with a period of about the same duration as the breathing period when shortness of breath occurs. Alternatively, temperature information corresponding to the detected position may be acquired from an external source, and tactile information may be generated based on the acquired temperature information.

[0068] An example of recording second information (information that cannot be obtained from the video signal of the imaging device 100) has been described, but information regarding the amount of processing for quality enhancement may be corrected based on the second information. For example, if the emotion estimated in S703 is "calm," the intensity of the tactile information based on the amount of processing for quality enhancement may be reduced. Then, the corrected information and the signal after quality enhancement may be recorded in relation to each other.

[0069] An example has been described in which the second information is tactile information that cannot be obtained from the video signal of the imaging device 100, but the second information does not have to be tactile information. For example, the second information may include information that includes at least one of the following: • A second video signal representing the image outside the imaging range of the imaging device 100. • Position information indicating the imaging position of the imaging device 100 • Temperature information indicating the ambient temperature around the imaging device 100 • Information showing the photographer's facial expression in the imaging device 100 • Information indicating the emotions of the photographer in the imaging device 100 • Information indicating the heart rate of the photographer captured by the imaging device 100 • Tactile information that cannot be obtained from the video signal of the imaging device 100 (for example, tactile information based on at least one of the photographer's facial expression, emotions, and heart rate)

[0070] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). Multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) may share the processing to control the entire device.

[0071] Furthermore, the above-mentioned processors are processors in a broad sense, including general-purpose processors and specialized processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Specialized processors include, for example, GPUs (Graphics Processing Units). These include ASICs (Application Specific Integrated Circuits) and PLDs (Programmable Logic Devices). Programmable logic devices include FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0072] Furthermore, although embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0073] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit that implements one or more functions.

[0074] This embodiment includes the following configurations, methods, programs, and media. (Composition 1) Acquisition means for acquiring at least one of a video signal and an audio signal, An improvement means for performing processing to improve the quality of the signal acquired by the acquisition means, A control means that records the signal after its quality has been improved by the improvement means and information regarding the amount of processing performed by the improvement means, in relation to each other. An electronic device characterized by having the following features. (Configuration 2) A generation means that generates tactile information included in the information relating to the processing amount based on the processing amount of the processing by the improvement means. Furthermore, it has The electronic device according to configuration 1, characterized by the features described above. (Composition 3) The aforementioned tactile information indicates at least one of the frequency and intensity of vibrations that reproduce the tactile sensation. The generating means controls at least one of the frequency and the intensity based on the processing amount of the processing by the improving means. The electronic device according to configuration 2, characterized by the features described above. (Composition 4) The aforementioned video signal is the video signal captured by the imaging device. The aforementioned improvement means includes vibration isolation processing means that removes vibrations caused by vibrations of the imaging device from the image represented by the video signal. An electronic device according to any one of configurations 1 to 3, characterized by the features described herein. (Composition 5) The vibration damping means removes vibrations from the image based on the output of a motion sensor provided in the imaging device or the time change of the video signal. The electronic device according to configuration 4, characterized by the features described above. (Composition 6) The information relating to the processing amount includes information indicating the time change of the output of the motion sensor or the video signal. The electronic device according to configuration 5, characterized by the features described herein. (Composition 7) The information relating to the processing amount includes information indicating the type and magnitude of the vibrations removed by the vibration isolation processing means. An electronic device according to any one of configurations 4 to 6, characterized by the features described above. (Composition 8) The improvement means includes noise reduction means for removing noise signals from the audio signal. An electronic device according to any one of configurations 1 to 5, characterized by the above. (Composition 9) The electronic device according to configuration 8, characterized in that the noise reduction means removes a preset type of noise signal from the audio signal. (Composition 10) The information relating to the processing amount includes information indicating the type and magnitude of the noise signal removed by the noise reduction means. The electronic device according to configuration 8 or 9, characterized by the above. (Composition 11) A second acquisition means for acquiring information relating to the aforementioned video signal that cannot be obtained from the aforementioned video signal. It further possesses, The control means is The signal after its quality has been improved by the improvement means, information regarding the amount of processing performed by the improvement means, and the information acquired by the second acquisition means are recorded in relation to each other, or Based on the information acquired by the second acquisition means, the information regarding the processing amount is corrected, and the corrected information and the signal after its quality has been improved by the improvement means are recorded in relation to each other. An electronic device according to any one of configurations 1 to 10, characterized by the above. (Composition 12) The aforementioned video signal is the video signal captured by the imaging device. The information that cannot be obtained from the video signal includes at least one of the following: a second video signal representing an image outside the imaging range of the video signal; position information indicating the imaging position of the video signal; temperature information indicating the temperature at the imaging position; information indicating the facial expression of the photographer in the video signal; information indicating the emotions of the photographer; information indicating the heart rate of the photographer; and tactile information that cannot be obtained from the video signal. The electronic device according to configuration 11, characterized by the features described above. (method) Acquisition step of acquiring at least one of the video signal and the audio signal, An improvement step which performs processing to improve the quality of the signal acquired in the acquisition step, A control step that records the signal after the quality has been improved in the improvement step and information regarding the amount of processing in the improvement step in relation to each other. A method for controlling electronic equipment, characterized by having the following features. (program) A program for causing a computer to function as one of the electronic devices described in any of configurations 1 to 12. (medium) A computer-readable storage medium that stores a program for causing the computer to function as one of the electronic devices described in configurations 1 to 12. [Explanation of symbols]

[0075] 100: Imaging device 102: Image sensor 104: Image processing unit 105: Recording unit 106: Control unit 111: Vibration isolation unit 112: Microphone 205 Audio Processing Unit 502 Video Correction Unit

Claims

1. Acquisition means for acquiring at least one of a video signal and an audio signal, An improvement means for performing processing to improve the quality of the signal acquired by the acquisition means, A control means that records the signal after its quality has been improved by the improvement means and information regarding the amount of processing performed by the improvement means, in relation to each other. An electronic device characterized by having the following features.

2. A generation means that generates tactile information included in the information relating to the processing amount based on the processing amount of the processing by the improvement means. Furthermore, it has The electronic device according to feature 1.

3. The aforementioned tactile information indicates at least one of the frequency and intensity of vibrations that reproduce the tactile sensation. The generating means controls at least one of the frequency and the intensity based on the processing amount of the processing by the improving means. The electronic device according to feature 2.

4. The aforementioned video signal is the video signal captured by the imaging device. The aforementioned improvement means includes vibration isolation processing means that removes vibrations caused by vibrations of the imaging device from the image represented by the video signal. The electronic device according to feature 1.

5. The vibration damping means removes vibrations from the image based on the output of a motion sensor provided in the imaging device or the time change of the video signal. The electronic device according to feature 4.

6. The information relating to the processing amount includes information indicating the time change of the output of the motion sensor or the video signal. The electronic device according to feature 5.

7. The information relating to the processing amount includes information indicating the type and magnitude of the vibrations removed by the vibration isolation processing means. The electronic device according to feature 4.

8. The improvement means includes noise reduction means for removing noise signals from the audio signal. The electronic device according to feature 1.

9. The electronic device according to claim 8, characterized in that the noise reduction means removes a preset type of noise signal from the audio signal.

10. The information relating to the processing amount includes information indicating the type and magnitude of the noise signal removed by the noise reduction means. The electronic device according to feature 8.

11. A second acquisition means for acquiring information relating to the aforementioned video signal that cannot be obtained from the aforementioned video signal. It further possesses, The control means is The signal after its quality has been improved by the improvement means, and the processing performed by the improvement means. The information relating to the quantity and the information obtained by the second acquisition means are recorded in relation to each other, or Based on the information acquired by the second acquisition means, the information regarding the processing amount is corrected, and the corrected information and the signal after its quality has been improved by the improvement means are recorded in relation to each other. The electronic device according to feature 1.

12. The aforementioned video signal is the video signal captured by the imaging device. The information that cannot be obtained from the video signal includes at least one of the following: a second video signal representing an image outside the imaging range of the video signal; position information indicating the imaging position of the video signal; temperature information indicating the temperature at the imaging position; information indicating the facial expression of the photographer in the video signal; information indicating the emotions of the photographer; information indicating the heart rate of the photographer; and tactile information that cannot be obtained from the video signal. The electronic device according to feature 11.

13. Acquisition step of acquiring at least one of the video signal and the audio signal, An improvement step which performs processing to improve the quality of the signal acquired in the acquisition step, A control step that records the signal after the quality has been improved in the improvement step and information regarding the amount of processing in the improvement step in relation to each other. A method for controlling electronic equipment, characterized by having the following features.

14. A program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 12.

15. A computer-readable storage medium storing a program for causing the computer to function as one of the means of the electronic device described in any one of claims 1 to 12.

Images