Image processing device, image processing method, photographing device, and program

Abstract

To provide an image processing device, an image processing method, a photographing device, and a program that can perform image processing to show changes over time.SOLUTION: An image processing device 101 includes: an image acquisition unit 100A that acquires a captured image to be processed from a memory unit that stores the captured image having first date / time information indicating the time of photographing; a date / time information acquisition unit 100B that acquires the first date / time information from the captured image to be processed and acquires second date / time information indicating when the captured image to be processed was acquired from the memory unit by the image acquisition unit; an elapsed date / time calculation unit 100C that compares the first date / time information with the second date / time information acquired by the date / time information acquisition unit and calculates an elapsed date / time from the time of photographing; and an image processing unit 100D that selects image processing based on a length of the elapsed date / time from among a plurality of pieces of image processing that vary according to the length of the elapsed date / time, and applies the selected image processing to the captured image to be processed.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, an image capturing device, and a program that change image processing depending on the length of time that has elapsed since the image was captured. [Background technology]

[0002] 2. Description of the Related Art Conventionally, techniques have been proposed for performing predetermined processing on images to make the images more interesting to viewers.

[0003] For example, Patent Document 1 describes a technique for inserting an effect corresponding to attribute information of each image content when outputting a plurality of image contents consecutively, and outputting the image contents consecutively. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-61032 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the changes that pique people's interest is the change that occurs over time. For example, the surface of leather products gradually changes over the years of use, and each change gives the product a different aesthetic appearance. Similarly, the taste of alcohol gradually changes over storage time, and drinkers can enjoy the change in taste.

[0006] The above-mentioned Patent Document 1 describes a technology in which the recording date and time of content is compared with the playback date and time, and if the value is equal to or greater than a threshold, a sepia effect is selected (Patent Document 1: paragraph 0055). However, this change is a one-time event, and is not a gradual change over time.

[0007] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing device, an image processing method, an image capturing device, and a program that are capable of processing images that change over time. [Means for solving the problem]

[0008] To achieve the above-mentioned object, one aspect of the present invention is an image processing device that includes an image acquisition unit that acquires a captured image to be processed from a memory unit in which a captured image having first date and time information indicating the time of shooting is stored; a date and time information acquisition unit that acquires the first date and time information from the captured image to be processed and acquires second date and time information indicating the date and time at which the captured image to be processed was acquired from the memory unit by the image acquisition unit; an elapsed date and time calculation unit that compares the first date and time information and the second date and time information acquired by the date and time information acquisition unit to calculate the date and time elapsed since the time of shooting; and an image processing unit that selects an image processing from a plurality of image processing methods that change depending on the length of the elapsed date and time, based on the length of the elapsed date and time, and performs the selected image processing on the captured image to be processed.

[0009] According to this aspect, the elapsed time from the time of shooting is calculated, and image processing is performed based on the calculated length of the elapsed time from among a plurality of image processing methods that change depending on the length of the elapsed time. As a result, this aspect allows image processing that changes over time to be performed on the captured image.

[0010] Preferably, the image processing device includes a display control unit that causes the display unit to display the captured image to be processed, which has been subjected to image processing by the image processing unit.

[0011] Preferably, the image processing device further includes a print control unit that causes the print unit to print the captured image to be processed, which has been subjected to image processing by the image processing unit.

[0012] Preferably, the captured image of the processing target that has been subjected to image processing by the image processing unit and the captured image of the processing target before being subjected to image processing by the image processing unit are stored in the storage unit.

[0013] Preferably, the photographed image to be processed, which has been subjected to image processing by the image processing unit, is not stored in the storage unit.

[0014] Preferably, the image processing device includes a relationship input unit that receives an input of a correspondence relationship on a time axis between a plurality of image processes and the length of elapsed date and time.

[0015] Preferably, the image processing section changes the overall color of the photographed image to be processed in accordance with the length of the elapsed date and time.

[0016] Preferably, the image processing section changes the color of a part of a subject in the captured image to be processed in accordance with the length of the elapsed date and time.

[0017] Preferably, the image processing section lightens the background color of the photographic image to be processed in accordance with the length of the elapsed date and time.

[0018] Preferably, the image processing section changes a part of the color or a part of the hue of the photographed image to be processed in accordance with the length of the elapsed date and time.

[0019] Preferably, the image processing section changes the degree of blurring of a partial area of ​​the captured image to be processed in accordance with the length of the elapsed date and time.

[0020] Preferably, the image processing unit changes the degree of blurring of the foreground or background of a person in the captured image in accordance with the length of the elapsed date and time.

[0021] Preferably, the image processing section changes the decorative image to be added to the photographic image to be processed in accordance with the elapsed date and time.

[0022] Another aspect of the present invention is an imaging device equipped with the image processing device described above.

[0023] Preferably, the photographing device includes a photographing mode setting unit that sets one photographing mode from among a plurality of photographing modes, and the image processing unit changes a plurality of image processings that change according to the length of the elapsed date and time in accordance with the photographing mode set by the photographing mode setting unit.

[0024] Another aspect of the image processing method of the present invention includes the steps of acquiring a captured image to be processed from a memory unit in which a captured image having first date and time information indicating the time of shooting is stored; acquiring the first date and time information from the captured image to be processed and acquiring second date and time information indicating the date and time at which the captured image to be processed was acquired from the memory unit; comparing the first date and time information with the second date and time information to calculate the date and time elapsed since the time of shooting; and selecting an image processing from a plurality of image processing methods that change depending on the length of the elapsed date and time, based on the length of the elapsed date and time, and performing the selected image processing on the captured image to be processed.

[0025] Another aspect of the present invention is a program that causes a computer to execute image processing steps including the steps of acquiring a captured image to be processed from a memory unit in which a captured image having first date and time information indicating the time of shooting is stored; acquiring the first date and time information from the captured image to be processed and acquiring second date and time information indicating the date and time at which the captured image to be processed was acquired from the memory unit; comparing the first date and time information with the second date and time information to calculate the date and time elapsed since the time of shooting; and selecting an image processing method from among a plurality of image processing methods that change depending on the length of the elapsed date and time, based on the length of the elapsed date and time, and performing the selected image processing method on the captured image to be processed. [Effects of the Invention]

[0026] According to the present invention, the elapsed date and time from the time of shooting is calculated, and image processing is performed based on the calculated length of the elapsed date and time from among multiple image processing methods that change depending on the length of the elapsed date and time, so that image processing that changes over time can be performed on the captured image. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a front perspective view showing an example of a digital camera with a printer. [Figure 2] FIG. 2 is a rear perspective view of the digital camera with a printer. [Figure 3] FIG. 3 is a front view of the instant film. [Figure 4] FIG. 4 is a rear view of the instant film. [Figure 5] FIG. 5 is a block diagram showing the main parts of the electrical configuration of the digital camera with a printer. [Figure 6] FIG. 6 is a diagram illustrating an example of the functional configuration of the image processing device. [Figure 7] FIG. 7 is a conceptual diagram illustrating image processing of aging changes performed by the image processing unit. [Figure 8] FIG. 8 is a flowchart showing the image processing method. [Figure 9] FIG. 9 is a diagram illustrating image processing in which the overall color of a captured image changes over time. [Figure 10] FIG. 10 is a diagram illustrating image processing for age-related changes in the color of a part of a subject in a photographed image. [Figure 11] FIG. 11 is a diagram illustrating image processing in which the same color or hue of a captured image changes over time. [Figure 12] FIG. 12 is a diagram illustrating image processing in which the degree of blur in a part of a captured image changes over time. [Figure 13] FIG. 13 is a diagram illustrating image processing in which a decorative image added to a photographed image changes over time. [Figure 14] FIG. 14 is a diagram showing an example of image processing using sounds in the shooting environment. [Figure 15] FIG. 15 is a diagram showing an example of processing using information on the shooting location. [Figure 16] FIG. 16 is a diagram showing an example of the emotion setting screen. [Figure 17] FIG. 17 is a diagram showing an example of image processing according to emotions. DETAILED DESCRIPTION OF THE INVENTION

[0028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image processing device, an image processing method, an image capturing device, and a program according to the present invention will now be described with reference to the accompanying drawings.

[0029] <Digital camera with printer> The digital camera with printer (photographing device) 10 equipped with the image processing device 101 (see FIG. 6) of the present invention is a digital camera with a built-in printer and has the function of printing captured images (photographed images) on the spot. The digital camera with printer 10 uses an instant film pack to print onto instant film. The digital camera with printer 10 of this embodiment also has a sound recording function, and can record audio in association with captured images.

[0030] Appearance Configuration Fig. 1 is a front perspective view showing an example of a digital camera with a printer, and Fig. 2 is a rear perspective view of the digital camera with a printer shown in Fig. 1.

[0031] 1 and 2, the digital camera with printer 10 has a portable camera body 12. The camera body 12 has a vertically elongated rectangular parallelepiped shape that is thin in the front-to-rear direction and longer in the vertical direction than in the horizontal direction.

[0032] As shown in FIG. 1, the front side of the camera body 12 is equipped with a photographing lens 14, a release button 16, a sound recording button 18, a flash window 20, etc. One side of the camera body 12 is equipped with a power button 22a, a menu button 22b, an OK button 22c, a mode switch button 22d, a microphone hole 24, a speaker hole 26, etc. The release button 16 is a button used to instruct recording of a photographed image. The power button 22a is used to turn the power of the printer-equipped digital camera 10 on and off. The menu button 22b is used to call up a menu screen. The OK button 22C is used to indicate OK. The mode switch button 22d is used to switch between auto print mode and manual print mode in the photographing mode. The photographing mode can be changed by calling up the menu screen with the menu button 22b, displaying a photographing mode change screen, and pressing the OK button 22C to confirm. Each photographing mode has its own appropriate photographing conditions. The photographing modes include, for example, portrait mode and landscape mode. In this case, the menu button 22b and the OK button 22c constitute a shooting mode setting section.

[0033] As shown in FIG. 2, the rear side of the camera body 12 is provided with a touch panel display (display unit) 28, a film cover 30, and various operation buttons. The film cover 30 is a cover that opens and closes the film loading chamber. The operation buttons include a joystick 32a, a print button 32b, a playback button 32c, a cancel button 32d, and the like. The print button 32b is a button that commands printing. The playback button 32c is a button that commands switching to playback mode. The cancel button 32d is a button that commands cancellation of an operation.

[0034] 1 and 2, a film ejection opening 34 is provided on the top surface of the camera body 12. Printed instant film is ejected from this film ejection opening 34.

[0035] [Printer configuration of a digital camera with a printer] The digital camera with printer 10 has a film loading chamber (not shown), a film feeding mechanism 52, a film transport mechanism 54, a print head 56, and other components of the printer section (see Figure 5). An instant film pack, which has a case containing multiple sheets of instant film, is loaded into the film loading chamber. Figure 3 is a front view of instant film 42, and Figure 4 is a rear view of instant film 42. In Figures 3 and 4, the direction indicated by arrow F is the direction in which instant film 42 is used, and instant film 42 is transported in the direction indicated by arrow F. Therefore, when loaded into digital camera with printer 10, the direction indicated by arrow F is the direction in which instant film 42 is ejected.

[0036] The instant film 42 is a self-developing instant film in the shape of a rectangular card. The back side of the instant film 42 is configured as an exposure surface 42a, and the front side is configured as an observation surface 42b. The exposure surface 42a is the surface on which an image is recorded by exposure, and the observation surface 42b is the surface on which the recorded image is observed.

[0037] As shown in FIG. 3, an observation area 42h is provided on the observation surface 42b of the instant film 42. Furthermore, as shown in FIG. 4, an exposure area 42c, a pod portion 42d, and a trap portion 42f are provided on the exposure surface 42a of the instant film 42. After exposure, the instant film 42 is developed by spreading the developing solution in the pod portion 42d in the exposure area 42c. A developing solution pod 42e containing the developing solution is built into the pod portion 42d. The developing solution in the pod portion 42d is squeezed out of the pod portion 42d by passing the instant film 42 between a pair of rollers and spread in the exposure area 42c. Any remaining developing solution during the spreading process is captured in the trap portion 42f. An absorbent 42g is built into the trap portion 42f.

[0038] The instant print pack is loaded into a film loading chamber (not shown) located inside the camera body 12. When printing, the film is advanced one sheet at a time by a claw (not shown) of the film advance mechanism 52, and transported by rollers (not shown) of the film transport mechanism 54. During the transport process, a pair of spreading rollers (not shown) crushes the pod portion 42d of the instant film 42, spreading the developing solution. The print head 56 is composed of a line-type exposure head, and irradiates the exposure surface 42a of the instant film 42 transported by the film transport mechanism 54 with printing light, line by line, to record the captured image on the instant film 42 in a single pass. A frame 42i is provided around the observation area 42h, and the captured image is displayed inside the frame 42i.

[0039] [Electrical configuration of a digital camera with a printer] FIG. 5 is a block diagram showing the main parts of the electrical configuration of the digital camera with printer 10. As shown in FIG.

[0040] As shown in the figure, digital camera with printer 10 includes a taking lens 14, a lens driver 62, an image sensor 64, an image sensor driver 66, an analog signal processor 68, a digital signal processor 70, a memory (storage unit) 72, a memory controller 74, a display 28, a display controller 76, a communication unit 78, and an antenna 80. Digital camera with printer 10 also includes a film feed driver 82, a film transport driver 84, a head driver 86, a flash 88, a flash emission controller 90, a microphone 92, a speaker 94, an audio signal processor 96, a clock unit 97, an operation unit 98, and a camera controller 100.

[0041] The photographing lens 14 forms an optical image of a subject on the light receiving surface of the image sensor 64. The photographing lens 14 has a focus adjustment function and is equipped with an aperture and a shutter (not shown). The lens driving unit 62 includes a motor and its drive circuit that drives the focus adjustment function of the photographing lens 14, a motor and its drive circuit that drives the aperture, and a motor and its drive circuit that drives the shutter, and operates the focus adjustment mechanism, aperture, and shutter in response to commands from the camera control unit 100.

[0042] The image sensor 64 is composed of a two-dimensional solid-state imaging element such as a CCD image sensor (CCD: Charge Coupled Device) or a CMOS image sensor (CMOS: Complementary Metal Oxide Semiconductor). The image sensor 64 has an imaging area with an aspect ratio corresponding to the printable area of ​​the instant film being used. The image sensor driver 66 includes a drive circuit for the image sensor 64 and operates the image sensor 64 in response to commands from the camera controller 100.

[0043] The analog signal processing unit 68 takes in the analog image signal for each pixel output from the image sensor 64, performs signal processing (for example, correlated double sampling processing, amplification processing, etc.), digitizes it, and outputs it.

[0044] The digital signal processing unit 70 takes in the digital image signal output from the analog signal processing unit 68 and performs signal processing (e.g., gradation conversion processing, white balance correction processing, gamma correction processing, synchronization processing, YC conversion processing, etc.) to generate image data.

[0045] The memory 72 is a non-transitory recording medium, such as a memory card, that stores image data (captured images) and audio data obtained by capturing images. The memory 72 is an example of a storage unit. The memory controller 74 reads and writes data from and to the memory 72 under the control of the camera control unit 100.

[0046] Display 28 is configured, for example, by a liquid crystal display (LCD), an organic electro-luminescence display (OLED), etc. Alternatively, display 28 may be configured by a plasma display, a field emission display (FED), electronic paper, etc. Display controller 76 causes display 28 to display an image under the control of camera control unit 100.

[0047] The communication unit 78, under the control of the camera control unit 100, wirelessly communicates with other digital cameras 10 with printers (other devices) via the antenna 80. The communication unit 78 can directly communicate with other devices over short distances using short-distance wireless communication such as the NFC standard (Near Field Communication) or Bluetooth (registered trademark). The communication unit 78 can also connect to an information and communication network such as the Internet via a Wi-Fi spot (Wi-Fi: registered trademark) or the like, and communicate with other digital cameras 10 with printers (other devices) regardless of distance.

[0048] The film delivery drive unit 82 includes a motor that drives a claw (claw-shaped member) (not shown) of the film delivery mechanism 52 and its drive circuit, and drives the motor to operate the claw under the control of the camera control unit 100.

[0049] The film conveying drive unit 84 includes a motor and its drive circuit that drives a pair of conveying rollers (not shown) of the film conveying mechanism 54, as well as a motor and its drive circuit that drives a pair of unfolding rollers (not shown), and under the control of the camera control unit 100, drives the motors of the pair of conveying rollers and the motors of the pair of unfolding rollers to operate the pair of conveying rollers and the pair of unfolding rollers.

[0050] The head drive unit 86 includes a drive circuit for the print head 56 and drives the print head 56 under the control of the camera control unit 100 .

[0051] The strobe 88 has a light source such as a xenon tube or an LED (Light Emitting Diode), and emits strobe light onto the subject by emitting light from the light source. The strobe light is emitted from a strobe light-emitting window 20 (see FIG. 1) provided on the front of the camera body 12. The strobe light-emitting control unit 90 includes a drive circuit for the strobe 88, and causes the strobe 88 to emit light in response to commands from the camera control unit 100.

[0052] The microphone 92 collects external sound through a microphone hole 24 (see FIG. 2) provided in the camera body 12. The speaker 94 outputs sound to the outside through a speaker hole 26 provided in the camera body 12. The audio signal processor 96 processes the audio signal input from the microphone 92, digitizes it, and outputs it. The audio signal processor 96 also processes audio data provided by the camera control unit 100, and outputs it from the speaker 94. The clock unit 97 holds date and time information, and the camera control unit 100 references this information to set the shooting time (date and time).

[0053] The operation unit 98 includes various operation members such as the release button 16, the sound recording button 18, the power button 22a, the menu button 22b, the OK button 22c, the joystick 32a, the print button 32b, the play button 32c, and the cancel button 32d, as well as their signal processing circuits, and outputs signals based on the operation of each operation member to the camera control unit 100.

[0054] The camera control unit 100 is a control unit that controls the overall operation of the printer-equipped digital camera 10. The camera control unit 100 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and EEPROM (Electronically Erasable and Programmable Read Only Memory). The camera control unit 100 is a computer configured with these components, such as the CPU, and realizes the various functions described below by executing control programs.

[0055] <Functional configuration of image processing device> The digital camera 10 with a built-in printer is equipped with an image processing device 101 , and the main functions of the image processing device 101 are provided in a camera control unit 100 .

[0056] 6 is a diagram showing an example of the functional configuration of an image processing device 101 provided in the camera control unit 100. The image processing device 101 includes an image acquisition unit 100A, a date and time information acquisition unit 100B, an elapsed date and time calculation unit 100C, an image processing unit 100D, a display control unit 100E, and a print control unit 100F.

[0057] The functions of each unit of the camera control unit 100 (image processing device 101) described above can be realized using various processors and recording media. The various processors include, for example, a CPU, which is a general-purpose processor that executes software (programs) to realize various functions. The various processors described above also include programmable logic devices (PLDs) such as a GPU (Graphics Processing Unit) and an FPGA (Field Programmable Gate Array), which are processors specialized for image processing. A programmable logic device is a processor whose circuit configuration can be changed after manufacturing. Furthermore, the various processors described above also include dedicated electrical circuits, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically to execute specific processing.

[0058] The functions of each unit may be realized by a single processor, or by multiple processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Also, a single processor may have multiple functions. Examples of multiple functions configured by a single processor include: a first configuration, as typified by a computer, in which a single processor is configured by combining one or more CPUs and software, and this processor realizes multiple functions; a second configuration, as typified by a system-on-chip (SoC), in which a processor is used to realize the functions of the entire system on a single IC (Integrated Circuit) chip; and various functions are thus configured as hardware structures using one or more of the various processors described above. Furthermore, the hardware structures of these various processors are, more specifically, electrical circuits combining circuit elements such as semiconductor devices. These electrical circuits may be electrical circuits that realize the above-mentioned functions using logical operations such as logical sum, logical product, logical negation, exclusive OR, and combinations of these.

[0059] When the above-mentioned processor or electrical circuit executes software (program), the computer-readable code of the software to be executed (various processors and electrical circuits, and / or combinations thereof, that constitute the camera control unit 100 (image processing device 101)) is stored in a non-transitory recording medium such as a ROM, and the computer references the software. The software stored in the non-transitory recording medium includes programs for capturing and synthesizing captured images and data used during execution. The non-transitory recording medium on which the code is recorded may not be a ROM, but may be various types of magneto-optical recording devices, semiconductor memory, etc. When processing using the software, for example, RAM is used as a temporary storage area.

[0060] The image acquisition unit 100A acquires a photographed image to be processed from photographed images stored in the memory 72. The memory 72 stores a plurality of photographed images taken with the printer-equipped digital camera 10. The photographed image has first date and time information indicating the time of photographing. Here, the first date and time information is information including at least one of the year, month, day, and time when the photographed image was taken. The photographed image to be processed can be selected from the plurality of photographed images stored in the memory 72 in various ways. For example, the plurality of photographed images stored in the memory 72 are displayed in a list on the display 28, and the user selects the photographed image to be processed from the displayed list. Note that one or more photographed images to be processed may be selected, or all of the photographed images stored in the memory 72 may be selected.

[0061] The date and time information acquisition unit 100B acquires first date and time information and second date and time information for the captured image acquired by the image acquisition unit 100A. Here, the second date and time information is information indicating the date and time when the captured image to be processed was acquired by the image acquisition unit 100A from the memory 72. When the captured image is acquired by the image acquisition unit 100A, the image processing device 101 immediately performs image processing, so the second date and time information can be considered to be substantially the same as information indicating the date and time when the image processing is performed.

[0062] The elapsed date and time calculation unit 100C compares the first date and time information and the second date and time information acquired by the date and time information acquisition unit 100B, and calculates the elapsed date and time from the time of shooting to the time the captured image is acquired by the image acquisition unit 100A. For example, if the first date and time information is 9:04 AM on December 20, 2018, and the second date and time information is 9:06 AM on December 21, 2018, the elapsed date and time calculation unit 100C calculates the elapsed date and time to be 1 day and 2 minutes.

[0063] The image processing unit 100D performs image processing that changes depending on the length of elapsed time on the captured image selected as the processing target. That is, the image processing unit 100D performs image processing that changes over time on the captured image. Specifically, the image processing unit 100D selects an image processing based on the length of elapsed time from among a plurality of image processing methods that change depending on the length of elapsed time, and performs the selected image processing on the captured image to be processed.

[0064] FIG. 7 is a conceptual diagram illustrating image processing for aging performed by the image processing unit 100D. FIG. 7 shows an example of the relationship between three preset image processes (image processing A, image processing B, and image processing C) (reference numeral 105) performed by the image processing unit 100D and elapsed time T (reference numeral 107). Image processing A is performed when the elapsed time T is from Ta to Tab, image processing B is performed when the elapsed time T is from Tab to Tbc, and image processing C is performed when the elapsed time T is from Tbc to Tc. For example, image processing A is selected when the elapsed time T1 is reached, image processing B is selected when the elapsed time T2 is reached, and image processing C is selected when the elapsed time T3 is reached. Note that in the example shown in FIG. 7, the period Ta to Tab for image processing A, the period Tab to Tbc for image processing B, and the period Tbc to Tc for image processing C are constant, but this is not limiting. The user may arbitrarily determine the correspondence relationship on the time axis between multiple image processes and the length of elapsed time. In the example shown in FIG. 7, the user can arbitrarily set the period Ta to Tab for image processing A, the period Tab to Tbc for image processing B, and the period Tbc to Tc for image processing C. In this case, the user inputs the correspondence relationship via the operation unit (relationship input unit) 98. Note that image processing A, image processing B, and image processing C are mutually different image processing, and the captured image changes over time as the processing content changes from image processing A to image processing B to image processing C. Furthermore, the processing content of the multiple image processings that are set in advance may be changed according to the shooting mode or shooting conditions. For example, the processing content of image processing A, image processing B, and image processing C may be changed for each shooting mode, or the processing content of image processing A, image processing B, and image processing C may be changed for each shooting condition.

[0065] The captured image to be processed by the image processing unit 100D may be stored in the memory 72, or the processed captured image may not be stored in the memory 72. If the processed captured image is stored in the memory 72, the user can enjoy the captured image that has changed over time even after a certain amount of time has passed. If the processed captured image is not stored in the memory 72, the user can enjoy the changes over time by printing the processed captured image, etc. Note that the captured image before image processing that is selected as the processing target may also be stored in the memory 72, or may not be stored in the memory 72.

[0066] 6, the display control unit 100E displays the photographed image to be processed, which has been subjected to image processing by the image processing unit 100D, on the display 28. The user can enjoy observing the changes over time in the photographed image by checking the displayed photographed image after image processing.

[0067] The print control unit 100F controls printing of the captured image that has undergone image processing. For example, the print control unit 100F automatically or in response to a user instruction causes the print unit to print the captured image that has undergone image processing by the image processing unit 100D. The user can enjoy observing the changes in the captured image over time by checking the printed captured image after image processing.

[0068] Next, a description will be given of an image processing method (image processing steps) using the image processing device 101. Fig. 8 is a flowchart showing the image processing method.

[0069] First, the image acquisition unit 100A acquires the selected captured image to be processed (step S10). Then, the date and time information acquisition unit 100B acquires first date and time information and second date and time information of the captured image acquired by the image acquisition unit 100A (step S11). Next, the elapsed date and time calculation unit 100C calculates the elapsed date and time from the first date and time information and the second date and time information (step S12). Then, the image processing unit 100D performs image processing based on the length of the calculated elapsed date and time (step S13).

[0070] The above-described configurations and functions can be realized by any hardware, software, or a combination of both. For example, the present invention can be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-transitory recording medium) on which such a program is recorded, or a computer on which such a program can be installed.

[0071] As described above, according to this embodiment, the elapsed date and time from the time of shooting is calculated, and image processing is performed based on the calculated length of the elapsed date and time from among multiple image processing operations that change depending on the length of the elapsed date and time, so that image processing that changes over time can be performed on the captured image.

[0072] <Example of image processing for aging> Next, the image processing of aging carried out by the image processing unit 100D will be described. Various modes can be adopted for the image processing of aging carried out by the image processing unit 100D. Specific examples of image processing of aging will be described below.

[0073] [First example] In a first example, the image processing unit 100D changes the overall color of the captured image to be processed according to the length of the elapsed time. For example, the image processing unit 100D lightens, darkens, or discolors the overall color of the captured image according to the length of the elapsed time.

[0074] FIG. 9 is a diagram illustrating image processing in which the overall color of a captured image changes over time.

[0075] Reference numeral 201 indicates a photographed image P1 when the elapsed date and time is T1, reference numeral 203 indicates a photographed image P1 when the elapsed date and time is T2, and reference numeral 205 indicates a photographed image P1 when the elapsed date and time is T3. Note that the length of the elapsed date and time is T1. <T2<T3である。

[0076] 9, the image processing unit 100D performs image processing to change the overall color of the captured image depending on the length of the elapsed date and time. Specifically, the overall color of the image (e.g., sepia) is gradually darkened in the order of the captured image P1 with reference numeral 201, the captured image P1 with reference numeral 203, and the captured image P1 with reference numeral 205. In this way, by performing image processing to change the overall color of the captured image depending on the elapsed date and time, the captured image P1 can be made to change over time.

[0077] [Second example] In a second example, the image processing unit 100D changes the color of a part of a subject in a captured image to be processed according to the length of elapsed time. For example, the image processing unit 100D changes the background color of the captured image to be processed by lightening, darkening, or discoloring according to the length of elapsed time.

[0078] FIG. 10 is a diagram illustrating image processing for age-related changes in the color of a part of a subject in a photographed image.

[0079] Reference numeral 207 indicates a photographed image P2 when the elapsed date and time is T1, reference numeral 209 indicates a photographed image P2 when the elapsed date and time is T2, and reference numeral 211 indicates a photographed image P2 when the elapsed date and time is T3.

[0080] 10, the image processing unit 100D performs image processing to gradually lighten the color of the background 213 of the photographed image P2 in accordance with the length of the elapsed date and time. Specifically, the color of the background 213 is gradually lightened in the order of the photographed image P2 indicated by the reference numeral 207, the photographed image P2 indicated by the reference numeral 209, and the photographed image P2 indicated by the reference numeral 211. In this way, by lightening the color of the background 213 in accordance with the elapsed date and time, it is possible to cause the photographed image P2 to change over time.

[0081] [Third example] In a third example, the image processing unit 100D changes a part of the color or a part of the hue of the captured image to be processed according to the length of the elapsed date and time. Here, a part of the color refers to the same color in the captured image, and a part of the hue refers to the same hue in the captured image. For example, the same color refers to a color with the same RGB signal value, and the same hue refers to a hue that belongs to the same group when the color circle is divided into six equal parts.

[0082] FIG. 11 is a diagram illustrating image processing in which the same color or hue of a captured image changes over time.

[0083] Reference numeral 215 denotes a photographed image P3 when the elapsed date and time is T1, reference numeral 217 denotes a photographed image P3 when the elapsed date and time is T2, and reference numeral 219 denotes a photographed image P3 when the elapsed date and time is T3. Note that the leaves 221 of the photographed image P3 have the same color.

[0084] 11, the image processing unit 100D performs image processing to change the color of the leaves 221 depending on the length of the elapsed date and time. Specifically, the color of the leaves 221 is gradually darkened in the order of the captured image P3 with reference numeral 215, the captured image P3 with reference numeral 217, and the captured image P3 with reference numeral 219. In this way, by changing the color of the leaves 221 depending on the elapsed date and time, it is possible to cause the captured image P3 to change over time.

[0085] [Example 4] In a fourth example, the image processing unit 100D changes the degree of blurring of a portion of the captured image to be processed according to the length of elapsed time and date. For example, the image processing unit 100D changes the degree of blurring of the foreground or background of a person in the captured image according to the length of elapsed time and date. In this way, by changing the degree of blurring of the foreground or background of the captured image, it is possible to artificially change the depth of field of the captured image. Note that the image processing unit 100D performs blurring processing by detecting a portion of the area for which the degree of blurring is to be changed, or an area other than the portion of the area for which the degree of blurring is to be changed. For example, the image processing unit 100D detects the area of ​​a person (face) and performs blurring processing on an area other than the detected area of ​​the person (for example, the background).

[0086] FIG. 12 is a diagram illustrating image processing in which the degree of blur in the background of a captured image changes over time.

[0087] Reference numeral 223 denotes a photographed image P4 when the elapsed date and time is T1, reference numeral 225 denotes a photographed image P4 when the elapsed date and time is T2, and reference numeral 227 denotes a photographed image P4 when the elapsed date and time is T3. The photographed image P4 has a person 229 as the main subject and a mountain 231 in the background.

[0088] In the case of elapsed date and time T1 (reference numeral 223), no changes are made to the captured image P4 by image processing, and no blurring occurs in the area of ​​mountain 231. In the case of elapsed date and time T2 (reference numeral 225), the person 229 is in focus, but the mountain 231 in the background has been blurred by image processing. Note that in the figure, the blurring is indicated by double lines, and the size of the gap between the double lines indicates the degree of blurring. In the case of elapsed date and time T3 (reference numeral 227), the person 229 is in focus, but the mountain 231 in the background has been blurred by image processing. Furthermore, in the case of elapsed date and time T3 (reference numeral 227), image processing has been performed to increase the degree of blurring compared to elapsed time T2. Here, the degree of blurring refers to the degree of blurring, and can be adjusted from small to large by image processing. In this way, by increasing the degree of blur of the mountain 231 in the background of the photographed image P4 according to the elapsed date and time, the depth of field of the photographed image P4 can be made shallower in an artificial manner, and the photographed image P4 can be made to change over time.

[0089] [Fifth Example] In a fifth example, the image processor 100D changes the decorative image added to the captured image to be processed according to the elapsed date and time. Here, the decorative image is an image added to the captured image by image processing, such as cracks, cinematic vertical lines, and blemishes.

[0090] FIG. 13 is a diagram illustrating image processing in which a decorative image added to a photographed image changes over time.

[0091] Reference numeral 235 indicates a photographed image P5 when the elapsed date and time is T1, reference numeral 237 indicates a photographed image P5 when the elapsed date and time is T2, and reference numeral 239 indicates a photographed image P5 when the elapsed date and time is T3.

[0092] At elapsed date and time T1 (reference number 235), the crack 241, which is a decorative image, has not yet been added to the photographed image P5. At elapsed date and time T2 (reference number 237), the crack 241 is added to the photographed image P5 by image processing. At elapsed date and time T3 (reference number 239), the crack 241 is added to the photographed image P5 by image processing.

[0093] 13, the cracks 241 change from the elapsed date and time T2 (reference symbol 237) to the elapsed date and time T3 (reference symbol 239). Specifically, the cracks 241 become longer and the number of cracks 241 increases from the elapsed date and time T2 (reference symbol 237) to the elapsed date and time T3 (reference symbol 239). In this way, by changing the cracks 241 added to the photographed image P5 depending on the elapsed date and time, it is possible to cause the photographed image P5 to change over time.

[0094] A specific example of image processing of aging that is performed by the image processing unit 100D has been described above, but the image processing of aging that is performed by the image processing unit 100D is not limited to the above-mentioned specific example.

[0095] <Other> In this embodiment, we have described an aspect in which image processing is changed depending on the date and time that has elapsed since the image was taken, but there are other types of processing that can arouse the interest of people who view the image. Other examples will be described below.

[0096] [Sound-based processing] By utilizing sounds related to the shooting environment or the captured image, image processing can be performed on the captured image. The digital camera with printer 10 is equipped with a microphone 92, which can collect sounds in the shooting environment. Therefore, the microphone 92 collects sounds in the shooting environment, and the camera control unit 100 stores the input sounds of the shooting environment together with the captured image in memory 72. The image processing unit 100D uses the sounds of the shooting environment stored in memory 72 to perform image processing on the captured image.

[0097] FIG. 14 is a diagram showing an example of image processing using sounds in the shooting environment, showing images shot at a concert venue.

[0098] Captured image P6 is an image captured at a concert venue. For example, the microphone 92 collects sounds from the concert venue, and image processing is applied to the captured image. In captured image P6, the microphone 92 collects sounds to identify the venue as a concert venue, and musical notes 250 are added to the captured image P6. Furthermore, image processing unit 100D performs soft image processing when sounds from the shooting environment of a classical concert are collected, and performs image processing with strong edge emphasis and contrast when sounds from the shooting environment of a heavy or hard concert are collected. Furthermore, when sounds from a crowd are collected as sounds from the shooting environment, image processing unit 100D performs image processing to add noise to the captured image.

[0099] [Processing using shooting location information] By using the information on the shooting location, the captured image can be subjected to image processing according to the shooting location and information related to the captured image can be added. The digital camera with printer 10 is equipped with a GPS (Global Positioning System) function to detect the shooting location and perform various processes on the captured image according to the shooting location. For example, when shooting at a concert venue, the image processing unit 100D performs image processing on the captured image to give it a live feel.

[0100] FIG. 15 is a diagram showing an example of processing using information on the photographing location, and is a diagram showing a photographed image of a lion.

[0101] Image P7 is a photograph of a lion in front of a lion cage at a zoo. If digital camera with printer 10 is equipped with a GPS function, the GPS function will detect that image P7 was taken in front of a lion cage. The lion's roar is then stored in memory 72 along with image P7, and a description of the lion (audio or text) is also stored along with image P7.

[0102] [Emotional processing] The user can input their own emotion and perform image processing on the captured image accordingly. For example, an emotion setting screen is displayed on the display 28 of the printer-equipped digital camera 10, and the user inputs their own emotion via the emotion setting screen. The image processing unit 100D performs image processing on the captured image according to the input emotion.

[0103] FIG. 16 is a diagram showing an example of an emotion setting screen displayed on display 28 of digital camera with printer 10. The emotion setting screen shown in FIG. 16 is circular and shows four emotions (normal (H), lucky (I), sad (J), and surprised (K)). The user inputs an emotion by operating each lever L on the emotion setting screen. Image processor 100D performs image processing on the captured image according to the input emotion.

[0104] FIG. 17 is a diagram showing an example of image processing according to emotions.

[0105] As shown in Fig. 17, photographed image P8 is subjected to image processing in accordance with the emotion input by the user on the emotion setting screen (Fig. 16). Note that Fig. 17 also displays emotion input value display 252 on the emotion setting screen together with photographed image P8.

[0106] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the present invention. [Explanation of symbols]

[0107] 10: Digital camera with printer 12: Camera body 14: Shooting lens 16: Release button 18: Record button 20: Strobe light window 22a: Power button 22b: Menu button 22c: OK button 22d: Mode switch button 24: Microphone hole 26: Speaker hole 28: Display 30: Film lid cover 32a: Joystick 32b: Print button 32c: Play button 32d: Cancel button 34: Film outlet 42: Instant film 42a: Exposed surface 42b: Observation surface 42c: Exposure area 42d: Pod section 42e: Processing solution pod 42f: Trap section 42g: absorbent material 42h: Observation area 42i: Frame 52: Film delivery mechanism 54: Film transport mechanism 56: Print head 62: Lens drive unit 64: Image sensor 66: Image sensor driver 68: Analog signal processing section 70: Digital signal processing section 72: Memory 74: Memory controller 76: Display Controller 78: Communications Department 80: Antenna 82: Film feed drive unit 84: Film transport drive unit 86: Head drive unit 88: Strobe 90: Strobe light control unit 92: Microphone 94: Speaker 96: Audio signal processing section 97: Clock section 98 :Operation section 100: Camera control unit 100A: Image acquisition unit 100B: Date and time information acquisition section 100C: Elapsed date and time calculation section 100D: Image processing section 100E: Display control unit 100F: Printing control unit 101: Image processing device

Claims

1. a date and time information acquisition unit that acquires first date and time information indicating the time when the photographed image to be processed was taken and second date and time information indicating the date and time when the photographed image to be processed was acquired from a storage unit; an elapsed date and time calculation unit that compares the first date and time information and the second date and time information acquired by the date and time information acquisition unit to calculate an elapsed date and time from the time of shooting; an image processing unit that selects an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performs the selected image processing process on the captured image to be processed; An image processing device comprising: the image processing unit acquires information related to the photographed image to be processed, and changes the plurality of image processes that change according to the length of the elapsed date and time based on the related information; The image processing device is configured to perform image processing by acquiring sounds from an imaging environment in which the captured image was captured and adding noise to the captured image in accordance with the sounds from the imaging environment.

2. the image processing unit changes a decorative image to be added to the photographed image to be processed in accordance with the elapsed date and time; The image processing device according to claim 1 , wherein the image processing unit performs image processing to add the decorative image corresponding to the sound of the shooting environment to the captured image.

3. a date and time information acquisition unit that acquires first date and time information indicating the time when the photographed image to be processed was taken and second date and time information indicating the date and time when the photographed image to be processed was acquired from a storage unit; an elapsed date and time calculation unit that compares the first date and time information and the second date and time information acquired by the date and time information acquisition unit to calculate an elapsed date and time from the time of shooting; an image processing unit that selects an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performs the selected image processing process on the captured image to be processed; An image processing device comprising: the image processing unit acquires information related to the photographed image to be processed, and changes the plurality of image processes that change according to the length of the elapsed date and time based on the related information; the image processing unit changes a decorative image to be added to the photographed image to be processed in accordance with the elapsed date and time; The image processing device is configured to acquire sounds of an environment in which the captured image was captured, and to perform image processing to add the decorative image to the captured image according to the sounds of the environment.

4. The image processing device according to claim 3 , wherein the ornamental symbols include musical symbols.

5. The image processing device according to claim 1 , wherein the information related to the photographed image to be processed includes information related to a location where the photographed image was taken, or an emotion input by a user.

6. a date and time information acquisition unit that acquires first date and time information indicating the time when the photographed image to be processed was taken and second date and time information indicating the date and time when the photographed image to be processed was acquired from a storage unit; an elapsed date and time calculation unit that compares the first date and time information and the second date and time information acquired by the date and time information acquisition unit to calculate an elapsed date and time from the time of shooting; an image processing unit that selects an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performs the selected image processing process on the captured image to be processed; An image processing device comprising: the image processing unit acquires information related to the photographed image to be processed, and changes the plurality of image processes that change according to the length of the elapsed date and time based on the related information; An image processing device in which the information related to the captured image to be processed includes sounds of the shooting environment in which the captured image was taken, information related to the shooting location in which the captured image was taken, or emotions input by a user.

7. The image processing device according to claim 6 , wherein the image processing unit changes the decorative image to be added to the photographed image to be processed in accordance with the elapsed date and time.

8. the information related to the photographed image to be processed includes information related to a photographing location where the photographed image was taken, 8. The image processing device according to claim 6, wherein the photographed image to be processed, which has been subjected to image processing by the image processing unit, and audio or text information relating to the photographing location are stored in the storage unit.

9. the information related to the photographed image to be processed includes an emotion input by a user; The image processing device according to claim 5 , further comprising a display control unit that displays an emotion setting screen on a display unit, where a user can input an emotion.

10. The image processing device according to claim 9 , wherein the emotion setting screen includes an operation member for a user to set a ratio of each of a plurality of emotions.

11. The image processing device according to claim 10 , wherein the display control unit causes the display unit to display the proportions of the plurality of set emotions.

12. The image processing device according to claim 1 , wherein the related information includes information about a shooting mode set when the photographed image was taken.

13. The image processing device according to claim 1 , further comprising a display control unit that displays the captured image to be processed, which has been subjected to image processing by the image processing unit, on a display unit.

14. The image processing apparatus according to claim 1 , further comprising a print control unit that causes a print unit to print the captured image to be processed, which has been subjected to image processing by the image processing unit.

15. 15. The image processing device according to claim 1, wherein the captured image of the processing target on which image processing has been performed by the image processing unit and the captured image of the processing target before image processing by the image processing unit are stored in the memory unit.

16. The image processing device according to claim 1 , wherein the captured image to be processed, which has undergone image processing by the image processing unit, is not stored in the storage unit.

17. The image processing device according to claim 1 , further comprising a relationship input unit that receives an input of a correspondence relationship on a time axis between the plurality of image processes and the length of the elapsed date and time.

18. The image processing device according to claim 1 , wherein the image processing unit changes the overall color of the captured image to be processed in accordance with the length of the elapsed date and time.

19. The image processing device according to claim 1 , wherein the image processing unit changes a color of a subject in a part of the captured image to be processed in accordance with the length of the elapsed date and time.

20. The image processing device according to claim 19 , wherein the image processing unit lightens the background color of the captured image to be processed in accordance with the length of the elapsed date and time.

21. The image processing device according to claim 1 , wherein the image processing unit changes a color or a hue of a part of the captured image to be processed in accordance with the length of the elapsed date and time.

22. The image processing device according to claim 1 , wherein the image processing unit changes a degree of blurring of a partial area of ​​the captured image to be processed in accordance with the length of the elapsed date and time.

23. The image processing device according to claim 22 , wherein the image processing unit changes the degree of blurring of the foreground or background of the person in the captured image depending on the length of the elapsed date and time.

24. An imaging device equipped with the image processing device according to any one of claims 1 to 23.

25. a photography mode setting unit that sets one photography mode from among a plurality of photography modes; The photographing device according to claim 24 , wherein the image processing unit changes the plurality of image processes that change according to the length of the elapsed date and time, depending on the photographing mode set by the photographing mode setting unit.

26. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing method comprising: The step of performing the selected image processing on the captured image to be processed includes: acquiring information related to the photographed image to be processed, and changing the plurality of image processing methods that change according to the length of the elapsed date and time based on the related information; The image processing method acquires sounds from an environment in which the captured image was taken, and performs image processing to add noise to the captured image in accordance with the sounds from the environment.

27. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing step comprising: The step of performing the selected image processing on the captured image to be processed includes: acquiring information related to the photographed image to be processed, and changing the plurality of image processing methods that change according to the length of the elapsed date and time based on the related information; a program that causes a computer to execute an image processing step of acquiring sounds from the shooting environment in which the captured image was taken, and performing image processing to add noise to the captured image in accordance with the sounds from the shooting environment;

28. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing method comprising: The step of performing the selected image processing on the captured image to be processed includes: acquiring information related to the photographed image to be processed, and changing the plurality of image processing methods that change according to the length of the elapsed date and time based on the related information; changing a decorative image to be added to the photographed image to be processed in accordance with the elapsed date and time; The image processing method acquires the sound of the shooting environment in which the photographed image was taken, and performs image processing to add a decorative image corresponding to the sound of the shooting environment to the photographed image.

29. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing step comprising: The step of performing the selected image processing on the captured image to be processed includes: acquiring information related to the photographed image to be processed, and changing the plurality of image processing methods that change according to the length of the elapsed date and time based on the related information; changing a decorative image to be added to the photographed image to be processed in accordance with the elapsed date and time; The program causes a computer to execute an image processing step of acquiring sounds from the shooting environment in which the photographed image was taken, and performing image processing to add a decorative image to the photographed image according to the sounds from the shooting environment.

30. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing method comprising: the step of performing the selected image processing on the photographed image to be processed acquires information related to the photographed image to be processed, and changes the plurality of image processings that change according to the length of the elapsed date and time based on the related information; An image processing method in which the information related to the captured image to be processed includes sounds in the shooting environment where the captured image was taken, information related to the shooting location where the captured image was taken, or emotions input by a user.

31. acquiring, from a storage unit, first date and time information indicating the time when the photographed image to be processed was taken, and second date and time information indicating the date and time when the photographed image to be processed was acquired; a step of comparing the acquired first date and time information with the acquired second date and time information to calculate an elapsed date and time from the time of shooting; selecting an image processing process based on the length of the elapsed date and time from among a plurality of image processing processes that change depending on the length of the elapsed date and time, and performing the selected image processing process on the captured image to be processed; An image processing step comprising: the step of performing the selected image processing on the photographed image to be processed acquires information related to the photographed image to be processed, and changes the plurality of image processings that change according to the length of the elapsed date and time based on the related information; A program that causes a computer to execute an image processing process in which the information related to the captured image to be processed includes the sounds of the environment in which the captured image was taken, information related to the location where the captured image was taken, or emotions input by a user.

Images