Image processing apparatus and image processing method

Abstract

To obtain an image obtained by adding decoration corresponding to the intention and taste of a photographer to an image actually put in a photographing visual field as a final photographing result.SOLUTION: A processing control part 10 selects at least one auxiliary image (a first kind auxiliary image and / or a second kind auxiliary image) stored in a storage part 20 in accordance with an auxiliary image selection voice (a character designation voice and / or a photographing notice voice) inputted to a voice input part 70. Then, the processing control part 10 generates data of a composite image obtained by combining the selected auxiliary image and the photographed image acquired by the photographing part, and sets the composite image at the time when the photographing command input to the operation input part 60 is performed as a photographing result image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device and an image processing method. [Background technology]

[0002] Conventionally, techniques have been proposed for generating images that use captured images to give a good impression to viewers, such as the photographer himself / herself (see Patent Documents 1 and 2; hereinafter referred to as "Prior Art 1" and "Prior Art 2"). In the technique of Prior Art 1, taking into consideration that the composition of the image within the field of view is one of the major factors in obtaining an image that reflects the will and preferences of the user, such as the photographer, the technique generates a plurality of reference images ("auxiliary images" in Prior Art 1) with different compositions based on an image acquired via a photographing unit ("imaging unit" in Prior Art 1). Then, in order to actually photograph each of the plurality of reference images, direction information indicating the direction in which the photographing unit should be moved is displayed in association with each of the plurality of reference images.

[0003] The technology of Conventional Example 2 is a technology for making it easier for a photographer to take a photograph as intended. In this technology of Conventional Example 2, for example, in order to make it easier to photograph the face of a subject, an example is disclosed in which an image that is likely to interest the subject is transformed (including superimposition of an effect image) on an image obtained via a photographing unit in accordance with a transformation instruction to a voice input unit, and then displayed and provided to the subject, so that the smiling face of the final subject can be photographed at the timing when the face of the subject finally comes into the field of view. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5880263 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-126018 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned conventional example 1 and conventional example 2 are both techniques for obtaining a photographic result after fitting an actual subject within the photographic field of view in a manner that meets the photographer's will and preference. That is, conventional example 1 is a technique for arranging the composition within the photographic field of view in a manner that meets the photographer's will and preference. Furthermore, in conventional example 2, the image that is fitted within the photographic field of view is an image of content that meets the photographer's will and preference (for example, the face of the subject). Therefore, in both conventional examples 1 and 2, the photographic result is the image that actually fits within the photographic field of view.

[0006] On the other hand, it is considered that there are many cases where the image that a photographer wants to obtain as a result of photography is an image that is actually within the field of view and has decoration added in accordance with the photographer's will and preference. However, neither Conventional Example 1 nor Conventional Example 2 discloses a technology for obtaining an image that is actually within the field of view and has decoration added in accordance with the photographer's will and preference as a final photography result.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an image processing device and an image processing method that can easily produce a final photographed image by adding decorations according to the photographer's will and preferences to an image that fits within the field of view. [Means for solving the problem]

[0008] The first method is an imaging unit that captures an image within a field of view; a display unit for displaying an image; an operation input unit for manually inputting operations; a voice input unit for inputting voice; an internal storage unit that stores data including image data of the auxiliary image; a processing control unit that selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation voice input to the voice input unit, generates data of a composite image by combining the selected auxiliary image with a photographed image acquired by the photographing unit, and sets the composite image at the time when a photographing command is input to the operation input unit as a photographed result image; The present invention is characterized by comprising:

[0009] The second means is the first means, The auxiliary images include a first type auxiliary image and a second type auxiliary image, the first type auxiliary image is a character image, the second type auxiliary image corresponds to the entire area of ​​the captured image and is a visual effect-imparting image having transparency; It is characterized by:

[0010] The third means is the second means, At least one of the first type auxiliary image and the second type auxiliary image is a dynamic image. It is characterized by:

[0011] The fourth means is the second or third means, a plurality of types of the first type auxiliary images are stored in the internal storage unit; the processing control unit selects at least one type of auxiliary image according to the designated voice of the at least one type of character input to the voice input unit, and displays the selected at least one type of first type auxiliary image at a predetermined position within a region for displaying the image of the subject of shooting on the display unit. It is characterized by:

[0012] The fifth means is the second or third means, a plurality of types of the second type auxiliary images are stored in the internal storage unit; the processing control unit starts displaying, on the display unit, second type auxiliary images sequentially selected for each shooting preview sound input to the sound input unit, and then displays the second type auxiliary images for a predetermined time while increasing the transparency over time. It is characterized by:

[0013] The sixth means is the second or third means, The processing control unit performs image synthesis in which the selected first type auxiliary image is used as a foreground image and the photographed image is used as a background image.

[0014] The seventh means is the first means, The photographing unit is characterized in that, in response to a photographing direction specification input to the operation input unit, the processing control unit photographs either the display screen side of the display unit or the opposite side.

[0015] The eighth means is the first means, an external storage medium control unit that controls the storage of data in an external storage medium and the reading of data from the external storage medium; The processing control unit is characterized in that it stores the data of the composite image in a storage location specified by inputting a storage location for the data of the composite image into the operation input unit, which specifies either the internal storage unit or the external storage medium.

[0016] The ninth measure is: An image processing method used in an image processing device including an imaging unit, an operation input unit for manual operation input, a voice input unit, an internal storage unit, and a processing control unit, an auxiliary image selection step in which the processing control unit selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation voice input to the voice input unit; an image synthesis step in which the processing control unit generates data of a composite image by synthesizing the selected auxiliary image with a photographed image acquired by the photographing unit in response to a photographing command input to the operation input unit, and sets the composite image at the time when the photographing command is input to the operation input unit as a photographed result image; The present invention is characterized by comprising: [Effects of the Invention]

[0017] According to the first means, the processing control unit selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation sound input to the sound input unit. Then, the processing control unit generates data for a composite image by combining the selected auxiliary image with a photographed image acquired by the photographing unit, and sets the composite image at the time when a photographing command is input to the operation input unit as the photographed result image. Therefore, by simply inputting the auxiliary image selection sound to the sound input unit and the photographing command input to the operation input unit, a composite image in which decoration according to the photographer's will and preferences has been applied to the photographed image photographed by the photographing unit can be obtained as the final photographed result.

[0018] According to the second or third aspect, the auxiliary images include a first type auxiliary image and a second type auxiliary image, which may be static or dynamic, and the first type auxiliary image is a character image, and the second type auxiliary image is a transparent image with visual effects that corresponds to the entire area of ​​the photographed image. Therefore, a composite image in which the character image and / or the visual effect image is composited with the photographed image can be obtained as the final photographed result.

[0019] According to a fourth aspect, a plurality of types of first-type auxiliary images are stored in the internal storage unit. The processing control unit selects at least one type according to a designated voice of at least one type of character input to the voice input unit, and displays the selected at least one type of first-type auxiliary image at a predetermined position in the subject image display area of ​​the display unit. As a result, a composite image can be obtained as the final photographed result, in which a character image that the photographer judges to be desirable as a decoration for the photographed image at the time of photographing is composited with the photographed image.

[0020] According to a fifth aspect, a plurality of types of second-type auxiliary images are stored in an internal storage unit. Then, the processing control unit starts displaying the second-type auxiliary images selected in sequence for each shooting preview sound input to the sound input unit on the display unit, and then displays the second-type auxiliary images for a predetermined period of time while gradually increasing their transparency. As a result, a composite image can be obtained as a final shooting result, in which a visual effect-added image with a desired transparency that the photographer determined to be desirable as decoration for the photographed image at the time of shooting is combined with the photographed image.

[0021] According to the sixth aspect, the processing control unit performs image composition in which the selected first-type auxiliary image is the foreground image and the photographed image is the background image. The selected second-type auxiliary image is subjected to image composition as a layer between the layer of the first-type auxiliary image and the layer of the photographed image. This allows the processing control unit to rationally perform image composition to obtain a final photographed result.

[0022] According to the seventh means, the photographing unit photographs either the direction toward the display screen of the display unit (the so-called "selfie direction") or the opposite direction (the so-called "other person's photograph direction") in response to a photographing direction specification input to the operation input unit by the processing control unit. This allows the photographer to select whether to use a selfie image or an image taken by another person as the photographed image.

[0023] According to an eighth aspect, the camera further includes an external storage medium control unit that controls the storage of data in an external storage medium such as an SD memory card and the reading of data from the external storage medium. The processing control unit stores the composite image data, which is the photographed image, in a storage destination designated by an input designating either the internal storage unit or the external storage medium to the operation input unit. This allows the photographer to select whether the storage destination of the composite image data is to be the internal storage unit or the external storage medium.

[0024] According to a ninth aspect, in the auxiliary image selection step, the processing control unit selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation sound input to the sound input unit. Then, in the image synthesis step, the processing control unit generates data for a composite image by synthesizing the selected auxiliary image with the captured image acquired by the image capture unit, and sets the composite image at the time when a shooting command is input to the operation input unit as the captured image. Therefore, by simply inputting the auxiliary image selection sound to the sound input unit and the shooting command input to the operation input unit, a composite image in which decoration according to the photographer's will and preferences has been applied to the captured image captured by the image capture unit can be obtained as the final captured image. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a block diagram showing a functional configuration of an image processing apparatus according to an embodiment of the present invention; [Figure 2] FIG. 2 is a schematic external configuration diagram of the image processing device of FIG. [Figure 3] 3 is a diagram for explaining the contents of image data in the internal storage unit of FIG. 2. FIG. [Figure 4] 3 is a diagram for explaining touch keys displayed on the display unit of FIG. 2; [Figure 5] 2 is a flowchart for explaining a process executed by a process control unit in FIG. 1; [Figure 6] 6 is a flowchart for explaining various touch input processes in FIG. 5. [Figure 7] 7A to 7C are diagrams showing an example of screen transitions in the guidance display process of FIG. 6. [Figure 8] 6 is a flowchart for explaining the voice input process of FIG. 5. [Figure 9] 9A to 9C are diagrams showing an example of screen transition in the synthesis process of the designated character image of FIG. 8. [Figure 10] 9 is a diagram showing an example of screen transition in the synthesis process of the visual effect-added image of FIG. 8. DETAILED DESCRIPTION OF THE INVENTION

[0026] An embodiment of the present invention will now be described with reference to Figures 1 to 10. In the following description and drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted.

[0027] [composition] 1 is a block diagram showing the functional configuration of an image processing device 100 according to one embodiment of the present invention, and FIG.

[0028] 1, the image processing device 100 includes a processing control unit 10, a storage unit 20, an external storage medium control unit 30, and an imaging unit 40. The image processing device 100 also includes a display unit 50, an operation input unit 60, and a voice input unit 70.

[0029] The processing control unit 10 is configured to include a central processing unit (CPU) and peripheral circuits. The processing control unit 10 executes programs to perform various processes and control connected units, thereby realizing the functional operation of the image processing device 100.

[0030] The storage unit 20 includes a storage device that can write and read data, and stores various information data used by the processing control unit 10. The information data stored in the storage unit 120 includes each program executed by the processing control unit 10, image data IMD, etc.

[0031] The image data IMD includes a group of auxiliary images and a group of photographed images, as shown in Fig. 3. Here, the group of auxiliary images includes a group of first type auxiliary images and a group of second type auxiliary images.

[0032] The group of first-type auxiliary images includes individual first-type auxiliary images #p (p=1, 2, ...). Each of the individual first-type auxiliary images #p includes an individual character image such as a dog, a rabbit, ..., and display position information of the character image in a shooting field-of-view corresponding region DMR (see FIG. 4) of the display surface 50S of the display unit 50, which will be described later.

[0033] The group of second type auxiliary images includes individual second type auxiliary images #q (q=1, 2, ...). Each of the individual second type auxiliary images #q is an image that is displayed over the entire area of ​​the shooting field of view corresponding region DMR.

[0034] The group of photographic result images includes individual photographic result images #r (r=1, 2, ...) with time stamps at the time of photographing. Each of the individual photographic result images #r is data of a display image in the photographic field of view corresponding region DMR at the time when a photographing command is issued with the inside of the memory unit 20 designated as the storage destination of the photographic result image data.

[0035] Returning to Figure 1, when the external storage medium EMM is connected, the external storage medium control unit 30, under the control of the processing control unit 10, stores data specified by the processing control unit 10 in the external storage medium EMM and reads data from the external storage medium EMM.

[0036] In addition, when an imaging command is issued with the external storage medium EMM specified as the storage destination for the imaging result image data, the data of the displayed image in the imaging field of view corresponding region DMR is stored in the external storage medium EMM.

[0037] The photographing unit 40 includes a selfie camera 40F and a camera 40R that takes pictures of other people. Both the selfie camera 40F and the camera 40R are configured to include a photographing device such as a CCD (Charge Coupled Device).

[0038] 2A, the selfie camera 40F is disposed on the +Z direction side of the −X direction end of the image processing device 100. When the selfie camera 40F is designated as the photographing camera by the processing control unit 10, image data within the photographing field of view on the +Z direction side acquired by the selfie camera 40F is sent to the processing control unit 10.

[0039] 2B, the other camera 40R is placed on the −Z side of the −X side end of the image processing device 100. When the other camera 40R is designated as the photographing camera by the processing control unit 10, image data within the photographing field of view on the −Z side acquired by the other camera 40R is sent to the processing control unit 10.

[0040] 1, the display unit 50 includes a display device such as an LCD or organic EL display, and is disposed in the center of the XY plane on the +Z direction side of the image processing device 100 so as to be visible from the display surface 50S from the +Z direction side, as shown in Fig. 2(A) (see Fig. 2(A)). When the display unit 50 receives display data sent from the processing control unit 10, it displays an image corresponding to the display data on the display surface 50S.

[0041] The operation input unit 60 includes various function keys. Manual operation information for the various function keys is sent to the processing control unit 10 as operation input information.

[0042] In this embodiment, the various function keys are configured as touch keys with transparent electrodes arranged in a grid pattern so as to cover the display surface 50S, and the position pressed by a finger, a touch pen, or the like is detected, and the position information is sent as operation information to the processing control unit 10. Based on the position information, the processing control unit 10 determines which touch key has been touched.

[0043] The touch key arrangement in this embodiment is shown in Fig. 4. As shown in Fig. 4(A), in this embodiment, the operation input unit 60 is provided with touch keys TCK1 to TCK6. The touch key display in Fig. 4 is an example of the display adopted in this embodiment.

[0044] Touch key TCK1 is a home screen key that is executed immediately after power-on and returns to home mode where the home screen is displayed. In home mode, when a touch operation is performed on a touch key (icon) on the home screen that corresponds to an application for realizing the functions of image processing device 100 of this embodiment (hereinafter referred to as "this application"), execution processing of this application is started.

[0045] The touch key TCK2 is a camera switching key for switching between selecting the selfie camera 40F or the other camera 40R as the camera to be photographed. Each time a touch input is made to the touch key TCK2, the camera to be photographed is switched. In this embodiment, the selfie camera 40F is selected as the camera to be photographed immediately after the execution of this application starts.

[0046] The touch key TCK3 is a guidance display key for displaying guidance that explains how to input voice when using this application. The guidance display will be described later.

[0047] Touch key TCK4 is a storage destination switching key for switching the storage destination of the photographing result data between the storage unit 20 and an external storage medium. Each time touch input is made to touch key TCK4, the storage destination is switched. In this embodiment, immediately after the execution of this application starts, the storage destination is selected to be the storage unit 20.

[0048] Fig. 4(A) shows an example of the display of the touch key TCK4 when the storage unit 20 is selected as the storage destination, while Fig. 4(B) shows an example of the display of the touch key TCK4 when the external storage medium EMM is selected as the storage destination.

[0049] The touch key TCK5 is a shooting key for acquiring shooting result data. When a touch input is made to the touch key TCK5, the data of the image displayed in the shooting field of view corresponding region DMR at that time is stored in the set storage location.

[0050] The touch key TCK6 is a voice input setting key for switching between enabling and disabling voice input by the voice input unit 70. In this embodiment, when voice input is enabled, the display of the touch key TCK6 is highlighted, and when voice input is disabled, the display of the touch key TCK6 is not highlighted.

[0051] 1, the audio input unit 70 is configured to include a sound collection device such as a microphone. When audio is input to the audio input unit 70, the audio input unit 70 sends audio input data corresponding to the audio input to the processing control unit 10.

[0052] [Operation] Next, the operation of the image processing device 100 will be described, focusing mainly on the processing performed by the processing control unit 10 when the application is executed. It is assumed that the application has already been executed.

[0053] Fig. 5 shows a flowchart of the process executed by the process control unit 10 when this application is executed. As shown in Fig. 5, when this application is executed, first, in step S11, the process control unit 10 determines whether or not a touch input has been performed. If the result of the determination in step S11 is positive (step S11: Y), the process proceeds to step S12. In this step S12, the process control unit 10 determines whether or not the home screen key (TCK1) has been touched.

[0054] If the result of the determination in step S12 is affirmative (step S12: Y), the process control unit 10 ends the process for executing this application, after which the home mode operation is executed.

[0055] If the result of the determination in step S12 is negative (step S12: N), the process proceeds to step S13. In step S13, the process control unit 10 executes various touch input response processes in response to touch operations on the touch keys (TCK2 to TCK6) other than the home screen key. Then, when the process in step S13 ends, the process returns to step S11.

[0056] The various touch input handling processes in step S13 will be described later.

[0057] If the result of the determination in step S11 is negative (step S11: N), the process proceeds to step S14. In step S14, the process control unit 10 determines whether or not voice input has been performed. If the result of the determination in step S14 is positive (step S14: Y), the process proceeds to step S15.

[0058] In step S15, the process control unit 10 executes the voice input handling process. Then, when the process of step S15 ends, the process returns to step S11.

[0059] The voice input handling process in step S15 will be described later.

[0060] <Various touch input processing> Next, various touch input handling processes in step S13 will be described.

[0061] 6, in the various touch input corresponding processes, first, in step S21, the process control unit 10 determines whether or not the photographing camera switching key (TCK2) has been touched. If the determination result in step S21 is affirmative (step S21: Y), the process proceeds to step S22. In this step S22, the process control unit 10 performs photographing camera switching processing.

[0062] In this photographing camera switching process, if the photographing camera before switching was the selfie camera 40F, the process control unit 10 switches the photographing camera to the other-photographing camera 40R. On the other hand, if the photographing camera before switching was the other-photographing camera 40R, the process control unit 10 switches the photographing camera to the selfie camera 40F.

[0063] When the camera is switched as described above, the process of step S22 ends, and the process of step S13 ends, and the process returns to step S11 in FIG.

[0064] If the determination result in step S21 is negative (step S21: N), the process proceeds to step S23. In this step S23, the process control unit 10 determines whether or not the guidance display key (TCK3) has been touched. If the determination result in step S23 is positive (step S23: Y), the process proceeds to step S24. In this step S24, the process control unit 10 performs guidance display processing.

[0065] In this guidance display process, the processing control unit 10 displays an image in which the guidance display image is superimposed on the image displayed in the shooting field of view corresponding region DMR. Fig. 7 shows the transition of the display image on the display surface 50S when the guidance display key is touched.

[0066] Here, Fig. 7(A) is an example of a display image on the display surface 50S immediately before the guidance display key is touched, and Fig. 7(B) is an example of a display image on the display surface 50S immediately after the guidance display key is touched. In the guidance display in Fig. 7(B), an example of a designated sound for the character image is indicated by a "☆". Also, an example of a shooting preview sound is indicated by a "★". Also, Fig. 7(A) is an example in which only an image within the shooting field of view acquired by the shooting camera is displayed.

[0067] If the display area of ​​the guidance display image is touched while the guidance display image is being displayed, the processing control unit 10 ends the superimposed display of the guidance display, changes the display image in the shooting field of view corresponding area DMR to the image when the guidance display key is not touched, and then ends the processing of step S24. As a result, the processing of step S13 ends, and the processing returns to step S11 in Fig. 5.

[0068] Returning to FIG. 6, if the determination result in step S23 is negative (step S23: N), the process proceeds to step S25. In this step S25, the process control unit 10 determines whether or not the storage destination switching key (TCK4) has been touched. If the determination result in step S25 is positive (step S25: Y), the process proceeds to step S26. In this step S26, the process control unit 10 performs storage destination switching processing.

[0069] In this storage destination switching process, if the storage destination before switching was within the memory unit 20, the process control unit 10 switches the storage destination to the external storage medium EMM and switches the image of the storage destination switching key to an image corresponding to the storage destination being the external storage medium EMM (see FIG. 4(B)). On the other hand, if the storage destination before switching was the external storage medium EMM, the process control unit 10 switches the storage destination to within the memory unit 20 and switches the image of the storage destination switching key to an image corresponding to the storage destination being within the memory unit 20 (see FIG. 4(A)).

[0070] When the process of step S26 ends, the process of step S13 ends, and the process returns to step S11 in FIG.

[0071] If the determination result in step S25 is negative (step S25: N), the process proceeds to step S27. In this step S27, the process control unit 10 determines whether or not the photographing key (TCK5) has been touched. If the determination result in step S27 is positive (step S27: Y), the process proceeds to step S28. In this step S28, the process control unit 10 performs photographing processing.

[0072] In this photographing process, the processing control unit 10 stores the image data displayed in the photographing field of view corresponding region DMR in a set storage location with a timestamp. Then, the processing control unit 10 limits the displayed image in the photographing field of view corresponding region DMR to only the image within the photographing field of view acquired by the photographing camera.

[0073] When the process of step S26 ends, the process of step S13 ends, and the process returns to step S11 in FIG.

[0074] If the determination result in step S27 is negative (step S27: N), the process proceeds to step S29. In this step S29, the process control unit 10 determines whether or not the voice input setting key (TCK6) has been touched. If the determination result in step S29 is positive (step S29: Y), the process proceeds to step S30. In this step S30, the process control unit 10 performs voice input setting processing.

[0075] In this voice input setting process, if the voice input setting before switching was the voice input enabled mode, the process control unit 10 changes the voice input enabled mode to the voice input disabled mode and unhighlights the image of the voice input setting key. On the other hand, if the voice input setting before switching was the voice input disabled mode, the process control unit 10 changes the voice input enabled mode to the voice input enabled mode and highlights the image of the voice input setting key.

[0076] When the process of step S30 ends in this way, the process of step S13 ends, and the process returns to step S11 in FIG.

[0077] <Voice input processing> Next, the voice input process in step S15 will be described. Note that correspondence information indicating which of the individual first type auxiliary image information #p (p=1, 2, ...) in the storage unit 20 each of the character-designating voices shown in the above-mentioned guidance display corresponds to is registered in advance in the storage unit 20.

[0078] In the voice input process, as shown in Fig. 8, first, in step S31, the process control unit 10 determines whether the voice input setting mode is the voice input enabled mode. If the result of the determination in step S31 is negative (step S31: N), the process of step S15 ends. Then, the process returns to step S11 in Fig. 5.

[0079] If the result of the determination in step S31 is negative (step S31: N), the process proceeds to step S32. In this step S32, the process control unit 10 determines whether or not the input voice is a character-designated voice. If the result of the determination in step S31 is positive (step S32: Y), the process proceeds to step S33. In this step S33, the process control unit 10 performs a synthesis process of a designated character image.

[0080] In the synthesis process of the designated character image, the process control unit 10 first identifies individual first type auxiliary image information corresponding to the character designation voice. Then, the process control unit 10 reads the identified individual first type auxiliary image information from the storage unit 20.

[0081] Next, the processing control unit 10 composites the character image in the read individual first-type auxiliary image information with the image currently displayed in the imaging field of view corresponding area DMR as the foreground image at a position within the imaging field of view corresponding area DMR specified in the individual first-type auxiliary image information.The composite image is then displayed in the imaging field of view corresponding area DMR.Note that when the new character image is composited, the character image previously displayed in the imaging field of view corresponding area DMR is deleted.

[0082] FIG. 9 shows the transition of the display image in the imaging field of view corresponding region DMR when a character designation voice is input.

[0083] Fig. 9(A) is an example of a display image in the imaging field of view corresponding area DMR immediately before input of a character-designated voice. Fig. 9(B) is an example of a display image in the imaging field of view corresponding area DMR when "cat" is input as the character-designated voice. Fig. 9(C) is an example of a display image in the imaging field of view corresponding area DMR when "rabbit" is input as the character-designated voice. Fig. 9(D) is an example of a display image in the imaging field of view corresponding area DMR when it is determined that the voices "cat" and "rabbit" have been input continuously as the character-designated voice.

[0084] When the process of step S33 ends in this way, the process of step S15 ends, and the process returns to step S11 in FIG.

[0085] Returning to FIG. 8, if the result of the determination in step S32 is negative (step S32: N), the process proceeds to step S34. In this step S34, the process control unit 10 determines whether or not the input sound is shooting preview sound. If the result of the determination in step S34 is positive (step S34: Y), the process proceeds to step S35. In this step S35, the process control unit 10 performs a synthesis process for the visual effect-added image.

[0086] In the synthesis process of such a visual effect-added image, the process control unit 10 first identifies an individual second type auxiliary image corresponding to the character designation sound as an image to be synthesized in response to the current shooting preview sound. In this identification, the process control unit 10 identifies the individual second type auxiliary image registered in the storage unit 20 next to the most recently adopted individual second type auxiliary image as the visual effect-added image to be synthesized. Note that if the most recently adopted individual second type auxiliary image was registered last, the process control unit 10 identifies the individual second type auxiliary image registered first as the visual effect-added image to be synthesized.

[0087] Next, the processing control unit 10 reads data of the identified visual effect-added image from the storage unit 20. Subsequently, the processing control unit 10 performs an initial transparency process based on the read data of the visual effect-added image, and then performs a synthesis process with the image that had been displayed in the shooting field of view corresponding region DMR up to that point. Note that the processing control unit 10 performs image synthesis such that the visual effect-added image is an image on a layer lower than the character image and on a layer higher than the image in the field of view of the shooting camera.

[0088] Thereafter, the process control unit 10 performs image synthesis for a predetermined time while gradually increasing the transparency of the visual effect-added image until the transparency reaches 1. Then, when the predetermined time has elapsed since the start of synthesis of the visual effect-added image, the process control unit 10 ends the synthesis process of the visual effect-added image.

[0089] When the process of step S35 ends in this way, the process of step S15 ends, and the process returns to step S11 in FIG.

[0090] 10 shows an example of image transition of the field of view corresponding region DMR during the synthesis process of the visual effect-added image. Here, the visual effect-added image is an image of "cherry blossom petals falling."

[0091] As described above, in this embodiment, the processing control unit 10 selects at least one auxiliary image (first type auxiliary image and / or second type auxiliary image) stored in the storage unit 20 in response to the auxiliary image selection sound (character designation sound and / or shooting preview sound) input to the sound input unit 70. Then, the processing control unit 10 generates data for a composite image by combining the selected auxiliary image with a photographed image acquired by the photographing unit, and sets the composite image at the time when a photographing command is input to the operation input unit 60 as the photographed result image. Therefore, by performing simple input operations of inputting the auxiliary image selection sound to the sound input unit 70 and inputting the photographing command input to the operation input unit 60, it is possible to obtain a composite image as the final photographed result, in which decoration according to the photographer's will and preferences is applied to the photographed image photographed by the photographing unit.

[0092] In this embodiment, the auxiliary images include a first type auxiliary image and a second type auxiliary image, the first type auxiliary image being a character image, and the second type auxiliary image being a transparent image with visual effects that corresponds to the entire captured image. Therefore, a composite image in which the character image and / or the visual effect image is composited with the captured image can be obtained as the final captured image.

[0093] In this embodiment, multiple types of first-type auxiliary images are stored in the storage unit 20. The processing control unit 10 then selects at least one type according to the designated voice of at least one type of character input to the voice input unit 70, and displays the selected at least one type of first-type auxiliary image at a predetermined position in the subject image display region DMR of the display unit 50. As a result, a composite image can be obtained as the final photographed result, in which a character image that the photographer determined to be desirable to add as a decoration to the photographed image at the time of photographing is composited with the photographed image.

[0094] In this embodiment, multiple types of second-type auxiliary images are stored in the storage unit 20. Then, the processing control unit 10 starts displaying, on the display unit 50, second-type auxiliary images selected in sequence for each shooting preview sound input to the sound input unit 70, and then displays the second-type auxiliary images for a predetermined period of time while increasing the transparency over time. As a result, a composite image can be obtained as the final shooting result, in which a visual effect-added image with a desired transparency that the photographer determined to be desirable as decoration for the captured image at the time of shooting is combined with the captured image.

[0095] In this embodiment, the processing control unit 10 performs image composition in which the selected first-type auxiliary image is the foreground image and the captured image is the background image. The selected second-type auxiliary image is used as a layer between the layer of the first-type auxiliary image and the layer of the captured image. This allows the processing control unit 10 to rationally perform image composition to obtain the final captured image.

[0096] In this embodiment, the photographing unit 40 photographs either the direction toward the display screen 50S of the display unit 50 (the so-called "selfie direction") or the opposite direction (the so-called "other-photographed direction") in response to a photographing direction specification input to the operation input unit 60 by the processing control unit 10. This allows the photographer to select whether to use a selfie image or an image taken by another person as the photographed image.

[0097] Furthermore, this embodiment further includes an external storage medium control unit 30 that controls the storage of data in an external storage medium EMM such as an SD memory card and the reading of data from the external storage medium EMM. The processing control unit 10 stores the composite image data, which is the photographed image, in a storage destination designated by an input to the operation input unit 60 that designates either the internal storage unit 20 or the external storage medium EMM as the storage destination for the composite image data. This allows the photographer to select whether the storage destination for the composite image data is to be the internal storage unit or the external storage medium.

[0098] [Modification of the embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the present invention.

[0099] For example, in the above embodiment, a "dog" image and a "rabbit" image are exemplified as selectable character images. However, other character images may be used in addition to or instead of these character images. In this case, different character-designating sounds are set for each selectable character image.

[0100] In addition, in the above embodiment, animal character images are exemplified as selectable character images, but character images other than animals may also be used as selectable character images.

[0101] In the above embodiment, an image of "cherry blossom shower" is used as an example of the visual effect-applied image, but other visual effect-applied images may be used in addition to or instead of this visual effect-applied image.

[0102] Furthermore, in the above embodiment, a still image is assumed as the image with visual effects. However, a moving image (dynamic image) may also be used as the image with visual effects. For example, an image with dynamic visual effects may be an image of "cherry blossom storm," "autumn leaves," or "soap bubbles flying." Similarly, a still image (static image) is assumed as the character image, but a dynamic image may also be used. For example, an image in which the character moves around or changes pose or expression may also be used.

[0103] In the above embodiment, the visual effect-applied image is changed sequentially with each input of the preview audio. Alternatively, the visual effect-applied image may be selected in accordance with the audio input of the attributes of the visual effect-applied image.

[0104] In addition, in the above embodiment, input of a character-designated voice and / or a shooting preview voice during the synthesis process of an image with visual effects is not assumed. However, input of a character-designated voice and / or a shooting preview voice during the synthesis process of an image with visual effects may be processed as valid.

[0105] In the above embodiment, an SD card is used as an example of an external storage medium, but a so-called cloud server may be used as the external storage medium.

[0106] Furthermore, in the above embodiment, a case has been described in which both a first type auxiliary image and a second type auxiliary image are simultaneously combined with a captured image, but it is also possible to configure the system so that one of the first type auxiliary image or the second type auxiliary image is selectively combined with a captured image.

[0107] In addition, in the above embodiment, the shooting preview sound is exemplified as "I'm taking it!" (Figure 7(B)), and one of several visual effect-added images is made to appear in response to the shooting preview sound. However, the shooting preview sound may also be a word such as "cherry blossom storm," which in the above embodiment directly indicates the frame that is to appear. [Explanation of symbols]

[0108] 10... Processing control section 20 … Storage section (internal storage section) 30 ... External storage medium control unit 40... Photography Department 40F ... Selfie camera (part of the photography section) 40R... Other camera (part of the shooting section) 50…Display section 50S…Display surface 60 ... Operation input section 70...Audio input section 100... Image processing device EMM … external storage medium DMR: Shooting field of view range IMD...image data TCK1~TCK6 ... Touch keys

Claims

1. an imaging unit that captures an image within a field of view; a display unit for displaying an image; an operation input unit for manually inputting operations; a voice input unit for receiving voice input; an internal storage unit that stores data including image data of the auxiliary image; a processing control unit that selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation voice input to the voice input unit, generates data of a composite image by combining the selected auxiliary image with a photographed image acquired by the photographing unit, and sets the composite image at the time when a photographing command is input to the operation input unit as a photographed result image; An image processing device comprising:

2. The auxiliary images include a first type auxiliary image and a second type auxiliary image, the first type auxiliary image is a character image, the second type auxiliary image corresponds to the entire area of ​​the captured image and is a visual effect-imparting image having transparency; 2. The image processing device according to claim 1, wherein:

3. At least one of the first type auxiliary image and the second type auxiliary image is a dynamic image.

3. The image processing device according to claim 2.

4. a plurality of types of the first type auxiliary images are stored in the internal storage unit; the processing control unit selects at least one type of auxiliary image according to the designated voice of at least one type of character input to the voice input unit, and displays the selected at least one type of first type auxiliary image at a predetermined position within a region for displaying the image of the subject of shooting on the display unit.

4. The image processing device according to claim 2 or 3.

5. a plurality of types of the second type auxiliary images are stored in the internal storage unit; the processing control unit starts displaying, on the display unit, second type auxiliary images sequentially selected for each shooting preview sound input to the sound input unit, and then displays the second type auxiliary images for a predetermined time while increasing the transparency over time.

4. The image processing device according to claim 2 or 3.

6. 4. The image processing device according to claim 2, wherein the processing control unit performs image synthesis in which the selected first type auxiliary image is used as a foreground image and the photographed image is used as a background image.

7. 2. The image processing device according to claim 1, wherein the photographing unit photographs either one of the display screen side and the opposite side of the display screen by the processing control unit in response to a photographing direction specification input to the operation input unit.

8. an external storage medium control unit that controls the storage of data in an external storage medium and the reading of data from the external storage medium; The image processing device according to claim 1, characterized in that the processing control unit stores the data of the composite image in a storage destination specified by inputting a storage destination for the data of the composite image into the operation input unit, which specifies either the internal storage unit or the external storage medium.

9. An image processing method used in an image processing device including an imaging unit, an operation input unit for manual operation input, a voice input unit, an internal storage unit, and a processing control unit, an auxiliary image selection step in which the processing control unit selects at least one auxiliary image stored in the internal storage unit in response to an auxiliary image designation voice input to the voice input unit; an image synthesis step in which the processing control unit generates data of a composite image by synthesizing the selected auxiliary image with a photographed image acquired by the photographing unit in response to a photographing command input to the operation input unit, and sets the composite image at the time when the photographing command is input to the operation input unit as a photographed result image; An image processing method comprising:

Images