Image processing device, photographing device, image processing method and storage medium

Abstract

The present invention provides an image processing device, a photographing device, an image processing method, and a storage medium that simplify the creation of a composite image and improve user operability related to the composite. The mobile terminal of the present invention includes a display control unit, an extraction unit, and a synthesis unit. The display control unit displays a determination frame that determines a determination area as a part of an area in a display image displayed on a touch screen display and a color designation mark for designating a reference color from the colors included in the image of the determination area in the determination frame in an overlapping manner in the display image. The extraction unit extracts an image corresponding to the reference color designated by the color designation mark from the image of the determination area of ​​the determination frame as a composite object image. The synthesis unit generates a composite image by synthesizing the composite object image with a selection image selected by a user.

Description

Technical Field

[0001] The invention relates to an image processing device, a photographing device, an image processing method and a storage medium. Background Art

[0002] Typically, a composite image is created by combining the composite target image with the selected image selected by the user, and is displayed on a display unit, etc. For example, Patent Document 1 discloses a technique in which the user specifies an area to be cropped from the original image using an inclusion line, and an area in which the area of ​​the areas connected by the same color accounts for a predetermined ratio or more is cropped from the area indicated by the inclusion line and is embedded and composited into an arbitrary image.

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 1-175076

[0004] In the technique described in Patent Document 1, the user specifies the cropped area or color by specifying pixels using a coordinate input device composed of a tablet computer and a stylus pen. In the technique described in Patent Document 1, cropping an image takes time, so there is a problem in user operability. Summary of the invention

[0005] The present invention has been made in view of the above circumstances, and its object is to provide an image processing device, a photographing device, an image processing method and an image processing program that simplify the creation of a composite image and improve the user operability related to the synthesis.

[0006] In order to achieve the above-mentioned purpose, the image processing device of the first mode of the present invention comprises: a display control unit, which displays a determination box that determines a part of the area in the display image displayed in the display unit and a color designation mark for designating a reference color from the color of the image in the determination box in an overlapping manner on the display image; an extraction unit, which extracts an image corresponding to the reference color specified by the color designation mark from the image in the determination box as a synthesized object image; and a synthesis unit, which generates a synthesized image by synthesizing the synthesized object image and a selected image selected by a user.

[0007] An image processing device according to a second aspect of the present invention is the image processing device according to the first aspect, wherein the color specification mark is displayed in a determination frame, and a relative positional relationship between the determination frame and the color specification mark is determined in advance.

[0008] The image processing device of the third mode of the present invention is further provided with, in the image processing device of the first mode or the second mode: a first receiving unit that receives an instruction to change the size of the confirmation frame to be enlarged or reduced, and a display control unit that controls the change of the size of the confirmation frame according to the instruction to change the size received by the first receiving unit.

[0009] The image processing device of the fourth mode of the present invention is an image processing device of any one of the first to third modes, further comprising: a second acceptance unit that accepts a moving instruction for the position of the moving determination frame, and a display control unit that controls the position of the moving determination frame according to the moving instruction accepted by the second acceptance unit.

[0010] An image processing device according to a fifth aspect of the present invention is the image processing device according to any one of the first to fourth aspects, wherein the color of the display image at a portion where the color specifying mark overlaps is used as the reference color.

[0011] An image processing device according to a sixth aspect of the present invention is the image processing device according to the fifth aspect, wherein the display control unit further displays an indicator having an area larger than the color specifying mark and indicating the reference color.

[0012] In the image processing device of the 7th mode of the present invention, in the image processing device of any one of the 1st to 4th modes, the color designation mark will have a shape surrounding a part of the displayed image, and the extraction unit will use the color of the displayed image surrounded by the color designation mark as the reference color.

[0013] An image processing device according to an eighth aspect of the present invention is the image processing device according to the seventh aspect, wherein the display control unit further displays an indicator having an area larger than a region of the display image surrounded by the color specifying mark and indicating the reference color.

[0014] In the image processing device of the 9th mode of the present invention, in the image processing device of any one of the 1st mode to the 8th mode, the extraction unit digitizes the grayscale of the entire image within the determined frame, and extracts an image whose grayscale value is within the reference color range including the grayscale value corresponding to the reference color as a synthesized object image.

[0015] An image processing device according to a tenth aspect of the present invention is the image processing device according to the ninth aspect, further comprising a setting unit for a user to set a reference color range.

[0016] An imaging device according to an eleventh aspect of the present invention comprises: an imaging unit for imaging a subject; a display unit for displaying an image captured by the imaging unit; and an image processing device according to any one of the first to tenth aspects.

[0017] The photographing device according to the twelfth aspect of the present invention is the photographing device according to the eleventh aspect, wherein a real-time preview of the display image is displayed on the display unit.

[0018] Furthermore, in order to achieve the above-mentioned purpose, the image processing method of the 13th mode of the present invention performs the following processing by a computer: a determination box for determining a portion of an area in a display image displayed on a display unit and a color designation mark for designating a reference color from the colors included in the image within the determination box are displayed overlappingly in the display image; an image corresponding to the reference color specified by the color designation mark is extracted from the image within the determination box as a synthesized object image; and a synthesized image is generated by synthesizing the synthesized object image with a selected image selected by a user.

[0019] Furthermore, in order to achieve the above-mentioned purpose, a computer-readable storage medium of the 14th mode of the present invention stores an image processing program for causing a computer to execute the following processing: displaying a determination box for determining a portion of an area in a display image displayed on a display unit and a color designation mark for designating a reference color from colors included in the image within the determination box in an overlapping manner in the display image; extracting an image corresponding to the reference color specified by the color designation mark from the image within the determination box as a synthesized object image; and generating a synthesized image by synthesizing the synthesized object image with a selected image selected by a user.

[0020] Furthermore, the image processing device of the present invention comprises a memory for storing instructions for causing a computer to execute and a processor configured to execute the stored instructions, the processor displays a determination box for determining a portion of an area in a display image displayed on a display unit and a color designation mark for designating a reference color from colors included in the image within the determination box in an overlapping manner in the display image, extracts an image corresponding to the reference color specified by the color designation mark from the image within the determination box as a synthesized object image, and generates a synthesized image by synthesizing the synthesized object image with a selected image selected by a user.

[0021] Effects of the Invention

[0022] According to the present invention, it is possible to simplify the creation of a composite image and improve user operability related to the composite. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a diagram showing an example of a situation in which a user is taking a photo using a mobile terminal as an example of a photographing device.

[0024] Figure 2 This is a schematic diagram showing an example of the electrical structure of a mobile terminal.

[0025] Figure 3 This is a functional block diagram showing an example of the structure of a mobile terminal.

[0026] Figure 4 It is a diagram showing a display example of a confirmation box with an indicator.

[0027] Figure 5 This is a functional block diagram showing an example of the structure of a mobile terminal.

[0028] Figure 6 This is a flowchart showing an example of composite photography processing.

[0029] Figure 7 It is a diagram for explaining the movement of the determination frame.

[0030] Figure 8 It is a diagram for explaining enlargement / reduction of a determination frame.

[0031] Fig. 9 This is a flowchart showing an example of synthesis target image extraction processing.

[0032] Fig.10 This is a diagram for explaining a process of extracting a synthesis target image from a specific region.

[0033] Fig.11 This is a diagram showing a state where a selected image to be combined with characters as a combination target image is displayed in real time preview.

[0034] Fig.12 This is a diagram showing an example of moving the display position of a character that is a synthesis target image.

[0035] Fig.13 This is a diagram showing an example of enlarging the display size of characters as a synthesis target image.

[0036] Fig.14 This is a diagram showing an example of rotating the display posture of a character that is a synthesis target image.

[0037] Fig.15 This is a diagram showing an example of changing the color of a character that is an image to be synthesized.

[0038] Fig.16 This is a diagram summarizing the synthetic photography processing in the mobile terminal of the implementation mode.

[0039] Fig.17 This is a diagram for explaining Modification Example 1.

[0040] Fig.18 This is a diagram for explaining Modification Example 1.

[0041] Fig.19 This is a diagram for explaining Modification Example 1.

[0042] Fig. 20 This is a diagram for explaining Modification Example 1.

[0043] Fig.21 This is a diagram for explaining Modification Example 2.

[0044] Fig. 22 This is a diagram for explaining Modification Example 2.

[0045] Fig.23A This is a diagram for explaining Modification Example 3.

[0046] Fig. 23B This is a diagram for explaining Modification Example 3.

[0047] Fig.23C This is a diagram for explaining Modification Example 3.

[0048] Fig.23D This is a diagram for explaining Modification Example 3.

[0049] Fig.23E This is a diagram for explaining Modification Example 3.

[0050] Fig.23F This is a diagram for explaining Modification Example 3.

[0051] Fig.24A This is a diagram for explaining Modification Example 4.

[0052] Fig. 24B This is a diagram for explaining Modification Example 4.

[0053] Fig.25A This is a diagram for explaining Modification Example 5.

[0054] Fig.25B This is a diagram for explaining Modification Example 5.

[0055] Fig.26A This is a flowchart showing an example of synthetic photography processing according to Modification Example 6.

[0056] Fig.26B This is a flowchart showing an example of synthetic photography processing according to Modification Example 6. DETAILED DESCRIPTION

[0057] Hereinafter, an example of a method for implementing the technology of the present invention will be described in detail with reference to the accompanying drawings. Figure 1 As shown, the mobile terminal 10 is, for example, a smartphone. In addition to the function of a mobile phone, the mobile terminal 10 also functions as a computer that executes various application programs called applications. Furthermore, the mobile terminal 10 can also take photos and is an example of a photographing device.

[0058] The mobile terminal 10 has a built-in photographing unit 11 for photographing a subject (see Figure 2 The mobile terminal 10 is provided with a touch screen display 12 on the front surface of the main body. The mobile terminal 10 can perform a live preview display, which sequentially displays the photographic images taken by the photographing unit 11 on the touch screen display 12 as real-time view images.

[0059] In the live preview display, the photographing unit 11 outputs the photographic image at a frame rate of, for example, 30 frames per second, and the live view image displayed on the touch screen display 12 is also updated at this frame rate. Therefore, the live view image displayed in the live preview is visually recognized as an animation. Since the live preview displays the subject, the user U can shoot while confirming the subject and the composition of the photographic image through the display of the touch screen display 12.

[0060] As an example, Figure 2 As shown in FIG. 1 , the mobile terminal 10 includes a photographing unit 11, a touch screen display 12, a CPU (Central Processing Unit) 14, an operation switch 16, a working memory 17, a storage device 18, a media controller 19, an external I / F (Interface) 21, and a communication unit 22. These are connected via a bus 23 so as to be able to communicate.

[0061] The CPU 14 functions as a main control unit 26 and a display control unit 31 by executing an operating system (not shown). The main control unit 26 controls the various units of the mobile terminal 10 in a centralized manner. The display control unit 31 controls the display of the touch screen display 12. The CPU 14 also functions as an image processing unit 32 that performs image processing on the image captured by the imaging unit 11.

[0062] The work memory 17 is a volatile memory used as a work area when the CPU 14 executes various programs, and a RAM (Random Access Memory) can be used. The work memory 17 is used as a work memory when performing live preview display and image processing as described later.

[0063] In addition to the above-mentioned operating system, the storage device 18 stores various programs such as an application program (AP) 27. The application program 27 includes image display software, a web browser, and a mail application program in addition to a photo synthesis application program described later.

[0064] In addition, the storage device 18 is provided with a user area 28 for storing user data such as image data representing images captured by the user U using the mobile terminal 10. The storage device 18 is, for example, a non-volatile memory such as a flash memory. In addition, as the storage device 18, a HDD (Hard Disk Drive) or an SSD (Solid State Drive) may also be used.

[0065] The touch screen display 12 is an example of a display unit that displays an operation screen that functions as a GUI (Graphical User Interface) stored in the storage device 18 and an image captured by the imaging unit 11. The touch screen display 12 has a function of detecting a touch operation by a finger of the user U and the like, and is used to detect the touch operation and input an operation instruction to the main control unit 26. As is well known, the touch screen display 12 is composed of a display unit such as an LCD (Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display and a touch panel unit such as a resistive film method or an electrostatic capacitance method.

[0066] The operation switch 16 is various mechanical operation parts such as a multi-function key and a power button provided on the main body of the mobile terminal 10 , and inputs operation instructions to the main control unit 26 .

[0067] The main control unit 26 is a receiving unit that receives input of operation instructions from the touch panel display 12 and the operation switches 16 .

[0068] The display control unit 31 controls the display of operation screens and images on the touch panel display 12 under the control of the main control unit 26. The display control unit 31 converts the operation screens and images into video signals that can be processed by the touch panel display 12 and outputs the signals to the touch panel display 12.

[0069] The image processing unit 32 performs various image processing such as image extraction and image synthesis described later on the image captured by the image capturing unit 11 in addition to general image processing such as brightness correction and compression and expansion processing.

[0070] The media controller 19 writes and reads data to and from a removable medium such as a memory card 29 .

[0071] The external I / F 21 is, for example, an interface such as a USB (Universal Serial Bus) connector, etc. A charging cable, a communication cable, a USB memory, etc. are connected to the external I / F 21 .

[0072] The communication unit 22 includes an antenna and a communication control circuit (both are not shown), and performs communication functions such as wireless communication and short-range wireless communication via a mobile communication network and a wireless LAN (Local Area Network).

[0073] The imaging unit 11 includes, for example, an imaging lens 11A, an image sensor 11B, an analog-to-digital (A / D) converter 11C, an image input controller 11D, an image signal processing circuit 11E, a timing generator (TG) 11F, and a driver 11G.

[0074] The photographic lens 11A collects light from the subject and forms an image of the subject on the imaging surface of the image sensor 11B. The image sensor 11B performs photoelectric conversion on the subject light captured by the imaging element and outputs it as an analog image signal. As the image sensor 11B, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used. In the case of live preview display, the image sensor 11B sequentially outputs analog image signals at a preset frame rate.

[0075] The AD converter 11C converts the analog image signal from the image sensor 11B into a digital image signal. The image input controller 11D records the digital image signal input from the AD converter 11C in the working memory 17 as a photographic image. During live preview photography, the image input controller 11D records the photographic image in the working memory 17 as a live view image. In the working memory 17, the live view image is used for live preview display and is sequentially updated according to the frame rate of the image sensor 11B.

[0076] The image signal processing circuit 11E performs various signal processing such as white balance processing, gamma correction processing, and YC conversion processing on the photographic image recorded in the working memory 17 by the image input controller 11D. The white balance processing is a process for adjusting the gain of each image signal of R (red), G (green), and B (blue) included in the photographic image to adjust the white balance. The gamma correction processing is a grayscale conversion processing performed on the image signal according to the gamma characteristics stored in advance. The YC processing is a process for converting the image signals of R, G, and B into a brightness signal Y and color difference signals Cr and Cb. The photographic image processed by the image signal processing circuit 11E is rewritten into the working memory 17.

[0077] The timing signal generator 11F generates a timing signal that specifies the operation timing of the image sensor 11B under the control of the main control unit 26. The driver 11G drives the image sensor 11B based on the timing signal from the timing signal generator 11F.

[0078] When the photographing unit 11 is activated so that the mobile terminal 10 functions as a photographing device, the display control unit 31 controls to display a live preview of the photographic image taken by the photographing unit 11 on the touch screen display 12 as a live view image. The live preview display starts, for example, at the same time as the photographing unit 11 is activated. The live preview display is interrupted for a while during the execution of the image saving operation corresponding to the operation of the shutter button, but basically continues until the operation of the photographing unit 11 is completed.

[0079] More specifically, the photographic unit 11 sequentially outputs photographic images at the above frame rate, and the photographic images are output to the working memory 17. The photographic images are recorded in the working memory 17 after being subjected to signal processing by the image signal processing circuit 11E. In the live preview display, the photographic images are recorded in the working memory 17 as live view images, and are sequentially updated at the above frame rate. The display control unit 31 reads the live view images sequentially updated in the working memory 17 to sequentially update the display of the touch screen display 12, thereby realizing the live preview display. And, here, for the purpose of performing the live preview display, the photography performed by the photographic unit 11 is referred to as live preview photography.

[0080] Furthermore, when the user U operates the shutter button during the live preview display, an image saving instruction is input. When the image saving instruction is input, the photographic image captured at the time is saved in the user area 28 of the storage device 18 or in a preset saving destination in the memory card 29. When recording the photographic image in the memory card 29, the image processing unit 32 performs the recording via the media controller 19.

[0081] The captured image for storage may be an image recorded in the working memory 17 to be used as a live view image, or may be an image newly acquired by the imaging unit 11 at the operation timing of the shutter button.

[0082] As one of the applications 27, a photo synthesis application for synthesizing a live view image for instant preview display is installed in the mobile terminal 10. The photo synthesis application, for example, extracts a portion of the display image as a synthesis target image during the instant preview display of the display image as the live view image. Then, a synthesized image is generated by synthesizing the extracted synthesis target image 54 with a selected image different from the display image, and a function of instant preview displaying the generated synthesized image as the live view image is realized in the mobile terminal 10.

[0083] The photo composition application of this embodiment is an example of the image processing program of the present invention. By starting the photo composition application, the mobile terminal 10 of this embodiment functions as an example of the image processing device of the present invention. Figure 3 to Figure 5The following describes the configuration related to the function of the mobile terminal 10 corresponding to the activation of the photo composition application.

[0084] First, refer to Figure 3 and Figure 4 The functions related to the pre-processing of the photo composition in the mobile terminal 10 will be described. When the photo composition application is activated, the CPU 14 functions as the display control unit 31 and the image processing unit 32 .

[0085] like Figure 3 As shown, the working memory 17 is provided with an area that functions as an image memory 17A and a VRAM (Video Random Access Memory) 17B. The image memory 17A is an area for temporarily recording the photographic image output from the photographing unit 11 as a live view image. The VRAM 17B is an area for recording the live view image that has been processed by the image processing unit 32.

[0086] When the photo synthesis application is started, the photographing unit 11 starts to shoot a live preview of the display image 50. The display image 50 is recorded in the image memory 17A as a live view image. The display image 50 in the image memory 17A is sequentially updated to the latest photographic image output at the frame rate of the photographing unit 11. The display image 50 is a live view image shot by the photographing unit 11 and is a synthesis target image 54 (see Figure 5 ) of the extracted source image.

[0087] The display image 50 is read from the image memory 17A by the image processing unit 32. The display image 50 is subjected to image processing such as brightness correction by the image processing unit 32 and then recorded in the VRAM 17B. The display control unit 31 reads the display image 50 of the live view image from the VRAM 17B and displays it on the touch panel display 12 as a live preview.

[0088] Furthermore, as data used by the photo composition application of the application 27 , data of a confirmation frame 60 with an indicator and a mark 62 for specifying a color are recorded in the storage device 18 .

[0089] In the case of the live preview display image 50, as shown in FIG. Figure 4 As shown, the display control unit 31 causes a confirmation frame 60 with an indicator and a color specifying mark 62 to be superimposed on the display image 50 as a GUI for extracting the synthesis target image 54 in the display image 50 and to be displayed on the touch panel display 12 .

[0090] Figure 4FIG. 2 shows an example of a state in which the display control unit 31 causes a confirmation frame 60 with an indicator and a mark 62 for color designation to be displayed superimposed on a display image 50 displayed on the touch screen display 12 of the mobile terminal 10. The confirmation frame 60 with an indicator is a GUI used by the user U to allow the user to determine the range of extracting the synthesis target image 54 from the display image 50. The area determined by the confirmation frame 60 with an indicator is the determination area 52, and a part of the determination area 52 is extracted as the synthesis target image 54. Figure 4 As shown, the determination frame 60 with an indicator of the present embodiment includes an indicator 60A, a determination frame 60B, and an enlargement / reduction indication mark 60C. The determination frame 60B of the determination frame 60 with an indicator of the present embodiment is an example of a determination frame that determines a portion of an area in the display image of the present invention. In addition, the indicator 60A of the determination frame 60 with an indicator of the present embodiment is an example of an indicator that indicates the reference color of the present invention. The determination area 52 of the present embodiment is an example of a portion of an area in the display image of the present invention, and is an example of an image within the determination frame of the present invention.

[0091] like Figure 4 As shown, the area surrounded by the determination frame 60B is the determination area 52. In the present embodiment, the size of the determination frame 60B can be enlarged or reduced. In other words, the user U can change the size (range) of the determination area 52. As an example, in the present embodiment, the determination frame 60B is enlarged or reduced by the user U operating the enlargement / reduction indication mark 60C. If an operation instruction to enlarge or reduce the determination frame 60B is accepted, the main control unit 26 outputs information of the accepted operation instruction to the display control unit 31. The operation instruction to enlarge or reduce the determination frame 60B in the present embodiment is an example of a size change instruction to change the size of the determination frame 60B of the present invention. The display control unit 31 changes the determination frame 60B to a size corresponding to the information of the operation instruction input from the main control unit 26. The main control unit 26 of the present embodiment at this time is an example of the first acceptance unit of the present invention. In addition, in the present embodiment, as an example, the determination frame 60B can be enlarged or reduced, but it can also be a method in which at least one of enlargement and reduction can be performed.

[0092] Furthermore, in the present embodiment, the confirmation box 60B can be moved. In other words, the user U can change the position of the confirmation box 60B relative to the display image 50. If an operation instruction to move the confirmation box 60B is accepted, the main control unit 26 outputs information about the accepted operation instruction to the display control unit 31. The operation instruction to move the confirmation box 60B in the present embodiment is an example of an instruction to move the position of the confirmation box 60B of the present invention. The display control unit 31 moves the confirmation box 60 with the indicator to a position corresponding to the information about the operation instruction input from the main control unit 26. The main control unit 26 of the present embodiment at this time is an example of a second accepting unit of the present invention.

[0093] And, if Figure 4 As shown, a color designation mark 62 is displayed in the determination frame 60B. In the present embodiment, the relative positional relationship between the determination frame 60B and the color designation mark 62 is predetermined. As an example, in the present embodiment, the position of the color designation mark 62 is set to the position of the center (center of gravity) of the determination frame 60B. The color designation mark 62 is an image representing a mark for designating a reference color from the color of the image in the determination area 52. The color designation mark 62 of the present embodiment is an example of a color designation mark of the present invention. As an example, the color designation mark 62 of the present embodiment is a mark having a circular ring shape surrounding the display image 50, and the color of the display image 50 inside the circular ring shape is designated as the reference color.

[0094] The indicator 60A has a larger area than the color designation mark 62 , and displays the reference color designated by the color designation mark 62 .

[0095] Next, refer to Figure 5 The functions related to the composite photography processing in the mobile terminal 10 will be described. When the photo composite application is activated, the CPU 14 functions as the extraction unit 32A and the composite unit 32B of the image processing unit 32 .

[0096] The extraction unit 32A extracts a part of the display image 50 as a synthesis target image 54 for synthesizing an image. Specifically, the extraction unit 32A extracts the synthesis target image 54 from the determination area 52 determined from the display image 50. More specifically, the extraction unit 32A extracts an image corresponding to the reference color specified by the color specifying mark 62 from the determination area 52 determined by the determination box 60B of the determination box 60 with the indicator as the synthesis target image 54. The image that becomes the synthesis target image 54 can be, for example, an image representing a mark or a character string. The mark includes at least one of a character, a graphic, a pattern, and a combination of two or more selected from these. The character is a concept including a number and a symbol. The mark includes a logo representing a company name, a product name, a store name, a station name, or a place name. The pattern includes a stripe pattern, a checkered pattern, a dot pattern, etc. In addition, the mark also includes a mark composed of any one of a character, a number, or a symbol.

[0097] Character strings include messages composed of handwritten or printed characters, traffic safety slogans, greetings for welcome or pick-up at tourist attractions, advertisements for products, etc. Messages are words that convey thoughts to specific people, such as "Thank you for the other day" or "Congratulations on your admission", and include words of blessing, words of thanks, and greetings.

[0098] In addition, stamps or seals such as stamps provided to tourists at tourist attractions are also included in the mark. In addition, the characters included in the mark or character string can be Japanese, English, Chinese, Korean and other languages.

[0099] The extracting unit 32A extracts the region of the image corresponding to the reference color as the synthesis target image 54 based on the contrast of the image in the specific region 52. When the synthesis target image 54 is extracted from the display image 50, the extracted synthesis target image 54 is recorded in the image memory 17A.

[0100] The photographing unit 11 starts live preview photographing of the selected image 56 while the synthesis target image 54 is held in the image memory 17A. The selected image 56 is recorded in the image memory 17A as a live view image. The live preview display of the selected image 56 starts on the touch screen display 12. The selected image 56 is a live view image photographed by the photographing unit 11 and is a synthesis target image to be synthesized with the synthesis target image 54.

[0101] The synthesizing unit 32B generates a synthesized image by synthesizing the selected image 56 with the synthesized image 54. The selected image 56 synthesized with the synthesized image 54 is recorded in the VRAM 17B as a synthesized image. The display control unit 31 starts live preview display of the selected image 56 synthesized with the synthesized image 54 as a synthesized image.

[0102] Furthermore, the main control unit 26 receives an instruction of at least one of the display position, display posture, display size, and display color of the synthesized object image 54 in the touch screen display 12 according to the operation instruction input through the touch screen display 12. The operation instruction received by the main control unit 26 is input to the synthesizing unit 32B. The synthesizing unit 32B changes the display position of the synthesized object image 54 in the selection image 56 according to the input operation instruction.

[0103] Furthermore, in the case of normal photography, if the user U operates the shutter button during the live preview display of the selected image 56 to be synthesized with the synthesized target image 54, an image saving instruction is input. If the image saving instruction is input, the selected image 56 synthesized with the synthesized target image 54 is saved at the timing as a synthesized image for storage in the user area 28 of the storage device 18 or the memory card 29 in a preset storage destination. As an example, in the present embodiment, the image processing unit 32 performs compression processing on the synthesized image and records the compressed synthesized image in a preset storage destination.

[0104] Refer to the above Figure 1 and Figure 4 and Figure 6 to Figure 15 The function of the above-mentioned structure will be described.

[0105] As mentioned above, Figure 1 FIG. 4 shows a scene in which a user U uses the mobile terminal 10 to take a photo. Figure 1 The scene shown is a zoo with "pandas" visited by user U and friend T, and the shirt 80 worn by friend T has characters 82 "JAPANDA" printed on it. In this example, the following example is given for explanation: at this place, user U thinks of taking a composite photo combining two subjects, "panda" and the characters 82 "PANDA" in "JAPANDA", based on the similarity in the sense of language between "PANDA" and "JAPANDA", and uses the mobile terminal 10 to take the photo.

[0106] like Figure 6 As shown, in the mobile terminal 10, if the photo synthesis application is started, the mobile terminal 10 switches to the synthesis photography mode to perform the synthesis photography process. If the synthesis photography process is started, first, in step S100, the display control unit 31 displays the real-time preview of the display image 50 on the touch screen display 12. Specifically, the real-time preview photography of the display image 50 is started in the photography unit 11, and the real-time preview display of the display image 50 is started in the touch screen display 12.

[0107] In the next step S102, as described above Figure 4 As shown, the display control unit 31 causes a confirmation frame 60 with an indicator and a mark 62 for specifying a color to be superimposed on the display image 50 being displayed in live preview, and displays the same on the touch panel display 12 .

[0108] If the confirmation frame 60 with an indicator and the mark 62 for color designation are displayed, the user U adjusts the state so that the character 82 to be extracted falls within the confirmation frame 60B. In addition, in the mobile terminal 10 of the present embodiment, the color of the character 82 to be extracted is used as the reference color. Therefore, the user U adjusts the position of the color designation mark 62 to a position overlapping the image of the color of the character 82 to be extracted. For example, the user U adjusts the state so that the color designation mark 62 overlaps the character 82 to be extracted. As a method for making these adjustments, a method of adjusting the photographic range (i.e., the subject projected in the displayed image 50 itself) by moving the mobile terminal 10 by the user U can be cited. As a method for making these adjustments, an adjustment method based on the movement of the confirmation frame 60B (determination area 52) and an adjustment method based on the enlargement or reduction of the confirmation frame 60B (determination area 52) can be cited.

[0109] Therefore, in the next step S104, as described above, the main control unit 26 determines whether to accept the operation instruction to move the confirmation frame 60B. As an example, in the present embodiment, the operation instruction to move the confirmation frame 60B by the user U is performed by performing a gesture of sliding the finger in the direction to be moved while the user U touches the confirmation frame 60B displayed on the touch screen display 12 with one finger. This gesture is called a slide, etc.

[0110] If the main control unit 26 has not accepted the operation instruction to move the confirmation frame 60B, the determination in step S104 becomes negative, and the process moves to step S108. On the other hand, if the main control unit 26 has accepted the operation instruction to move the confirmation frame 60B, the determination in step S104 becomes positive, and the process moves to step S106.

[0111] In step S106, the display control unit 31 moves the confirmation frame 60B to a position corresponding to the operation instruction for moving the confirmation frame 60B. Specifically, the coordinate information indicating the moved confirmation frame 60B is output from the main control unit 26 to the display control unit 31 as the information of the operation instruction. The display control unit 31 displays the confirmation frame 60B at a position corresponding to the coordinate information input from the main control unit 26.

[0112] Figure 7 An example of display of the moved confirmation frame 60B is shown in FIG. Figure 7 In the example shown, the position of the determination frame 60B (more specifically, the position of the determination frame 60B relative to the display image 50) is shown to be moved to a position smaller than that of the display image 50. Figure 4 The example shown is an example of a state further downward (close to the operation switch 16 ).

[0113] In the next step S108, as described above, the main control unit 26 determines whether an operation instruction to enlarge or reduce the confirmation frame 60B is accepted. As an example, in the present embodiment, the operation instruction by the user U is performed by performing a gesture of sliding a finger in a direction corresponding to the size to be enlarged or reduced while the user U touches the enlargement / reduction instruction mark 60C displayed on the touch screen display 12 with one finger. In addition, the operation instruction method of enlarging or reducing the confirmation frame 60B is not limited to the present embodiment, and for example, it may be a method as follows: the user U performs a gesture such as a two-finger spread or a two-finger pinch on the confirmation frame 60B or the determination area 52 displayed on the touch screen display 12 to perform the operation instruction. The two-finger spread is a gesture of expanding the interval between two fingers while the display screen is touched with two fingers on the touch screen display 12. If the two-finger spread is performed while the confirmation frame 60B or the determination area 52 is touched with two fingers up, down, or left, and right, the confirmation frame 60B is enlarged and the area of ​​the determination area 52 is enlarged. Contrary to the two-finger spread, the two-finger pinch is a gesture of reducing the interval between the two fingers. If two fingers are moved inward, the confirmation frame 60B will be reduced, and the area of ​​the confirmation region 52 will be reduced.

[0114] If the main control unit 26 has not accepted the operation instruction of the enlargement or reduction confirmation frame 60B, the determination of step S108 becomes negative, and the process moves to step S112. On the other hand, if the main control unit 26 has accepted the operation instruction of the enlargement or reduction confirmation frame 60B, the determination of step S108 becomes positive, and the process moves to step S110.

[0115] In step S110, the display control unit 31 changes the size of the confirmation frame 60B to a size corresponding to the operation instruction of enlarging or reducing the confirmation frame 60B. By changing the size of the confirmation frame 60B, the size of the confirmation area 52 is also changed. Specifically, the coordinate information indicating the enlarged or reduced confirmation frame 60B is output from the main control unit 26 to the display control unit 31 as the information of the operation instruction. The display control unit 31 displays the confirmation frame 60B in a size (position) corresponding to the coordinate information input from the main control unit 26.

[0116] Figure 8 An example of display of the confirmation frame 60B whose size has been changed is shown in FIG. Figure 8 In the example shown, the ratio Figure 7 The example shown is a state in which the size of the determination frame 60B is further reduced.

[0117] In this embodiment, the user U wants to extract only the character 82 "PANDA" from the character 82 "JAPANDA". Figure 7As shown, the user U adjusts the confirmation box 60B as described above so that only the character 82 "PANDA" of the characters 82 "JAPANDA" falls within the confirmation box 60B and the character 82 "JA" falls outside the confirmation box 60B. Furthermore, the user U adjusts the confirmation box 60B as described above with reference to the color of the indicator 60A so that the reference color specified by the color specifying mark 62 becomes the color of the character 82 (in this example, the color specifying mark 62 overlaps the character 82). Figure 4 In the example shown, as shown by the indicator 60A, the reference color specified by the color specifying mark 62 is different from the color of the character 82, which is not preferable. Figure 8 In the example shown, the characters 82 included in the determination frame 60B are only the characters 82 "NDA", so the characters 82 "PA" will not be extracted, which is not preferred. Figure 7 In the example shown, the character 82 "PANDA" is included in the confirmation frame 60B, but the character 82 "JA" is not included, and the color designation mark 62 is superimposed on the character 82. As shown in the indicator 60A, the color of the character 82 is in the state designated by the color designation mark 62, which is preferable. Therefore, the user U adjusts the relationship between the display image 50 and the confirmation frame 60 with the indicator to Figure 7 Status shown.

[0118] In this way, if the adjustments of the position and size of the confirmation box 60B, the position of the color designation mark 62, etc. are completed, then in this embodiment, the extraction instruction of the synthetic object image 54 performed by user U is performed by clicking the shutter button 70 displayed as a GUI in the touch screen display 12 with a finger.

[0119] Therefore, in the next step S112, the main control unit 26 determines whether to accept the instruction to extract the above-mentioned synthetic target image 54. If the main control unit 26 has not accepted the instruction to extract the synthetic target image 54, the determination of step S112 becomes a negative determination, and returns to step S104 to repeat the processing of the above-mentioned steps S104 to S110. On the other hand, if the main control unit 26 has accepted the instruction to extract the synthetic target image 54, the determination of step S112 becomes an affirmative determination, and transfers to step S114.

[0120] In step S114, by Fig. 9 The synthetic object image extraction process illustrated in FIG. 5 extracts a synthetic object image 54 from a specific region 52 of a display image 50 .

[0121] exist Fig. 9In step S150 shown in FIG. 1 , the extraction unit 32A determines the determination area 52 from the display image 50. Specifically, as described above, when the user U operates the shutter button 70 to instruct extraction, the main control unit 26 outputs coordinate information corresponding to the position of the determination frame 60B to the extraction unit 32A. The extraction unit 32A determines the determination area 52 from the display image 50 based on the coordinate information input from the main control unit 26.

[0122] In the next step S152, the extraction unit 32A grayscales the entire determination region 52. Specifically, the extraction unit 32A temporarily converts the colors of all pixels included in the determination region 52 into grayscale pixel values ​​from white to black represented by 256 gradations of pixel values ​​from 0 to 255.

[0123] The method of converting the color of each pixel into grayscale is not particularly limited, and existing technologies can be applied. For example, when the determined area 52 is an RGB image, the grayscale pixel value Gray is obtained from the pixel value of Red (R: red), the pixel value of Green (G: green), and the pixel value of Blue (B: blue) in each pixel according to the following formula (1).

[0124] Gray=Red×0.3+Green×0.59+Blue×0.11……(1)

[0125] Fig.10 An example of graying the image of the specific area 52 is shown in FIG. Fig.10 The determination area 52 shown is the same as that in the above Figure 7 In the illustrated state, the determination area 52 is determined by the determination frame 60 with an indicator. Fig.10 The specified area 52 shown includes the friend T, the background B, the shirt 80, and the characters 82 as subjects, and includes the respective images.

[0126] In the next step S154, the extraction unit 32A determines whether the reference color is white. As an example, in the present embodiment, in the above step S152, the pixels with grayscale values ​​of 0 to 127 are regarded as black pixels, and the pixels with pixel values ​​of 128 to 255 are regarded as white pixels. That is, in the present embodiment, the color of the pixels in the grayscale determination area 52 is binarized to either white or black.

[0127] When the pixel value of the pixel of the reference color (specifically, the pixel value of the pixel of the display image 50 surrounded by the color designation mark 62) is any one of 128 to 255, the extraction unit 32A determines that the reference color is white. When the reference color is white, the determination of step S154 becomes an affirmative determination, and the process moves to step S156. The range of 128 to 255 of the pixel value in this embodiment is an example of the reference color range including the grayscale value corresponding to the reference color of the present invention.

[0128] In step S156 , the extraction unit 32A extracts a region of white pixels (that is, pixels of colors whose pixel values ​​are 128 to 255) in the specific region 52 as the synthesis target image 54 .

[0129] On the other hand, when the pixel value of the pixel of the reference color (specifically, the pixel value of the pixel of the display image 50 surrounded by the color designation mark 62) is any one of 0 to 127, the extraction unit 32A determines that the reference color is black. When the reference color is black, the determination of step S154 becomes a negative determination, and the process moves to step S158. The range of 0 to 127 of the pixel value in this embodiment is an example of a reference color range including the grayscale value corresponding to the reference color of the present invention.

[0130] In step S158 , the extraction unit 32A extracts a region of black pixels (that is, pixels of colors with pixel values ​​of 0 to 127) in the specific region 52 as the synthesis target image 54 .

[0131] exist Fig.10 In the example shown, the pixel values ​​of the pixels in the character 82A "P", the character 82B "A", the character 82C "N", the character 82D "D", and the character 82E "A" obtained by graying the image of the specific area 52 are within the range of 0 to 127. Therefore, the character 82 is determined to be a black pixel. On the other hand, the pixel values ​​of the pixels in the friend T, the background B, and the shirt 80 are within the range of 128 to 255, and are determined to be white pixels.

[0132] Therefore, in the example of this embodiment, if Fig.10 As shown, the extraction unit 32A determines that the reference color is black, and extracts the images of the character 82A "P", the character 82B "A", the character 82C "N", the character 82D "D", and the character 82E "A" from the determination area 52 as the synthesis target image 54. The extraction unit 32A regards the character 82A "P", the character 82B "A", the character 82C "N", the character 82D "D", and the character 82E "A" as one synthesis target image 54.

[0133] When the processing of step S156 or step S158 is completed, the synthesis target image extraction processing is completed. When the synthesis target image extraction processing is completed, step S114 of the synthesis photography processing is completed, and the process moves to step S116.

[0134] When the synthesis target image 54 is extracted, the photographing unit 11 immediately starts live preview photographing of the selected image 56 in step S116 .

[0135] When live preview photography of the selected image 56 is started, in the next step S118, the synthesis unit 32B synthesizes the extracted synthesis target image 54 with the selected image 56. In the present embodiment, images other than the synthesis target image 54 in the specific area 52 are regarded as transparent images.

[0136] In the next step S120 , the display control unit 31 displays the selected image 56 to be combined with the combination target image 54 on the touch panel display 12 as a live preview.

[0137] Fig.11 4 shows a situation where a selected image 56 is synthesized with a character 82 as a synthesized target image 54 in real-time preview display. Fig.11 As shown, when the selected image 56 synthesized with the synthesized object image 54 is displayed in real-time preview, the display control unit 31 displays the character 82 as the synthesized object image 54 at the center of the selected image 56 (that is, the center of the display screen of the touch screen display 12) in the initial state. Fig.11 As shown, the display control unit 31 displays a shutter button 70 and a color palette 72 as a GUI, for example, at the lower position of the display screen of the touch screen display 12. As described later, the color palette 72 is an operation button for changing the color of the synthesis target image 54. The shutter button 70 is an operation button for inputting an image saving instruction, and is an operation button for saving the selected image 56 synthesized with the synthesis target image 54 as a synthesized image.

[0138] The user U determines the composition of the selected image 56 and the relative positional relationship between the image of the subject in the selected image 56 and the image to be synthesized 54 while viewing the live preview display of the selected image 56 synthesized with the image to be synthesized 54 in this way.

[0139] Specifically, if Fig.11 As shown, the user U wants to take a composite photo combining the characters 82 "PANDA" and "Panda". Therefore, the user U adjusts the position and orientation of the mobile terminal 10 to include "Panda" as the subject of the selected image 56. Then, the position, size, and viewing angle of "Panda" in the selected image 56 are adjusted. The user U determines the composition of the selected image 56 through such operations.

[0140] While determining the composition of the selected image 56, the user U also determines the relative positional relationship between the "panda" as the subject of the selected image 56 and the character 82 as the synthesis target image 54. At this time, the display position, display posture, display size, display color, etc. of the character 82 as the synthesis target image 54 are edited.

[0141] In step S122, the main control unit 26 determines whether to edit the synthesis target image 54 in the selected image 56. If the editing instruction by the user U operating the touch screen display 12 is not input within a predetermined time after the transition to step S122, the determination of step S122 becomes a negative determination, and the process transitions to step S126. On the other hand, if the editing instruction by the user U operating the touch screen display 12 is input within a predetermined time after the transition to step S122, the determination of step S122 becomes an affirmative determination, and the process transitions to step S124.

[0142] In step S124 , the image processing unit 32 edits the synthesis target image 54 in the selected image 56 according to the content of the editing instruction. Figure 12 to Figure 15 This is an example of the synthesized target image 54 corresponding to the editing instruction.

[0143] Fig.12 FIG. 4 shows an example of moving the display position of the character 82 as the synthesis target image 54. Fig.12 The example shown shows a state where the image processing unit 32 moves the display position of the character 82 downward (toward the side close to the operation switch 16). When the user U issues an editing instruction to move the display position, the user U performs a gesture of sliding the finger in the desired direction of movement while touching the character 82 with one finger on the touch screen display 12.

[0144] and, Fig.13 , an example of enlarging the display size of the character 82 as the synthesized target image 54 is shown. When the user U issues an editing instruction to enlarge / reduce the display size, for example, it is performed by a gesture such as two-finger spread or two-finger pinch. If the two-finger spread is performed while touching the top, bottom or left, right of the character 82 with two fingers, the display size of the character 82 is enlarged. On the other hand, if the two-finger pinch is performed, the display size of the character 82 is reduced.

[0145] and, Fig.14 , an example of a display gesture of rotating a character 82 as a synthesized object image 54 is shown. When the user U gives an editing instruction to rotate the display gesture, on the touch screen display 12, a gesture of rotating the two fingers while maintaining the interval between the two fingers is performed while touching the character 82 up and down or left and right with two fingers.

[0146] and, Fig.15 FIG. 4 shows an example of changing the color of the character 82 as the synthesis target image 54. Fig.15 As shown in FIG. 1 and FIG. 2 , a plurality of colors are arranged on the color palette 72. When the user U issues an editing instruction to change the color of the character 82, for example, the user performs a gesture of clicking a desired color from among the plurality of colors arranged on the color palette 72 with a finger. Fig.15 In the example shown, a state is shown in which the color of the character 82 is changed to white by the user U clicking the white color 72W arranged on the color palette 72 with a finger to give an edit instruction.

[0147] In addition, in the present embodiment, the method of changing the color of the character 82 as the synthesized object image 54 using the color palette 72 is described, but the method of changing the synthesized object image 54 is not limited to the method using the color palette 72. For example, a color change button (not shown) is displayed on the selection image 56, and when the user U clicks it with a finger, the color is changed between a plurality of colors predetermined in advance, such as white and black, white and red, white, black and red, etc., according to the number of times the color change button is clicked. The color change of the synthesized object image 54 is performed, for example, to avoid the color of the synthesized object image 54 and the color of the background part of the selection image 56 overlapping in color, which makes it difficult to observe the synthesized object image 54. Alternatively, the color change may be performed to change the color of the synthesized object image 54 to match the background color according to the sensibility of the user U, etc.

[0148] The main control unit 26 accepts such a gesture as an edit instruction for the synthesis target image 54. The main control unit 26 outputs the information of the accepted edit instruction to the synthesis unit 32B. The synthesis unit 32B edits the synthesis target image 54 in the selection image 56 according to the information of the edit instruction input from the main control unit 26.

[0149] When the editing of the synthesis target image 54 is finished, the user U clicks the shutter button 70 with a finger. When the shutter button 70 is clicked, the main control unit 26 accepts the click operation as an instruction to save the synthesized image.

[0150] In step S126, the main control unit 26 determines whether the instruction to save the synthesized image is accepted. If the main control unit 26 does not accept the instruction to save the synthesized image, the determination in step S126 becomes negative, and the process returns to step S122 to repeat the processes in steps S122 and S124.

[0151] On the other hand, when the main control unit 26 has accepted the instruction to save the synthesized image, the determination in step S126 becomes affirmative, and the process moves to step S128 .

[0152] The main control unit 26 outputs the information of the received storage instruction of the composite image to the synthesis unit 32B. When the information of the storage instruction of the composite image is input from the main control unit 26, the image processing unit 32 determines the selected image 56 captured at the timing when the shutter button 70 is clicked, and determines the display position of the synthesis target image 54 in the determined selected image 56. The synthesis unit 32B synthesizes the synthesis target image 54 of which the display position has been determined with the determined selected image 56, and outputs it to the image memory 17A as a composite image for storage.

[0153] In the next step S128, the image processing unit 32 stores the synthesized image. Specifically, the image processing unit 32 compresses the synthesized image for storage output to the image memory 17A, and stores the compressed synthesized image in the user area 28 of the storage device 18 or the memory card 29.

[0154] In the next step S130, the main control unit 26 determines whether to terminate the composite photography mode. In the present embodiment, when the main control unit 26 has accepted an operation instruction to terminate the composite photography mode by operating the operation switch 16 or the like by the user U, the composite photography mode is terminated. Therefore, when the main control unit 26 has not accepted an operation instruction to terminate the composite photography mode, the determination in step S130 becomes a negative determination, and the process returns to step S100 to repeat the process of steps S100 to S128. On the other hand, when the main control unit 26 has accepted an operation instruction to terminate the composite photography mode, the determination in step S130 becomes a positive determination, and the composite photography mode is terminated.

[0155] Fig.16 FIG. 2 is a diagram summarizing the synthetic photography processing in the mobile terminal 10 of the present embodiment. Fig.16 As shown in FIG. 1 , the mobile terminal 10 includes a display control unit 31, an extraction unit 32A, and a synthesis unit 32B. The display control unit 31 displays a determination frame 60B that specifies a determination area 52 that is a partial area in a display image 50 displayed on the touch screen display 12 and a color designation mark 62 for designating a reference color from the colors included in the image of the determination area 52 in the determination frame 60B, in an overlapping manner on the display image 50. The extraction unit 32A extracts an image corresponding to the reference color designated by the color designation mark 62 from the image of the determination area 52 in the determination frame 60B as a synthesis target image 54. The synthesis unit 32B generates a synthesized image by synthesizing the synthesis target image 54 with the selection image 56 selected by the user U.

[0156] That is, in the mobile terminal 10 of this embodiment, by specifying the color of the image to be extracted from the determination area 52 (i.e., the image to be used as the synthesis target image 54) as the reference color using the color specifying mark 62 in the determination frame 60B, a synthetic image desired by the user U can be generated.

[0157] According to the mobile terminal 10 of the present embodiment, when specifying an image to be used as the synthesis target image 54, the user U only needs to overlap the color specification mark 62 on the color of the image to be used as the synthesis target image 54, so the user U does not need to directly specify the image to be used as the synthesis target image 54 from the multiple images included in the display image 50. For example, the user U does not need to perform a gesture such as touching the area of ​​the image to be used as the synthesis target image 54 with one finger or sliding one finger to surround the entire area. Therefore, according to the mobile terminal 10 of the present embodiment, even if the operation is not performed on the touch screen display 12, it is possible to specify the image to be used as the synthesis target image 54, so for example, even if only one hand holding the mobile terminal 10 is used, it is possible to specify the image to be used as the synthesis target image 54. Therefore, according to the mobile terminal 10 of the present embodiment, the operability is further improved, and synthesis photography corresponding to various scenes can be performed.

[0158] Furthermore, according to the mobile terminal 10 of the present embodiment, even if the image to be synthesized 54 has a complicated shape, it is possible to appropriately extract the image to be synthesized 54 from the specific area 52 through a simple operation.

[0159] Therefore, according to the mobile terminal 10 of the present embodiment, creation of a composite image is simplified, and operability of the user U related to the composite image is improved.

[0160] Furthermore, in the mobile terminal 10 of the present embodiment, the relative positional relationship between the confirmation box 60B and the color specification mark 62 is predetermined. Therefore, in the mobile terminal 10 of the present embodiment, the user U does not need to specify both the position of the confirmation box 60B and the position of the color specification mark 62, thereby further improving the operability of the user U.

[0161] Furthermore, in the mobile terminal 10 of the present embodiment, the determination frame 60B can be enlarged and reduced. Furthermore, in the mobile terminal 10 of the present embodiment, the determination frame 60B can be moved. For example, the position of the synthesis target image 54 relative to the display image 50 may inevitably become a position offset due to restrictions on the shooting position, etc. Furthermore, for example, an image that is not intended to be included in the synthesis target image 54 may be scattered in the display image 50. In the mobile terminal 10 of the present embodiment, since the determination frame 60B can be enlarged and reduced, even in such a case, the desired image that the user U wants to use as the synthesis target image 54 can be appropriately included in the range of the determination frame 60B (that is, in the determination area 52) on the display image 50. Furthermore, similarly, in the mobile terminal 10 of the present embodiment, since the determination frame 60B can be moved, the desired image that the user U wants to use as the synthesis target image 54 can be appropriately included in the range of the determination frame 60B (that is, in the determination area 52) on the display image 50.

[0162] In the mobile terminal 10 of this embodiment, the color of the display image 50 surrounded by the color designation mark 62 is used as the reference color. In the mobile terminal 10 of this embodiment, the image of the reference color is displayed inside the color designation mark 62, so the user U can confirm the designated reference color.

[0163] Furthermore, in the mobile terminal 10 of the present embodiment, the reference color is displayed in the indicator 60A having a larger area than the color specifying mark 62. The user U can confirm the reference color specified by the color specifying mark 62 through the indicator 60A, and thus can easily confirm the reference color even when the color specifying mark 62 is small, for example.

[0164] Furthermore, in the mobile terminal 10 of the present embodiment, the grayscale of the entire determination area 52 of the determination frame 60B is numerically digitized, and an image whose grayscale value is within the reference color range including the grayscale value corresponding to the reference color is extracted as the synthesis target image 54. In the mobile terminal 10 of the present embodiment, the synthesis target image 54 can be extracted from the determination area 52 based on the numerical value obtained by digitizing the grayscale, so that the synthesis target image 54 can be easily extracted in the mobile terminal 10, and the processing load in the mobile terminal 10 can be reduced.

[0165] [Modification 1]

[0166] In the above-described embodiment, a method is described in which the reference color range is determined in advance when the extraction unit 32A extracts the synthesis target image 54 from the determination area 52. In contrast, in this modification, a method is described in which the user U sets the reference color range.

[0167] Fig.17 FIG. 4 shows an example of a state in which the display control unit 31 displays a real-time preview of the selected image 56 to be synthesized with the synthesis target image 54 on the touch panel display 12 in step S120 of the synthesis photography process. Fig.17 In the illustrated example, in order to make the synthesis target image 54 (character 82 ) easier to see, the display control unit 31 inserts a semi-transparent background image between the selection image 56 and the synthesis target image 54 .

[0168] In the determination area 52, the characters 82, the friend T, the background B, the shirt 80, and the water droplets 84 are shown in the image (refer to Fig.10 ) is relatively small, sometimes the extraction unit 32A may fail to extract only the character 82, and may not become Fig.17 For example, when the pixel value of the grayscaled background B pixel is any value between 0 and 127, the extraction unit 32A determines that the color of the background B pixel is also black, which is the same as the reference color. At this time, the extraction unit 32A extracts the character 82 and the background B as the synthetic object image 54. Therefore, Fig.18 As shown, the display control unit 31 displays the selection image 56 synthesized with the characters 82 and the background B as the synthesis target images 54 on the touch panel display 12 .

[0169] In this case, in this modification, the user U sets a threshold value that becomes a boundary of the reference color range (in other words, a threshold value for determining whether the reference color is white or black) to set the reference color range, thereby excluding the background B from the synthesized target image 54. In this modification, Fig.18 As shown, the display control unit 31 displays, for example, a setting button 73 for setting a threshold value that becomes a boundary of the reference color range as a GUI at the lower position of the display screen of the touch screen display 12. By performing an operation instruction by the user U clicking the setting button 73 with one finger, as shown in FIG. Fig.19 As shown, the display control unit 31 displays a threshold value setting tool 74 as a GUI. The threshold value setting tool 74 includes a slide bar 74A and a slider 74B representing colors of pixel values ​​from 0 to 255.

[0170] The user U changes the position of the slider 74B by sliding a finger along the slider 74A while touching the slider 74B with one finger. When the position of the slider 74B is changed, the extraction unit 32A changes the threshold value that is the boundary of the reference color range to the color of the slider 74A corresponding to the changed position (specifically, the pixel value of the pixel of the slider 74A specified by the slider 74B). Fig.19In the example shown, as the slider 74B moves to the right, the threshold value that becomes the boundary of the reference color range becomes larger and closer to white. On the other hand, as the slider 74B moves to the left, the threshold value becomes smaller and closer to black.

[0171] In the case where the background B is excluded from the synthetic object image 54, as shown in FIG. Fig. 20 As shown in FIG. 1 , the user U moves the slider 74B to a value between the color of the background B and the color of the character 82. In this way, by moving the slider 74B by the user U, the threshold value that becomes the boundary of the reference color range becomes a value between the pixel value of the pixel of the background B and the pixel value of the pixel of the character 82. In this state, the above-mentioned synthetic object image extraction process (see FIG. 1 ) is performed again by the mobile terminal 10. Fig. 9 ),like Fig. 20 As shown, it is possible to set the state in which only the characters 82 are included in the synthesis target image 54 .

[0172] As described above, in this modification, the user U can set a threshold value that serves as a boundary of the reference color range (in other words, a threshold value that serves as a boundary of the contrast between the synthesis target image 54 and other images).

[0173] Therefore, according to the mobile terminal 10 of this modification, only the image desired by the user U can be extracted from the determination area 52 as the synthesis target image 54 through a simple operation. In addition, the user U can set the threshold value that becomes the boundary of the reference color range while observing the color of the slide bar 74A, so the threshold value can be set more intuitively.

[0174] In addition, the threshold setting tool 74 is not limited to the form including the slide bar 74A and the slider 74B. For example, the threshold setting tool 74 may also be a form including a color space and a pointer for specifying the coordinates of the color space. In this case, the pixel value corresponding to the coordinates of the color space specified by the user U with the pointer becomes the threshold of the boundary of the reference color range.

[0175] Furthermore, in the above-mentioned embodiment and this variation, a method of converting grayscaled pixels into two values ​​and using one of the values ​​as the reference color range is described, but the reference color range is not limited to this method. For example, a method of converting the values ​​into three values ​​or more and using the range including the reference color pixel value as the reference color range may also be used. Furthermore, for example, a method of using a predetermined range (in the specific example, the range of ±20 of the reference color pixel value) from the pixel value of the reference color pixel as the reference color range may also be used.

[0176] [Modification 2]

[0177] In the above-mentioned methods, the method of specifying the color of the character 82 to be synthesized as the reference color by the color specifying mark 62 is described, but sometimes the synthesized object image 54 cannot be properly extracted because the color of the character 82 is not properly specified as the reference color. In contrast to this situation, in this modified example, a modified example of the method of specifying the color of the character 82 as the reference color is described.

[0178] For example, Fig.21 As shown, when the character 82 is smaller than the color designation mark 62, the proportion of the character 82 in the display image 50 surrounded by the color designation mark 62 is smaller than that of other images (in Fig.21 In the example of shirt 80). When the display image 50 surrounded by the color designation mark 62 includes a plurality of images, if the color of the image with a high ratio is used as the reference color, the color of the pixel of shirt 80 becomes the reference color. As described above, the extraction unit 32A determines the color of the pixel of shirt 80 to be white, so the reference color becomes white. Therefore, Fig. 22 As shown, the extraction unit 32A extracts the friend T, the background B, and the shirt 80 determined to be white, as the synthesis target images 54 .

[0179] In this case, in this modification, the user U can use the color of the image to be extracted (the color of the character 82 in this modification) as the reference color by inverting the reference color. Fig.21 As shown, the display control unit 31 displays, for example, an inversion button 76 for inverting the reference color as a GUI at the lower position of the display screen of the touch screen display 12. When the user U clicks the inversion button 76 with one finger to give an inversion instruction, the main control unit 26 receives the inversion instruction and outputs the inversion instruction information to the extraction unit 32A. When the inversion instruction information is input from the main control unit 26, the extraction unit 32A causes the above-mentioned synthetic object image extraction process (see Fig. 9 ) in step S154, and extract the synthesis object image 54 based on the inverted reference color.

[0180] The extraction unit 32A extracts the synthesis target image 54 based on the inverted reference color. In this modification, black is used as the reference color, and the extraction unit 32A extracts the synthesis target image 54 using the color of the character 82 as the reference color. Fig. 22 As shown, the extraction unit 32A can extract the characters 82 as the synthesis target image 54 .

[0181] Therefore, according to the mobile terminal 10 of the present modification, it is possible to appropriately extract the synthesis target image 54 from the specific area 52 through a simple operation.

[0182] In addition, the timing at which the extraction unit 32A inverts the reference color is not limited. For example, the following method may be used: when the user U feels that the character 82 is too thin to accurately align the color designation mark 62 on the character 82, or when the user U feels that the color displayed in the indicator 60A is different from the color of the character 82, the inversion instruction is given before the extraction instruction of the synthesis target image 54 is given. In addition, the following method may be used: after the selected image 56 synthesized with the synthesis target image 54 is previewed, the user U who feels that the synthesis target image 54 is not the desired image gives the inversion instruction.

[0183] Furthermore, in this modification, a method of appropriately extracting the synthetic target image 54 by reversing the reference color when the synthetic target image 54 cannot be appropriately extracted is described, but the method for appropriately extracting the synthetic target image 54 is not limited to this modification. For example, a method may be used in which the relationship between the image extracted as the synthetic target image 54 and the image not extracted is reversed. In the case of this modification, as described above, if white is used as the reference color, the extraction unit 32A extracts the friend T, the background B, and the shirt 80 separately, but does not extract the character 82. The extraction unit 32A can extract the character 82 without extracting the friend T, the background B, and the shirt 80 separately by reversing (changing) the relationship of whether to extract.

[0184] [Variation 3]

[0185] In this modification, a modification of the confirmation frame 60B of the confirmation frame with indicator 60 will be described. The “frame” of the confirmation frame 60B only needs to clearly show the range of the confirmation area 52.

[0186] For example, Figures 23A to 23C As shown, in the determination frame 60B, a portion of the line surrounding the determination area 52 may be discontinuous. Fig.23A and Fig. 23B The determination box 60B shown has a so-called square bracket shape. Fig.23A and Fig. 23B In the manner shown, the determination box 60B can also clearly show that the range of the determination area 52 is the range marked with oblique hatching in the figure. Fig.23C The determination box 60B has a so-called right-angle quotation mark shape. Fig.23C In the manner shown, the determination box 60B can also clearly present that the range of the determination area 52 is the range marked with oblique hatching in the figure.

[0187] And, for example, Fig.23D As shown, the determination frame 60B may also be a mark shape indicating the framed position. Fig.23D In the manner shown, the determination box 60B can also clearly present that the range of the determination area 52 is the range marked with oblique hatching in the figure.

[0188] Furthermore, the confirmation frame 60B is not limited to a rectangular shape, but may also be other shapes. Fig.23E and Fig.23F As shown, the determination frame 60B may also be a circular frame.

[0189] [Variation 4]

[0190] In this modification, a modification of the indicator 60A of the confirmation frame 60 with an indicator will be described.

[0191] In the above description, the indicator 60A is arranged on the frame of the confirmation frame 60B, and the indicator 60A and the confirmation frame 60B are integrally regarded as the confirmation frame 60 with the indicator. However, the indicator 60A and the confirmation frame 60B may be regarded as independent.

[0192] In the case where the indicator 60A and the determination box 60B are considered to be independent, for example, Fig.24A As shown, the display control unit 31 may display the indicator 60A at a position away from the determination frame 60B (eg, displaying the indicator 60A at a position above the determination frame 60B in the display image 50 ).

[0193] Furthermore, the shape of the indicator 60A is not particularly limited. In the above-mentioned embodiments, the case where the shape of the indicator 60A is circular is described, but for example, Fig. 24B As shown, the shape of the indicator 60A may be a rectangle. Also, the shape of the indicator 60A may be the same as the shape of the color designation mark 62, or the shape of the indicator 60A may be the same as the shape of the confirmation frame 60B.

[0194] [Variation 5]

[0195] This modification will describe a modification of the color designation mark 62. In the above, the color designation mark 62 is described as a mark having a ring shape surrounding the display image 50. However, the shape of the color designation mark 62 is not limited to a ring shape.

[0196] For example, Fig.25A As shown, the shape of the color designation mark 62 may be circular or a so-called solid circular image. In this case, the color of the portion of the display image 50 overlapped by the color designation mark 62 is designated as the reference color.

[0197] And, for example, Fig.25BAs shown, the shape of the color specification mark 62 may be a rectangle. Also, for example, the shape of the color specification mark 62 may be the same as the shape of the confirmation frame 60B.

[0198] [Variation 6]

[0199] Furthermore, in the above description, the display image 50 and the selected image 56 are live view images that can be previewed immediately, but at least one of the display image 50 and the selected image 56 may not be a live view image. That is, at least one of the display image 50 and the selected image 56 may be an image read out from a photographic image (hereinafter referred to as a stored image) that was previously photographed and stored in the memory card 29 or the user area 28 of the storage device 18.

[0200] First, a case where the display image 50 is not a live view image but a stored image will be described. Fig.26A 2 is a flowchart of the synthetic photography process when the display image 50 is a stored image. Fig.26A The composite photography process shown includes the processes of steps S101A to S101C instead of the above Figure 6 The composite photography process is shown in step S100.

[0201] In step S101A, the display control unit 31 creates a list of stored images and displays it on the touch panel display 12. The list of stored images may be a list of stored images in which thumbnail images of the stored images are displayed side by side, for example.

[0202] The user U refers to the list of stored images and performs an operation instruction for selecting a stored image to be included in the image to be synthesized as the target image 54. As an example, in this modification, the user U performs the operation instruction for selecting a stored image by touching a thumbnail image of the stored image displayed on the touch screen display 12 with one finger. If the main control unit 26 receives the operation instruction for selecting a stored image, it outputs information of the operation instruction to the display control unit 31.

[0203] In the next step S101B, the display control unit 31 determines whether the user has selected to save the image. The determination in step S101B becomes a negative determination until information of an operation instruction for selecting to save the image is input from the main control unit 26. On the other hand, when information of an operation instruction for selecting to save the image is input from the main control unit 26, the determination in step S101B becomes a positive determination, and the process moves to step S101C.

[0204] In step S101C, the display control unit 31 displays one stored image corresponding to the information of the operation instruction from among the stored stored images on the touch panel display 12 as the display image 50 .

[0205] The processing after the next step S102 is the same as the above. In this example, under the control of the main control unit 26, the stored image is read from the memory card 29 or the user area 28 as the display image 50. The read display image 50 is recorded in the image memory 17A. The display image 50 read from the memory card 29 or the user area 28 is compressed. The image processing unit 32 reads the display image 50 from the image memory 17A and performs expansion processing on the display image 50. The extraction unit 32A extracts the synthesis object image 54 from the expanded display image 50.

[0206] Next, a case where the selected image 56 is not a live view image but a stored image will be described. Fig.26B 2 shows a flowchart of the composite photography process when the image 56 is selected as the storage image. Fig.26B The synthetic photography processing shown includes the processing of steps S117A and S117B instead of the above Figure 6 The composite photography process shown is step S116.

[0207] In step S117A, similarly to step S101A described above, the display control unit 31 creates a list of stored images and displays it on the touch panel display 12 .

[0208] The user U refers to the list of stored images and performs an operation instruction for selecting a stored image to be used as the selected image 56 in the same manner as the operation instruction for selecting a stored image to be used as the synthesis target image 54 from the list of stored images. If the operation instruction for selecting a stored image is accepted, the main control unit 26 outputs information on the operation instruction to the display control unit 31.

[0209] In the next step S117B, the display control unit 31 determines whether the user has selected to save the image, similarly to the above-mentioned step S101B. The determination of step S117B becomes a negative determination until information of an operation instruction for selecting to save the image is input from the main control unit 26. On the other hand, when information of an operation instruction for selecting to save the image is input from the main control unit 26, the determination of step S117B becomes a positive determination, and the process moves to step S118.

[0210] The processing after the next step S118 is the same as the above. In addition, in the case of this example, under the control of the main control unit 26, one stored image corresponding to the information of the operation instruction is read from the stored images stored in the memory card 29 or the user area 28 as the selected image 56. The read selected image 56 is recorded in the image memory 17A. The selected image 56 read from the memory card 29 or the user area 28 is compressed. The image processing unit 32 reads the selected image 56 from the image memory 17A and performs expansion processing on the selected image 56. The synthesis unit 32B synthesizes the synthesis target image 54 with the expanded selected image 56.

[0211] In addition, in the case where the display image 50 and the selected image 56 are not live view images but stored images, the above Fig.26A The composite photography process shown and Fig.26B The synthetic photography processing shown is sufficient.

[0212] Thus, if past photographic images can be used as the display image 50, for example, when the user U associates a mark or a character string that is reflected in a photographic image taken in the past with a subject found by the user U at the travel destination, a composite photo of the subject and the mark or character string associated with the subject can be obtained. And, for example, conversely, when the user U associates a photographic image taken in the past with a subject of the mark or character string found by the user U at the travel destination, a composite photo of the mark or character string that is the current subject and the photographic image associated with the subject can be obtained.

[0213] Furthermore, the composite image that is composited and recorded in the memory card 29 or the user area 28 may be used as at least one of the display image 50 and the selection image 56. In this way, multiple composite images can be performed, thereby further increasing the number of ways to enjoy the image.

[0214] Furthermore, as at least one of the display image 50 and the selection image 56, not only a stored image stored in the memory card 29 or the user area 28 but also an image downloaded from a server via the Internet or the like may be used as at least one of the display image 50 and the selection image 56. When an image is downloaded from a server, it is also temporarily recorded in the memory card 29 or the user area 28, so the subsequent processing is the same as when the stored image is used from the memory card 29 or the user area 28 as at least one of the display image 50 and the selection image 56.

[0215] As described above, in the mobile terminal 10 of each of the above-mentioned embodiments, by specifying the color of the image to be extracted from the determination area 52 (that is, the image to be used as the synthesis target image 54) as the reference color using the color specifying mark 62 in the determination frame 60B, it is possible to generate a synthesized image desired by the user U. Therefore, according to the mobile terminal 10 of each of the above-mentioned embodiments, creation of a synthesized image is simplified, and operability of the user U related to synthesis is improved.

[0216] In addition, the technology of the present invention can be variously modified within the scope of the above-mentioned modes without departing from the purpose of the present invention. For example, when extracting the synthetic object image 54 from the display image 50, the subject displayed in the synthetic object image 54 is sometimes a curved object such as a cylindrical shape. In this case, the extraction unit 32A can be set to perform a planarization process on the synthetic object image 54 extracted from the determination area 52 to flatten the curved surface. This planarization process can be performed according to the operation instruction of the user U, or it can be automatically performed by the extraction unit 32A after determining the shape of the subject.

[0217] Furthermore, in each of the above-mentioned methods, a mobile terminal 10 such as a smartphone is used as an example of a photographic device, but it may be a mobile terminal other than a smartphone, such as a tablet terminal, or an ordinary digital camera as a dedicated camera.

[0218] Furthermore, the device incorporating the image processing device of the present invention is not limited to the mobile terminal 10, and may be, for example, incorporated in a general-purpose personal computer or the like, or may be incorporated in a device without a photographing function. In addition, when incorporated in a device without a photographing function, it is a device without a live preview display function, so both the display image 50 and the selected image 56 are images that are not live view images (i.e., still images).

[0219] Furthermore, the image processing program, which shows a photo synthesis application as an example, may be pre-installed in the storage device 18 when the mobile terminal 10 is sold, or the user may purchase the mobile terminal 10 and then use the mobile terminal 10 to access a server and download a startup program from the server for installation.

[0220] Furthermore, the processing of the composite photography mode executed by the mobile terminal 10 of the present invention is only an example, and therefore, it is of course possible to delete unnecessary steps, add new steps, or change the processing order within the scope of the present invention.

[0221] Furthermore, in the above embodiment, Figure 6 The combination of a plurality of steps of the composite photography mode shown as an example in FIG. 1 corresponds to an example of the image processing method of the present invention.

[0222] Furthermore, in the above-mentioned embodiment, for example, as a hardware structure of a processing unit (Processing Unit) that performs various processes such as a main control unit, an extraction unit, a synthesis unit, and a display control unit, various processors (Processor) shown below can be used. As various processors, in addition to a CPU that is a general-purpose processor that executes software (programs) to perform the functions of various processing units, there are also PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) that can be changed into circuit structures after manufacturing, and dedicated circuits such as processors having circuit structures specially designed for performing specific processes such as ASICs (Application Specific Integrated Circuits).

[0223] Furthermore, the various processes described above may be performed by one of these various processors, or by a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs and a CPU and an FPGA, etc.). Furthermore, multiple processing units may be constituted by one processor. As an example of a plurality of processing units constituted by one processor, there is the following method: a processor that implements the functions of the entire system including multiple processing units by one IC (Integrated Circuit) chip, such as a System On Chip (SOC), is used.

[0224] In this way, the various processing units are configured as a hardware structure using one or more of the above-mentioned various processors.

[0225] Furthermore, more specifically, as the hardware configuration of these various processors, a circuit (Circuitry) in which circuit elements such as semiconductor elements are combined can be used.

[0226] Furthermore, in addition to the startup program of the photographic device, the technology of the present invention also relates to a storage medium (USB memory or DVD (Digital Versatile Disc)-ROM (Read Only Memory), etc.) that can be read by a computer that non-temporarily stores the startup program of the photographic device.

[0227] Explanation of symbols

[0228] 10-mobile terminal (photographic device), 11-photographic unit, 11A-photographic lens, 11B-image sensor, 11C-AD converter (A / D), 11D-image input controller, 11E-image signal processing circuit, 11F-timing signal generator (TG), 11G-driver, 12-touch screen display, 14-CPU, 16-operation switch, 17-working memory, 17A-image memory, 18-storage device, 19-media controller, 21-external I / F, 22-communication unit, 23-bus, 26-main control unit, 27-application (AP), 28-user area, 29-memory card, 31-display control unit, 32-image processing unit, 32A-extraction unit, 32B-synthesis unit, 50-display image, 52-determination area, 54-synthesized object image, 56-selected image, 60-determination box with indicator, 60, 60A-indicator, 60B-determination box, 60C-zoom in / out indication mark, 62-mark for color specification, 70-shutter button, 72-color palette, 72W-white, 73-setting button, 74-threshold setting tool, 74A-sliding bar, 74B-sliding block, 76-reverse button, 80-shirt, 82, 82A~82E-characters, B-background, T-friend, U-user.

Claims

1. An image processing device comprising: a display control unit that displays a determination frame that determines a portion of an area in a display image displayed on the display unit and a color designation mark for designating a reference color from among colors included in the image within the determination frame, superimposed on the display image; an extracting unit that extracts, from the image within the determination frame, an image corresponding to the reference color specified by the color specifying marker as a synthesis target image; and a synthesis unit that generates a synthesized image by synthesizing the synthesis target image and a selected image selected by a user, The extraction unit converts the grayscale of the entire image within the determination frame into numerical values, and extracts an image whose grayscale numerical value is within a reference color range including a grayscale numerical value corresponding to the reference color as the synthesis target image.

2. The image processing device according to claim 1, wherein: The color specification mark is displayed in the determination frame, and a relative positional relationship between the determination frame and the color specification mark is predetermined.

3. The image processing device according to claim 1, wherein: The image processing device further comprises: A first receiving unit receives a size change instruction to enlarge or reduce the determination frame. The display control unit performs control to change the size of the confirmation frame according to the size change instruction accepted by the first acceptance unit.

4. The image processing device according to claim 1, wherein: The image processing device further comprises: a second receiving unit for receiving a movement instruction for moving the position of the determination frame; The display control unit performs control to move the position of the confirmation frame according to the movement instruction accepted by the second acceptance unit.

5. The image processing device according to claim 1, wherein: The color of the portion of the display image where the color specifying mark overlaps is set as the reference color.

6. The image processing device according to claim 5, wherein: The display control unit further displays an indicator having a larger area than the color specifying mark and indicating the reference color.

7. The image processing device according to claim 1, wherein: The color designation mark has a shape surrounding a portion of the display image, The extraction unit uses the color of the display image surrounded by the color specifying mark as the reference color.

8. The image processing device according to claim 7, wherein: The display control unit further displays an indicator having an area larger than a region of the display image surrounded by the color specifying mark and indicating the reference color.

9. The image processing apparatus according to claim 1, wherein: The image processing device further comprises: The setting unit is used for the user to set the reference color range.

10. A photographing device comprising: The photography department photographs the subjects; A display unit that displays the image captured by the imaging unit; and The image processing device according to any one of claims 1 to 9.

11. The imaging device according to claim 10, wherein: The display image is displayed in real time preview on the display unit.

12. An image processing method, wherein a computer performs the following processing: displaying a determination frame that determines a portion of an area in a display image displayed on a display unit and a color specifying mark for specifying a reference color from among colors included in the image within the determination frame in a superimposed manner on the display image; extracting, from the images within the determination frame, an image corresponding to the reference color specified by the color specifying marker as a synthesis target image; and generating a composite image by combining the composite target image with a selected image selected by a user, The grayscale of the entire image within the determination frame is quantified, and an image whose grayscale value is within a reference color range including the grayscale value corresponding to the reference color is extracted as the synthesis target image.

13. A computer-readable storage medium storing an image processing program for causing a computer to perform the following processing: displaying a determination frame that determines a portion of an area in a display image displayed on a display unit and a color specifying mark for specifying a reference color from among colors included in the image within the determination frame in a superimposed manner on the display image; extracting, from the images within the determination frame, an image corresponding to the reference color specified by the color specifying marker as a synthesis target image; and generating a composite image by combining the composite target image with a selected image selected by a user, The grayscale of the entire image within the determination frame is quantified, and an image whose grayscale value is within a reference color range including the grayscale value corresponding to the reference color is extracted as the synthesis target image.

Images