Image processing device, image processing method, program, and print production system

Abstract

To detect only a subject out of a photographic image without installing a screen behind the subject.SOLUTION: An image processing device comprises: an analysis unit which extracts a person out of a photographic image obtained by photographing a user, on the basis of a result of analyzing whether each pixel in the photographic image belongs to a person; a display processing unit which displays on a display device the extracted person; and an image processing unit which generates a synthetic image by receiving selection of the person displayed on the display device and synthesizing the image of the selected person and a template image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, a program, and a print production system. [Background technology]

[0002] Chromakey compositing, which removes a specific color from a photographed image to make it transparent and then composites another image onto it, is used in various situations. For example, a service is known in which a person is photographed against a green or blue screen, the green or blue parts of the photographed image are made transparent, another image is composited, and a print of the composite image is provided. When using chromakey compositing, it is necessary to place a green or blue screen behind the person, which can be difficult to install depending on the shooting location. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6767541 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in consideration of the above-described conventional situation, and an object of the present invention is to provide an image processing device, an image processing method, a program, and a print production system that are capable of detecting only a subject from a captured image without placing a screen behind the subject. [Means for solving the problem]

[0005] The image processing device of the present invention includes an analysis unit that extracts a person from a captured image of a user based on an analysis result of whether each pixel in the image belongs to a person, a display processing unit that displays the extracted person on a display device, and an image processing unit that accepts a selection of a person to be displayed on the display device and combines an image of the selected person with a template image to generate a composite image.

[0006] The image processing method of the present invention extracts a person from a captured image of a user based on an analysis result of whether each pixel in the image belongs to a person, displays the extracted person on a display device, accepts a selection of the person displayed on the display device, and combines the image of the selected person with a template image to generate a composite image.

[0007] The program of the present invention causes a computer to execute the steps of extracting a person from a captured image of a user based on an analysis result of whether each pixel in the image belongs to a person, displaying the extracted person on a display device, and accepting a selection of the person displayed on the display device, and combining the image of the selected person with a template image to generate a composite image.

[0008] The print production system of the present invention includes an imaging device that photographs a user and generates a photographed image, an analysis unit that extracts a person from the photographed image based on the analysis results of whether each pixel in the photographed image belongs to a person, a display device that displays the extracted person, an image processing unit that accepts a selection of a person displayed on the display device and generates a composite image by combining an image of the selected person with a template image, and a printer that prints and outputs the composite image on paper. Effect of the Invention

[0009] According to the present invention, it is possible to detect only the subject from the captured image without placing a screen behind the subject. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a print producing system according to an embodiment of the present invention. [Diagram 2] FIG. 1A is a diagram showing an example of a captured image, and FIG. 1B is a diagram showing an example of person extraction. [Diagram 3] FIG. 2 is a hardware configuration diagram of the print production system. [Figure 4] FIG. 2 is a functional block diagram of the image processing device. [Diagram 5] 1 is a flowchart illustrating a method for producing a print. [Figure 6] FIG. 13A is a diagram showing an example of a subject selection screen, and FIG. 13B is a diagram showing an example of a subject confirmation screen. [Figure 7] 13A and 13B are diagrams showing an example of a subject selection screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] 1 is a schematic diagram of a print production system according to an embodiment of the present invention. The print production system includes a print production device 1 and a server device 2. The print production device 1 and the server device 2 are connected to each other so as to be able to communicate with each other via a network such as the Internet.

[0013] The print producing device 1 includes a housing H and legs L that support the housing H. Inside the housing H, there are provided an arithmetic and control device 10 (see FIG. 3), an image capturing device 11 (image capturing unit), a touch panel 12, a printer 13 (see FIG. 3), a communication unit 14 (see FIG. 3), and the like.

[0014] The print producing device 1 captures an image of a user (subject P), detects the subject P from the captured image, and generates a composite image by combining the detected subject image with a template image. The print producing device 1 prints the composite image on paper and outputs a print 3.

[0015] Methods such as semantic segmentation are used to detect subjects from captured images. For example, semantic segmentation classifies all pixels in a captured image as belonging to a person or background (non-person). Pixels classified as belonging to the background are made transparent to extract the person area. Because it is possible to classify people from the background, there is no need to install a green or blue screen behind the device as in chromakey compositing. Therefore, it is easy to install in an open space indoors or outdoors.

[0016] When the print producing device 1 is installed in an open space, when a subject P is photographed, surrounding people (non-subject D) are also captured in the captured image. Therefore, for example, when a person is detected from the captured image shown in Fig. 2(a), not only the subject P but also the non-subject D is detected as shown in Fig. 2(b). As described later, the print producing device 1 according to this embodiment erases the non-subject D and generates a composite image using only the image of the subject P.

[0017] 3 is a diagram showing the hardware configuration of the print producing apparatus 1. The print producing apparatus 1 includes an arithmetic and control device 10, a photographing device 11, a touch panel 12, a printer 13, and a communication unit 14.

[0018] The photographing device 11 is, for example, a digital camera, and outputs an image signal to the arithmetic and control device 10.

[0019] The touch panel 12 displays operation buttons, and when an operation button is pressed, it accepts an operation instruction corresponding to the operation button. The touch panel 12 can also perform a live view in which image signals output from the image capture device 11 are sequentially displayed. The touch panel 12 may be of any type, including a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type, and is not particularly limited thereto.

[0020] Instead of the touch panel 12, a display without a touch function and input means such as a keyboard or a mouse may be provided.

[0021] The printing method of the printer 13 is not particularly limited, and an inkjet method, a dye-sublimation type thermal transfer method, a melting type thermal transfer method, or the like can be used.

[0022] The communication unit 14 communicates with the server device 2 to transmit and receive data.

[0023] The arithmetic and control device 10 is a computer that controls each part of the print producing device 1 and transfers data. As shown in Fig. 4, the CPU of the arithmetic and control device 10 executes an image processing program stored in the storage unit 100, thereby realizing the functions of an analysis unit 101, an imaging control unit 103, a display processing unit 104, an image processing unit 105, a print control unit 106, an address information acquisition unit 107, and an image transmission unit 108, and the arithmetic and control device 10 operates as an image processing device.

[0024] The storage unit 100 is a hard disk or a semiconductor memory, and stores template images for synthesis in advance. The template images are characters, illustrations related to the installation location of the print producing apparatus 1, and the like.

[0025] 5 is a flowchart for explaining the print producing method according to the present embodiment. The processing of each part of the arithmetic and control device 10 will be explained with reference to this flowchart.

[0026] When a user operates the touch panel 12 to give an instruction to shoot, the shooting control unit 103 controls the shooting device 11 to perform shooting processing (step S1). For example, in the shooting processing, after shooting preparation operations such as AF (autofocus) processing, AE (auto exposure) processing, and AWB (auto white balance) processing, an image captured by the shooting device 11 is written to the storage unit 10 as an image file.

[0027] A live view mode may be executed before the photographing process. In the live view mode, the display processing unit 104 sequentially displays image signals acquired from the photographing device 11 on the touch panel 12. The user adjusts his / her standing position and posture while viewing the live view.

[0028] The analysis unit 101 analyzes the image based on the image signal and extracts a person in the image (step S2). For example, all pixels in the image are classified as belonging to a person or background by semantic segmentation. Then, pixels classified as belonging to the background are made transparent, and the person area is extracted.

[0029] The image analysis may be performed within the arithmetic and control device 10, or the image may be transmitted to an external server, the analysis may be performed on the external server, and the analysis results may be obtained from the external server.

[0030] The display processing unit 104 displays a subject selection screen on the touch panel 12 for accepting selection of a subject from the extracted people (step S3), and accepts the selection of a subject from the user (step S4). For example, a subject selection screen including an image in which everything other than the person is made transparent (erased) as shown in Fig. 6(a) is displayed, and the user selects himself (themselves).

[0031] On the subject selection screen, pixels that are determined to be people may be displayed semi-transparently in a specific color such as gray, and the selected people may be changed to another color such as red. Once the selection is complete, the user presses the Done button B1.

[0032] When the image processing unit 105 accepts a person selection via the touch panel 12, it makes the people not selected transparent and erases them, and displays a subject confirmation screen as shown in Fig. 6(b) on the touch panel 12. The user checks that they are displayed and that no other people are included, and presses the approval button B2. To reselect a subject, the user presses the reselection button B3 to return to the subject selection screen shown in Fig. 6(a).

[0033] After the user's approval, the image processor 105 generates a composite image by combining the selected subject with the template image (step S5).

[0034] The print control unit 105 transfers the composite image to the printer 13, controls the printer 13 to print the composite image on paper, and outputs a printout (step S6).

[0035] The print producing device 1 displays an email address input screen on the touch panel 12. The user operates the email address input screen to input an email address (step S7). The address information acquiring unit 107 acquires the email address information via the touch panel 12 and stores it in the storage unit 100. The email address may be inputted not only after the print is outputted, but also after the image is captured and before the print is outputted, or before the image is captured.

[0036] The image transmission unit 108 transmits the composite image and the input email address information to the server device 2. The server device 2 transmits the composite image as an email attachment, so that the user can obtain the composite image data.

[0037] Thus, according to this embodiment, by extracting people from the captured image and having the user select themselves from among the extracted people, it is possible to eliminate non-subjects and generate a composite image using only the image of the subject, thereby making it possible to create a print that will interest the user.

[0038] In the above embodiment, an example was described in which a composite image is attached to an email and sent using an input email address, but a card on which is printed the URL of a website from which the composite image data can be downloaded (and a two-dimensional code indicating the URL) may also be printed out from the print production device 1.

[0039] The subject selection screen displayed by the display processing unit on the touch panel 12 may be a list of face images obtained by cutting out rectangular areas including the faces of people from the captured images, as shown in Fig. 7(a). The color of the frame of the selected person or the color of the entire rectangular area may be changed to make it clear whether or not the person has been selected.

[0040] In FIG. 7(a), the facial images of each person are displayed in a list of the same size, but as shown in FIG. 7(b), the facial images of each person may be displayed in a list with the size of each person's face changed in proportion to the size of the face in the captured image.

[0041] The present invention is not limited to the above-described embodiment, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0042] 1 Printing equipment 2. Server equipment 3 Printed materials 10 Arithmetic and control device 11 Imaging Device 12 Touch Panel 13 Printers 100 Storage section 101 Analysis Department 103 Shooting control section 104 Display processing unit 105 Image processing section 106 Print control unit 107 Address information acquisition unit 108 Image transmission unit

Claims

1. an analysis unit that extracts a person from a captured image of a user based on an analysis result of whether each pixel in the captured image belongs to a person; a display processing unit that displays the extracted person on a display device; an image processing unit that receives a selection of a person displayed on the display device and generates a composite image by combining an image of the selected person with a template image; Equipped with The display processing unit displays on the display device a list screen of facial images of people extracted by the analysis unit, and the size of each facial image of each person is changed in proportion to the size of the face in the captured image.

2. The image processing device according to claim 1 , wherein the display processing unit changes a color of a selected person from among the people displayed on the display device.

3. extracting a person from a photographed image of a user based on an analysis result as to whether each pixel in the photographed image belongs to a person; displaying a list screen of the extracted face images of the persons on a display device, and changing the size of the face image of each person on the list screen in proportion to the size of the face in the photographed image; The image processing method includes accepting a selection of a person displayed on the display device, and generating a composite image by combining an image of the selected person with a template image.

4. extracting a person from a captured image of a user based on an analysis result of whether each pixel in the captured image belongs to a person; displaying a list screen of the extracted facial images of the persons on a display device, and changing the size of the facial image of each person on the list screen in proportion to the size of the face in the photographed image; receiving a selection of a person displayed on the display device, and generating a composite image by combining an image of the selected person with a template image; A program that causes a computer to execute the following.

5. an image capturing device that captures an image of a user and generates a captured image; an analysis unit that extracts a person from the photographed image based on an analysis result of whether each pixel in the photographed image belongs to a person; a display device that displays the extracted person; an image processing unit that receives a selection of a person displayed on the display device and generates a composite image by combining an image of the selected person with a template image; a printer for printing and outputting the composite image on paper; Equipped with The display device displays a list screen of facial images of people extracted by the analysis unit, and the size of each facial image on the list screen is changed in proportion to the size of the face in the captured image.

Images