Image processing device, image processing method, program, and recording medium

Abstract

To provide an image processing device, an image processing method, a program, and a recording medium that are capable of improving a presentation effect when two or more display images are displayed.SOLUTION: An image processing flow of an image processing device that includes a processor and displays two or more display images in a display mode containing a mode in which the number of display images within a screen changes, includes obtaining at least one piece of information regarding image acquisition conditions and image features for a plurality of images, selecting a plurality of candidate images from the plurality of images based on at least one piece of information obtained for the plurality of images, and selecting two or more candidate images satisfying a predetermined condition from among the plurality of candidate images as two or more display images, and setting a display method for the two or more display images.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] One embodiment of the present invention relates to an image processing device, an image processing method, a program, and a recording medium. [Background technology]

[0002] Techniques for displaying multiple images have already been developed, and the technique described in Patent Document 1 is one example.

[0003] Patent Document 1 discloses a technology in which, in a slideshow of images stored in a storage unit such as a memory card, all images are efficiently displayed as slides within a specified playback time, allowing a user to grasp the entire picture of the images stored in the storage unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-36737 A Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, images or videos (hereinafter, referred to as images, etc.) are frequently made available to acquaintances, the public, and other people through social networking services (SNS), etc. Under such circumstances, there is a demand for images, etc. that can be viewed by others in a fun way. In this regard, the slideshow generated by the technology described in Patent Document 1 is intended for the owner of the slideshow to grasp the entirety of the multiple images, and does not take into consideration that others may view the slideshow, making it insufficient as a presentation effect.

[0006] One embodiment of the present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an image processing device, an image processing method, a program, and a recording medium that are capable of enhancing the presentation effect when displaying two or more display images. [Means for solving the problem]

[0007] In order to achieve the above object, an image processing device according to one embodiment of the present invention is an image processing device that includes a processor and displays two or more display images in a display mode including a mode in which the number of display images on a screen changes, wherein the processor acquires at least one piece of information regarding image acquisition conditions and image features for the multiple images, selects multiple candidate images from the multiple images based on the at least one piece of information acquired for the multiple images, sets two or more candidate images from the multiple candidate images that satisfy predetermined conditions as two or more display images, and sets a display method for the two or more display images.

[0008] The plurality of images may also include photographed images.

[0009] The image acquisition conditions may also include the shooting conditions of the captured image.

[0010] Furthermore, the photographing conditions may be conditions set based on at least one of the photographing date and time, the photographing interval, the photographing event, the photographing mode, the photographing model, and the photographer.

[0011] The processor may also select a number of candidate images from the number of images based on image features obtained by analyzing the number of images.

[0012] The processor may also select two or more candidate images from the plurality of candidate images based on image features derived from an analysis of the plurality of candidate images.

[0013] The processor may also calculate a score based on image features for two or more display images, and set a display method for the two or more display images according to the score.

[0014] The display method may be at least one of the display position of the display image within the display area, the display angle of the display image with respect to the display area, and the display time during which the entire image of the display image is displayed.

[0015] The processor may also set as the display image at least one of a composite image obtained by combining an image area including the subject with a first area different from the image area, and a composite image obtained by combining two or more images.

[0016] The processor may also set, as the display image, an image with a frame area obtained by combining a frame area as the first area with the outside of the image area.

[0017] The processor may also select the type of frame region based on at least one of preset selection criteria and input information from a user.

[0018] The processor may also set a display method for two or more images with frame areas according to the type of the frame area.

[0019] The processor may also set the display position of an image with a frame area having a second outer size smaller than the first outer size to a position away from a reference position within the display area compared to the display position of an image with a frame area having a frame area of ​​the first outer size.

[0020] The processor may also set a display order for two or more images with frame areas according to the type of the frame area.

[0021] The processor may also set the number of display images to two or more depending on the number of images that can be arranged on the same printing surface when printing.

[0022] In addition, in order to solve the above-mentioned problems, an image processing method according to one embodiment of the present invention is an image processing method for displaying two or more display images in a display mode including a mode in which the number of display images on a screen changes, and is characterized in that the method includes a processor executing the following processes: a process of acquiring at least one piece of information regarding image acquisition conditions and image features for the multiple images; a process of selecting multiple candidate images from the multiple images based on the at least one piece of information acquired for the multiple images; and a process of setting two or more candidate images from the multiple candidate images that satisfy a predetermined condition as two or more display images, and setting a display method for the two or more display images.

[0023] According to one embodiment of the present invention, a program for causing a computer to execute each process included in the image processing method described above can be realized.

[0024] Furthermore, according to one embodiment of the present invention, it is possible to realize a computer-readable recording medium having a program recorded thereon for causing a computer to execute each process included in the above-mentioned image processing method. Effect of the Invention

[0025] According to one embodiment of the present invention, it is possible to provide an image processing device, an image processing method, a program, and a recording medium that are capable of improving the dramatic effect when two or more display images are displayed. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing a state in which a display image is displayed in image processing according to an embodiment of the present invention (part 1). [Diagram 2] FIG. 2 is a diagram showing how a display image is displayed in image processing according to an embodiment of the present invention (part 2). [Diagram 3] FIG. 11 is a diagram showing a state in which a display image is displayed in image processing according to an embodiment of the present invention (part 3). [Figure 4] FIG. 4 is a diagram showing how a display image is displayed in image processing according to an embodiment of the present invention (part 4). [Diagram 5] 1 is a diagram showing an example of the configuration of an image processing system including an image processing device according to an embodiment of the present invention. [Figure 6] FIG. 2 is an explanatory diagram of functions of an image processing device according to an embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing a selection screen for selecting multiple images in image processing according to an embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing a selection screen for selecting whether or not to print a displayed image in image processing according to an embodiment of the present invention. [Figure 9] FIG. 13 is a diagram showing a selection screen for selecting a layout pattern of a display image at the time of printing in image processing according to an embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing a selection screen for selecting a display image for setting a frame region in image processing according to an embodiment of the present invention. [Figure 11] FIG. 13 is a diagram showing a selection screen for selecting the type of frame area in image processing according to an embodiment of the present invention. [Figure 12] FIG. 2 is a diagram showing a procedure of an image processing flow according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] A specific embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example given to facilitate understanding of the present invention, and does not limit the present invention. Furthermore, the present invention may be modified or improved from the following embodiment without departing from the spirit of the present invention. Furthermore, the present invention includes its equivalents.

[0028] In this specification, the concept of "device" includes a single device that performs a specific function by itself, and also includes a combination of multiple devices that exist in a distributed and independent manner but cooperate (link) to perform a specific function.

[0029] In this specification, an "image" is an object to be displayed, and specifically, an image represented by digital image data (hereinafter, image data). Examples of image data include lossy compressed image data such as JPEG (Joint Photographic Experts Group) format, and lossless compressed image data such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format. The image data may include supplementary information such as image acquisition conditions, which will be described later.

[0030] "Image" may include, for example, images (captured images) captured by a user terminal with a camera function and photographic equipment such as a camera, as well as images (created images) other than captured images, such as CG (Computer Graphics) images drawn using a computer, etc.

[0031] The "image" may be a composite image obtained by combining an image area including a subject with a first area different from the image area. Specifically, the image may be an image with a frame area obtained by combining a frame area (corresponding to the first area) outside the image area, or a decorated image obtained by combining a decorative area (corresponding to the first area) such as a stamp inside or outside the image area. Furthermore, the "image" may be an image that does not have a frame area and is composed only of an image area. Here, the "frame area" refers to a blank area forming a rectangular or square frame formed along the outer edge of a rectangular or square image, and more specifically, to an area surrounding the outside of an image area that includes a subject such as a person. An image with a frame area may be obtained, for example, when photographed by a photographing device such as a camera having a function for setting a frame area, or may be obtained by synthesizing a frame area on the outside of an image area of ​​a photographed image using a computer or the like. Furthermore, the "image" may be a composite image obtained by combining two or more images, specifically a collage image in which a plurality of images are arranged on a template or the like.

[0032] In this specification, a "user" refers to a user who uses the image processing device of the present invention. Using the image processing device means using the functions of the image processing device, and includes directly operating the image processing device as well as using the functions of the image processing device through a device (e.g., a user terminal) that can communicate with the image processing device.

[0033] <<Overview of image processing>> A series of image processing steps (hereinafter simply referred to as "image processing") performed using an image processing device and an image processing method according to an embodiment of the present invention will be described with reference to Figures 1 to 4. Figures 1 to 4 show the transition of screens on a user terminal, and are arranged in chronological order from Figure 1 to Figure 4. Each of the four display images P1, P2, P3, and P4 shown in FIGS. 1 to 4 is an image with a frame area. A "user terminal" is a computer used by a user, specifically a smart device such as a smartphone, a tablet terminal, or a notebook PC (Personal Computer), etc. An application program for selecting multiple images for image processing (hereinafter, an image selection application) is installed in the user terminal.

[0034] In the image processing, a plurality of candidate images are selected from a plurality of images, and two or more candidate images are further selected from the plurality of candidate images as two or more display images, and a display method is set for the selected two or more display images. Then, in the set display method, the two or more display images are displayed in a display mode including a mode in which the number of display images on a screen changes. As a result, as shown in Figs. 1 to 4, a plurality of (four) display images P1, P2, P3, and P4 are displayed in such a way that the display images are added stepwise from the back side to the front side in the depth direction of the screen. More specifically, the four display images P1, P2, P3, and P4 are displayed in a display area A having, for example, a rectangular outer shape in the order of display image P1, display image P2, display image P3, and display image P4. The newly added display image is arranged so as to hide a part or the whole of the display image already arranged in the display area A. Alternatively, the newly added display image is arranged in a position that does not hide the display image already arranged in the display area A.

[0035] To explain "a mode in which the number of display images changes" in more detail, when display images are displayed, the display image in front may hide all or part of the display image in the back, or may not hide it. When even a part of the display image in the back is displayed, the number of display images on the screen changes. In other words, the number of display images increases by one in proportion to the addition of the display image in the front. This mode is "a mode in which the number of display images changes." On the other hand, if the display image on the front side hides all the display images on the back side, the number of display images on the screen does not change. In other words, even if a display image on the front side is added, the display image on the back side is not displayed, so the number of display images on the screen does not increase or decrease. This is a mode in which the number of display images does not change. In the image processing, two or more display images are displayed in a display mode including a mode in which the number of display images on the screen changes.

[0036] Next, the "display method" will be described in more detail. The display method refers to at least one of the display position of the display image within the display area, the display angle of the display image relative to the display area, and the display time during which the entire image of the display image is displayed. 1 to 4, the display angle of each of the four display images P1, P2, P3, and P4 is set randomly within a predetermined angle range (for example, −5° to +5°) with respect to the display area A. The display time of each of the four display images P1, P2, P3, and P4 is set to a fixed time. Meanwhile, the display position of each of the four display images P1, P2, P3, and P4 is set according to the type of frame area.

[0037] "Type of frame area" refers to the outer size and aspect ratio of the frame area, etc. "Outer size of frame area" refers to the area of ​​the inner area surrounded by the outer shape of the frame area, i.e., the area of ​​the entire area of ​​the image with frame area. "Aspect ratio of frame area" refers to the aspect ratio of the outer shape of the frame area, i.e., the aspect ratio of the image with frame area. 1 to 4, a display image having a frame area with a larger outer size is displayed at a position closer to a reference position C located at the center of the display area A. On the other hand, a display image having a frame area with a smaller outer size is displayed at a position farther from the reference position C.

[0038] More specifically, among the four display images P1, P2, P3, and P4, the two display images P1 and P4 have the same outer size of the frame region and are the smallest in outer size, as shown in Figures 1 and 4. The display image P2 has a larger outer size of the frame region than the two display images P1 and P4 and is smaller than the display image P3, as shown in Figure 2. The display image P3 has the largest outer size of the frame region among the four display images P1, P2, P3, and P4, as shown in Figure 3.

[0039] For this reason, assuming that the longer side of the screen of the user terminal is the up-down direction and the shorter side of the screen is the left-right direction, each of the two display images P1 and P4 is displayed at a position where its center position is a distance V1 in the up-down direction and a distance H1 in the left-right direction from the reference position C of the display area A, as shown in Figures 1 and 4. Display image P2 is displayed at a position where its center position is a distance V2 shorter than the distance V1 in the up-down direction and a distance H2 shorter than the distance H1 in the left-right direction from the reference position C of the display area A, as shown in Figure 2. Display image P3 is displayed at a position where its center position is a distance V3 shorter than the distance V2 in the up-down direction and a distance H3 shorter than the distance H2 in the left-right direction from the reference position C of the display area A, as shown in Figure 3. In addition, the display positions of the four display images may be set at any position around the reference position C as long as the above distances V1, V2, V3, H1, H2, and H3 are set, and in this example, they are set randomly.

[0040] Based on the display method thus set, four display images P1, P2, P3, and P4 are displayed within display area A of the screen while the number of display images is changed in stages.

[0041] It should be noted that the display method described above is merely one example, and other display methods may be used. For example, along with the display position, at least one of the display angle and the display time may be set based on the type of frame area. In the above example, a display image having a larger frame area in outer size is displayed at a position closer to the reference position C located at the center of the display area A, and a display image having a smaller frame area in outer size is displayed at a position farther from the reference position C. However, the above example is merely an example and is not limited to this. For example, a display image having a larger frame area in outer size may be displayed at a position farther from the reference position C located at the center of the display area A, and a display image having a smaller frame area in outer size may be displayed at a position closer to the reference position C.

[0042] After the display method is set, display data (moving image) in which two or more display images are displayed in the set display method is generated. The display data is made public to acquaintances or the public, for example, through SNS. Examples of the display data include a slide show or a GIF (Graphics Interchange Format) moving image.

[0043] As described above, in an image processing device according to an embodiment of the present invention, a plurality of candidate images are selected from a plurality of images, and two or more candidate images are further selected from the plurality of candidate images as two or more display images. That is, since the image selection is performed in two stages, a display image according to a predetermined theme or the like is appropriately selected. In this way, by using two or more display images according to a theme or the like, it is possible to enhance the presentation effect when two or more display images are displayed. In addition, in an image processing device according to an embodiment of the present invention, two or more display images are displayed in a display mode including a mode in which the number of display images on a screen changes. This allows not only the display content of the display image but also the changing state of the relationship between the two or more display images to be displayed, thereby enhancing the dramatic effect when two or more display images are displayed. In particular, in an image processing device according to one embodiment of the present invention, a display method for two or more display images is set, making it possible to further vary the relationship between the two or more display images, thereby further enhancing the presentation effect.

[0044] In addition, in an image processing device according to one embodiment of the present invention, when a photographed image is included among two or more display images, the presentation effect when displaying the photographed image can be enhanced.

[0045] In addition, in the image processing device according to an embodiment of the present invention, the display method is at least one of the display position, the display angle, and the display time, so that when two or more display images are displayed, the presentation effect of at least one of the display position, the display angle, and the display time can be improved.

[0046] <<Example of the configuration of an image processing system according to an embodiment of the present invention>> The configuration of an image processing system (hereinafter referred to as "image processing system S") including an image processing device according to one embodiment of the present invention will be described with reference to FIG. 5, the image processing system S is configured with an image processing device 10 and a plurality of user terminals 100. The image processing device 10 and the plurality of user terminals 100 are connected via a network N so as to be able to communicate with each other.

[0047] The image processing device 10 is configured by a server computer. The image processing device 10 may be configured by one server computer, or may be configured by multiple server computers distributed in parallel. The server computer configuring the image processing device 10 may be a server computer for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). When necessary information is input to the user terminal 100, the server computer performs various processes (calculations) based on the input information, and the calculation results are output on the user terminal 100 side. In other words, the functions of the server computer, which is the image processing device 10, can be used on the user terminal 100 side.

[0048] The computer constituting the image processing device 10 includes a processor 21, a memory 22, a communication interface 23, and a storage device 24, as shown in FIG.

[0049] The processor 21 is composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a TPU (Tensor Processing Unit), or an ASIC (Application Specific Integrated Circuit).

[0050] The memory 22 is composed of semiconductor memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). A program for executing image processing (hereinafter, referred to as an image processing program) is stored in the memory 22. The image processing program is a program for causing the processor 21 to execute each process of an image processing method described later. The image processing program may be obtained by reading it from a computer-readable recording medium, or may be obtained by downloading it via a communication network such as the Internet or an intranet.

[0051] The communication interface 23 may be, for example, a network interface card, a communication interface board, etc. The computer constituting the image processing device 10 can communicate with other devices connected to the Internet, a mobile communication line, etc. via the communication interface 23.

[0052] The storage device 24 is configured by, for example, a flash memory, a hard disc drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), a universal serial bus memory (USB memory), etc. The storage device 24 may be built into the computer main body constituting the image processing device 10, or may be attached to the computer main body in an external form.

[0053] The user terminal 100 includes a processor, a memory, a communication interface, a recording device, a camera, an input device, and an output device.

[0054] <<Functions of an image processing device according to one embodiment of the present invention>> Next, the configuration of the image processing device 10 according to an embodiment of the present invention will be described again from a functional perspective. As shown in FIG. 6, the image processing device 10 has an image acquisition unit 31, an information acquisition unit 32, a candidate image selection unit 33, a display image number setting unit 34, a display image setting unit 35, a frame area setting unit 36, a display order setting unit 37, a display method setting unit 38, a display data generation unit 39, an image display unit 40, and a print data generation unit 41. These functional units are realized by cooperation between hardware devices included in the computer constituting the image processing device 10 and a program (that is, software) installed in the computer. Each functional unit will be described below.

[0055] <Image acquisition section> The image acquisition unit 31 acquires a plurality of images transmitted from the user terminal 100. More specifically, the user selects a plurality of images to be subjected to image processing on the screen of the user terminal 100, and the plurality of selected images are transmitted from the user terminal 100, whereby the image acquisition unit 31 receives and acquires the plurality of images via the network N. In the example shown in FIG. 7, after the user starts an image selection application installed in the user terminal 100, the screen of the user terminal 100 transitions to a screen on which a plurality of images to be selected are displayed through a predetermined operation. When a plurality of images are selected through the displayed selection screen, the selected plurality of images are transmitted from the user terminal 100 to the image processing device 10, and the image acquisition unit 31 acquires the plurality of images.

[0056] The multiple images to be selected may be the above-mentioned captured images, created images, images with frame areas, images without frame areas, decorated images, collage images, etc., or may be a mixture of these. Note that the example shown in Fig. 7 shows a case where both images with frame areas and images without frame areas are mixed. The multiple images to be selected may be images stored in a storage device of the user terminal 100, or may be images stored in an external device with which communication is possible via the network N, such as an online storage or a database server.

[0057] <Information acquisition section> The information acquisition unit 32 acquires at least one piece of information regarding the image acquisition conditions and image features for the multiple images. The information acquisition unit 32 may acquire the above information for each of the multiple images, or may acquire the above information for at least one image of the multiple images.

[0058] (Image acquisition conditions) The "image acquisition conditions" include the shooting conditions of the captured image and the creation conditions of the created image such as a CG image.

[0059] The "photography conditions" are conditions set based on at least one of the following information: photography date and time, photography interval, photography event, photography mode, camera model, photographer, and photography location. "Photographing date and time information" includes information such as the date and time when the image was photographed. "Information about the photographing interval" is information about the time interval between the photographing dates and times of two photographed images that are related to each other. "Photographed event information" is information related to the photographed event, and includes information such as the event name. "Information on the shooting mode" includes information on the continuous shooting mode or the automatic shooting mode (interval mode). Both the "continuous shooting mode" and the "automatic shooting mode" are used when multiple shot images are continuously acquired within a predetermined time. The "continuous shooting mode" is used, for example, when multiple shot images are acquired while changing the angle of view and the position of the shooting equipment at the time of shooting, and the "automatic shooting mode" is used, for example, when multiple shot images are acquired without changing the angle of view and the position at the time of shooting. "Information about the imaging device model" includes, for example, the model name and manufacturer name of the imaging device such as a camera. "Photographer information" includes personal information of the photographer to the extent that disclosure is permitted. "Photographing location information" includes, for example, information such as the latitude, longitude, and altitude of the photographing location based on a GPS (Global Positioning System).

[0060] On the other hand, the "creation conditions" are conditions set based on at least one piece of information, such as the creation date and time and the creator. "Created date and time information" includes information on the date and time when the created image was created. "Creator information" includes personal information of the creator to the extent that disclosure is permitted.

[0061] (Image features) The "image features" are obtained by analyzing multiple images. The image features are information on the image quality of each region of the image, the gradation values ​​of the pixels contained in each region, and information on the subject estimated from these pieces of information. The subject information may include the type of subject, the state of the subject, the position of the subject in the image, and, if the subject is a person, the facial expression, etc. If the subject is an object, information on the object specifying an event such as an athletic meet or wedding, more specifically, information on a wedding dress, etc. may be included. If the subject is a landscape, information on the landscape specifying, for example, mountains, the sea, and the sky, etc. may be included. Note that even if the landscape (same place) is the same, if the shooting range (angle of view, etc.) is different, it may be considered as a different subject. Moreover, it is preferable that the image features can be quantified, vectorized, or tensorized. In this case, the image analysis result is the quantified, vectorized, or tensorized image features, that is, feature quantities. In this specification, the term "image characteristics" refers to characteristics in an image region, and is to be distinguished from characteristics in a frame region that surrounds the outside of the image region. The "frame region characteristics" will be described later.

[0062] <Candidate image selection section> The candidate image selection unit 33 selects multiple candidate images from the multiple images based on at least one of the above-mentioned information acquired for the multiple images. The candidate images may include an image (e.g., an image with a frame area and a decorated image, etc.) obtained by combining an image area including a subject with a first area (e.g., a frame area and a decorated area, etc.) different from the image area, and an image (e.g., a collage image, etc.) obtained by combining two or more images (image areas). Two selection methods are given below as examples.

[0063] The first selection method is a selection method based on the image acquisition conditions. For example, based on information regarding the continuous shooting mode or automatic shooting mode among the shooting conditions, multiple images taken continuously in the continuous shooting mode or automatic shooting mode are selected as multiple candidate images. In this manner, the image processing device 10 can select multiple candidate images from multiple captured images based on conditions set based on at least one of the shooting conditions of the captured image, more specifically, the shooting date and time, the shooting interval, the shooting event, the shooting mode, the camera model, and the photographer.

[0064] The second selection method is a selection method based on image features, in which the candidate image selection unit 33 selects multiple candidate images from multiple images based on image features obtained by analyzing the multiple images. As an example, the candidate image selection unit 33 may compare the feature amounts of the subjects contained in multiple images to calculate the similarity of the subjects, and may select a predetermined number of images in descending order of the similarity of the subjects as candidate images, or, conversely, select a predetermined number of images in descending order of the similarity of the subjects as candidate images. As another example, the candidate image selection unit 33 calculates the scores of multiple images based on the feature amount of the subject, and sets a predetermined number of images in descending order of the score as candidate images. For example, if the subject is a person, the score of multiple images may be calculated based on the feature amount such as the degree of smile or the number of people, and a predetermined number of images in descending order of the score may be set as candidate images. In this manner, the image processing device 10 can select a plurality of candidate images from a plurality of images based on the image features obtained by analyzing the plurality of images.

[0065] <Display image count setting section> The display image number setting unit 34 sets the number of display images to be displayed within the display area of ​​the screen. Below, a method for setting the number of display images will be described using four setting methods as examples.

[0066] In the first setting method, the total number of the selected candidate images is set as the number of images to be displayed. The second setting method is to set an upper limit on the number of images to be displayed, and if the number of multiple candidate images is equal to or greater than the upper limit, set the upper limit as the number of images to be displayed, and if the number of multiple candidate images is less than the upper limit, set that number as the number of images to be displayed. As a third setting method, the number of images input by the user is set as the number of images to be displayed. As a fourth setting method, when a user prints the display images, the number of display images is set to two or more according to the number of images that can be arranged on the same print surface at the time of printing. In this way, when two or more display images are displayed, the number of display images corresponding to the number of images at the time of printing can be displayed.

[0067] To explain the above fourth setting method in more detail, the screen of the user terminal 100 transitions to the screen shown in FIG. 8, and a screen is displayed requesting the user to select whether or not to print two or more displayed images later. If the user selects not to print the displayed images, the number of displayed images is set by any one of the first to third setting methods described above. On the other hand, if the user selects to print the displayed images, the number of displayed images is set by the fourth setting method described above. More specifically, the screen of the user terminal 100 transitions to the screen shown in Figure 9, which displays multiple candidate layout patterns for two or more display images to be arranged on the same print surface, and displays a screen requesting the user to select one of the layout patterns from among the candidates.

[0068] The "layout pattern" may be a pattern in which multiple display images of the same outer size are arranged on the same print surface, or a pattern in which multiple display images of different outer sizes are arranged on the same print surface. The number of display images arranged on the same print surface may be, for example, the square value of n (n is a natural number of 2 or more), or may be set to a number based on the square value of n.

[0069] In the above example, when the user selects to print the display images, the number of display images is set by the fourth setting method, but this is not limited thereto, and for example, even when the user selects to print the display images, the number of display images may be set by any one of the first to third setting methods. That is, the number of display images included in the display data (moving image) may be set to a value different from the number of images arranged on the same print surface when printed.

[0070] <Display image settings section> The display image setting unit 35 sets, from among the plurality of candidate images, two or more candidate images that satisfy a predetermined condition as two or more display images. The "predetermined condition" is a condition set based on the number of display images set by display image number setting unit 34 and at least one piece of information on the image acquisition conditions and image features. In other words, display image setting unit 35 selects two or more candidate images according to the number of display images from among the multiple candidate images based on the number of display images and at least one piece of information on the image acquisition conditions and image features, and sets them as two or more display images.

[0071] More specifically, the display image setting unit 35 may select two or more candidate images from among the multiple candidate images based on image features obtained by analyzing the multiple candidate images, and set them as two or more display images. As an example, the display image setting unit 35 may compare the feature amounts of the subjects included in a plurality of candidate images to calculate the similarity of the subjects, and set as display images the number of images set by the display image number setting unit 34 in descending order of the candidate images with the highest similarity of the subjects. Alternatively, conversely, the display image setting unit 34 may set as display images the number of images set by the display image number setting unit 34 in ascending order of the candidate images with the lowest similarity of the subjects. As another example, the display image setting unit 35 may calculate scores for multiple candidate images based on the features of the subject, and set the number of images set by the display image number setting unit 34 as display images in order of the candidate images with the highest scores. As described above, in the image processing device 10, two or more candidate images selected based on the analysis results of a plurality of candidate images can be displayed as two or more display images.

[0072] The display image setting unit 35 may set as the display image at least one image (hereinafter, a composite image) among an image obtained by synthesizing an image area including a subject with a first area (e.g., a frame area and a decorated area) different from the image area, and an image obtained by synthesizing two or more images (image areas) (e.g., a collage image). Therefore, in the image processing device 10, when the display image is a composite image, it is possible to enhance the presentation effect when the composite image is displayed. Also, when the display image is an image with a frame area, it is possible to enhance the presentation effect when the image with a frame area is displayed.

[0073] <Frame area setting section> The frame region setting unit 36 ​​sets a frame region for the two or more set display images. More specifically, the screen of the user terminal 100 transitions to the screen shown in FIG. 10, and a screen requesting the user to set frame regions for the two or more display images set above is displayed.

[0074] In the example shown in FIG. 10, two of the four display images are display images for which frame areas have already been set. Therefore, the screen shown in FIG. 10 is a screen requesting that frame areas be set for the remaining two display images. When a display image for which a frame area has not been set is selected, the screen of the user terminal 100 transitions to the screen shown in FIG. 11, and multiple (four in FIG. 11) types of frame area candidates are displayed. Note that the multiple types of frame area candidates also include candidates with no frame area. When the user selects one of the types of frame area, the type of frame area is selected based on information input by the user, and an image having the selected type of frame area (image with frame area) is set as the display image. In the example shown in FIG. 11, when a candidate without a frame area is selected, the image without a frame area is set as the display image.

[0075] Also, in the example shown in FIG. 11, for images in which a frame area has already been set, no request is made to set a frame area, but this is not limited thereto. For example, even for images in which a frame area has already been set, a request may be made to select whether or not to change the type of frame area, and the type of frame area may be selected (changed) based on information input by the user. 11, the type of frame area is selected based on the input information from the user, but the present invention is not limited to this, and the type of frame area may be selected based on a preset selection criterion. For example, the frame area may be selected based on a selection criterion related to the aspect ratio, and specifically, a frame area having an aspect ratio closer to the aspect ratio of the target display image may be selected.

[0076] As described above, the frame area setting unit 36 ​​selects the type of frame area for two or more display images based on at least one of the preset selection criteria and the input information from the user. This makes it possible to set an image with a frame area according to the selected type of frame area as the display image.

[0077] <Display order setting section> The display order setting unit 37 sets the display order of two or more display images based on the image acquisition conditions, the characteristics of the images (image regions), and the characteristics of the frame regions. The "characteristics of a frame area" include the presence or absence of a frame area in a displayed image, the type of the frame area, the presence or absence of text in the frame area, and the presence or absence of decoration (pictures, patterns, etc.) in the frame area. Below, a method for setting the display order of the displayed images will be described using three setting methods as examples.

[0078] The first setting method is a setting method based on the image acquisition conditions, and the display order setting unit 37 may set the display order of two or more display images in ascending or descending order of the shooting date and time, for example, based on information on the shooting date and time.

[0079] The second setting method is based on the characteristics of the images, and the display order setting unit 37 may set the display order of two or more display images in order of the number of people in the image, for example, in the case where the subject includes people, based on the number of people in the image. Alternatively, the display order may be set so that display images including the same subject are consecutive, or conversely, the display order may be set so that display images including the same subject are not consecutive.

[0080] The third setting method is a setting method based on the characteristics of the frame area, and the display order setting unit 37 may set the display order of two or more frame area-attached images based on, for example, the type of the frame area. For example, the display order setting unit 37 may set the display order so that display images including the same type of frame area are consecutive, or conversely, may set the display order so that display images including the same type of frame area are not consecutive. As described above, the image processing device 10 can enhance the presentation effect when displaying two or more display images by displaying the display images in a display order that corresponds to the characteristics of the frame area, more specifically, the type of frame area.

[0081] <Display method setting section> The display method setting unit 38 sets a display method for two or more display images. Note that the display method setting unit 38 may set a display method for each of the two or more display images, or may obtain a display method for at least one of the two or more display images.

[0082] More specifically, the display method setting unit 38 sets at least one of the display methods of two or more display images, i.e., the display position, the display angle, and the display time, based on at least one of the characteristics of the image (image area) and the characteristics of the frame area. The display method setting unit 38 may, for example, calculate a score based on the image features for two or more display images, and set a display method for the two or more display images according to the score. This allows a performance according to the score when displaying two or more display images. Furthermore, as described above, the display method setting unit 38 may set the display method of two or more frame area-added images according to the type of frame area. More specifically, the display method setting unit 38 may set the display position of the frame area-added image having a second outer size smaller than the first outer size to a position away from the reference position within the display area, as compared to the display position of the frame area-added image having a frame area of ​​a first outer size, as shown in Figures 1 to 4. This makes it possible to perform a presentation according to the type of frame area, specifically, the outer size of the frame area-added image, when displaying two or more display images.

[0083] <Display data generation unit> The display data generating unit 39 generates display data in which two or more display images are displayed in a set display method. The generated display data may be stored in a storage device of the user terminal 100, or in an external device capable of communicating with the image processing device 10 via the network N, such as an online storage or a database server.

[0084] <Image display section> The image display unit 40 transmits display data to the user terminal 100 and displays two or more display images on the screen of the user terminal 100 in a display mode, including a mode in which the number of display images on the screen changes, in the display method set above.

[0085] <Print data generation section> The print data generating unit 41 generates print data in which two or more display images are arranged on the same print surface in the selected arrangement pattern. A code area such as a QR code (registered trademark) is provided on the printed surface, and the code area contains coded information indicating the storage destination of the display data. Therefore, by reading the code area, it is possible to identify the storage destination of the display data from the information contained in the code area and access the display data stored in the identified storage destination. Then, by accessing the display data, two or more display images indicated by the display data can be displayed in the set display method.

[0086] <<Image processing flow according to one embodiment of the present invention>> Next, an image processing flow, which is a data processing flow using the above-mentioned image processing device 10, will be described. The image processing flow employs the image processing method of the present invention, and proceeds along the flow shown in Fig. 12. In other words, each step in the flow shown in Fig. 12 corresponds to each element constituting the image processing method of the present invention. It should be noted that the flow shown in FIG. 12 is merely an example, and unnecessary steps may be deleted and new steps may be added without departing from the spirit of the present invention.

[0087] First, when a user starts an image selection application installed in the user terminal 100, a signal generated in conjunction with the start of the application is transmitted to a server computer constituting the image processing device 10. The image processing flow is started upon the transmission of the signal.

[0088] When a plurality of target images are transmitted from the user terminal 100, the processor 21 acquires the plurality of target images (S001). Next, the processor 21 acquires at least one piece of information regarding the image acquisition conditions and the image features for the plurality of images (S002). Next, the processor 21 selects a plurality of candidate images from among the plurality of images based on at least one of the above-mentioned pieces of information acquired for the plurality of images (S003).

[0089] Next, processor 21 requests the user to select whether or not to print two or more display images afterwards (S004), and if it receives input information from the user indicating that two or more display images will not be printed, it skips step S005 and proceeds to step S006. On the other hand, when the processor 21 receives input information from the user indicating that two or more display images are to be printed, the processor 21 requests the user to select one of a plurality of candidates for the layout pattern of the two or more display images to be arranged on the same print surface. After that, the processor 21 sets the layout pattern of the display images on the print surface based on the selection information from the user (S005).

[0090] Next, the processor 21 sets the number of display images to be displayed in the display area of ​​the screen of the user terminal 100 (S006). More specifically, when the processor 21 receives input information from the user in step S004 indicating that two or more display images will not be printed, the processor 21 sets an upper limit on the number of display images, and when the number of multiple candidate images is equal to or greater than the upper limit, the processor 21 sets the upper limit as the number of display images, and when the number of multiple candidate images is equal to or less than the upper limit, the processor 21 sets the upper limit as the number of display images. On the other hand, when the processor 21 receives input information from the user in step S004 indicating that two or more display images will be printed, the processor 21 sets the number of images based on the arrangement pattern set in step S005 as the number of display images.

[0091] Next, the processor 21 sets two or more candidate images that satisfy a predetermined condition from among the plurality of candidate images as two or more display images (S007). Next, processor 21 determines whether or not there is a display image without a frame area among the two or more display images (S008), and if there is no display image without a frame area, that is, if all of the two or more display images are images with frame areas, skips step S009 and proceeds to step S010.

[0092] On the other hand, when a display image without a frame area exists, the processor 21 sets a frame area for the display image without a frame area (S009). More specifically, the processor 21 sets a frame area for the image without a frame area based on information input by the user regarding the type of the frame area. It should be noted that the processor 21 may set an image without a frame area as the display image based on information input by the user. In addition, the processor 21 may request the user to select whether or not to change the type of frame area even for an image with a frame area in which a frame area has already been set, and change the type of frame area based on input information from the user.

[0093] Next, the processor 21 sets the display order of the two or more display images (S011). After that, the processor 21 sets the display method of the two or more display images (S012).

[0094] Next, the processor 21 generates display data in which two or more display images are displayed in the set display method (S012). The processor 21 transmits the display data to the user terminal 100, and displays the two or more display images on the screen of the user terminal 100 in a display mode in the set display method, including a mode in which the number of display images on the screen changes. Thereafter, the processor 21 generates print data in which two or more display images are arranged on the same print surface in the arrangement pattern set in step S005 (S013). When the series of processes described above is completed, the image processing flow ends.

[0095] <<Other embodiments>> In the above image processing system, the image processing device 10 is configured as a server computer, but this is not limiting. For example, the image processing device of the present invention may be configured as a user terminal.

[0096] <Processor configuration> The image processing device of the present invention and the processor included in the image processing device include various processors, including, for example, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units. The various processors also include programmable logic devices (PLDs) such as field programmable gate arrays (FPGAs) whose circuit configurations can be changed after manufacture. Furthermore, various processors include dedicated electric circuits such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed exclusively for performing specific processing.

[0097] In addition, the image processing device of the present invention and one of the processing units possessed by the image processing device may be configured by one of the various processors described above, or may be configured by a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc. Furthermore, the multiple functional units of the image processing device of the present invention may be configured by one of various processors, or two or more of the multiple functional units may be integrated into one processor. Also, as in the above-described embodiment, one or more CPUs and software may be combined to form one processor, and this processor may function as multiple functional units.

[0098] Also, for example, a processor may be used that realizes the image processing device of the present invention and the functions of the entire system including a plurality of functional units in the image processing device on a single IC (Integrated Circuit) chip, as typified by SoC (System on Chip), etc. Also, the hardware configuration of the above-mentioned various processors may be an electric circuit (Circuitry) that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0099] 10 Image processing device 21 Processors 22 Memory 23 Communication Interface 24 Storage device 31 Image acquisition unit 32 Information Acquisition Department 33 Candidate Image Selection Section 34 Display image number setting section 35 Display image setting section 36 Frame area setting section 37 Display order setting section 38 Display method setting section 39 Display data generation unit 40 Image display unit 41 Print data generation unit 100 user terminals A Display area C Reference position H1,H2,H3,V1,V2,V3, distance N Network P1,P2,P3,P4 Display image S Image Processing System

Claims

1. An image processing device including a processor, the image processing device displaying two or more display images in a display mode including a mode in which the number of display images on a screen is changed, The processor, acquiring information on at least one of image acquisition conditions and image characteristics for a plurality of images; selecting a plurality of candidate images from among the plurality of images based on the at least one information obtained for the plurality of images; The image processing device sets two or more candidate images among the plurality of candidate images that satisfy a predetermined condition as the two or more display images, and sets a display method for the two or more display images.

2. The image processing device according to claim 1 , wherein the plurality of images includes a photographed image.

3. The image processing device according to claim 2 , wherein the image acquisition conditions include shooting conditions of the captured image.

4. The image processing device according to claim 3 , wherein the photographing conditions are set based on at least one of a photographing date and time, a photographing interval, a photographing event, a photographing mode, a camera model, and a photographer.

5. The image processing device according to claim 1 , wherein the processor selects the candidate images from among the images based on image features obtained by analyzing the images.

6. The image processing device according to claim 1 , wherein the processor selects the two or more candidate images from the plurality of candidate images based on image features obtained by analyzing the plurality of candidate images.

7. The processor, calculating a score based on image features for the two or more displayed images; The image processing device according to claim 1 , further comprising: setting the display method for the two or more display images in accordance with the score.

8. The image processing device according to claim 1 , wherein the display method is at least one of a display position of the display image within a display area, a display angle of the display image with respect to the display area, and a display time during which the entire image of the display image is displayed.

9. The image processing device according to claim 1 , wherein the processor sets as the display image at least one of a composite image obtained by combining an image area including a subject with a first area different from the image area, and a composite image obtained by combining two or more images.

10. The image processing device according to claim 9 , wherein the processor sets, as the display image, an image with a frame area obtained by combining a frame area as the first area with an outside of the image area.

11. The image processing device according to claim 10 , wherein the processor selects the type of the frame region based on at least one of a preset selection criterion and information input by a user.

12. The image processing device according to claim 10 , wherein the processor sets the display method for two or more of the frame area-added images in accordance with a type of the frame area.

13. The image processing device according to claim 10, wherein the processor sets the display position of the framed image having a second outer size smaller than the first outer size to a position away from a reference position within the display area compared to the display position of the framed image having the frame area of ​​the first outer size.

14. The image processing device according to claim 10 , wherein the processor sets a display order of the two or more frame area-added images according to a type of the frame area.

15. The image processing device according to claim 1 , wherein the processor sets the number of the two or more display images in accordance with the number of images that can be arranged on the same printing surface when printing.

16. An image processing method for displaying two or more display images in a display mode including a mode in which the number of display images on a screen is changed, comprising: A process of acquiring information on at least one of image acquisition conditions and image features for a plurality of images; selecting a plurality of candidate images from among the plurality of images based on the at least one information obtained for the plurality of images; An image processing method executed by a processor, the image processing method including a process of selecting two or more candidate images among the plurality of candidate images that satisfy a predetermined condition as the two or more display images, and setting a display method for the two or more display images.

17. A program for causing a computer to execute each step included in the image processing method according to claim 16.

18. A computer-readable recording medium, 17. A recording medium having recorded thereon a program for causing a computer to execute each process included in the image processing method according to claim 16.

Images