Image processing apparatus, image processing method, and recording medium

Abstract

The invention provides an image processing apparatus, an image processing method, and a recording medium, capable of improving a presentation effect when two or more display images are displayed. An image processing device according to one embodiment of the present invention is provided with a processor, and displays two or more display images in a display form including a change in the number of display images within a screen, the processor acquires at least one piece of information of an image acquisition condition and an image feature for the plurality of images, selects a plurality of candidate images from the plurality of images according to the at least one piece of information acquired for the plurality of images, and performs processing of selecting the plurality of candidate images from the plurality of candidate images, two or more candidate images satisfying a predetermined condition are set as two or more display images, and a display method of the two or more display images is set.

Description

Technical Field

[0001] An embodiment of the present invention relates to an image processing device, an image processing method, and a recording medium. Background Art

[0002] The technology for displaying a plurality of images has been developed, and the technology described in Patent Document 1 is cited as an example.

[0003] Patent Document 1 discloses a technology for efficiently displaying all images in a slide show of images stored in a storage unit such as a memory card within a predetermined playback time so that a user can understand all the contents of the images stored in the storage unit.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2007-36737

[0005] In recent years, images and moving images (hereinafter referred to as images, etc.) are frequently disclosed to acquaintances or the public, etc., through SNS (Social Networking Service) etc. Under such circumstances, images, etc. that can be browsed with pleasure by others are required.

[0006] In this regard, the slide show generated by the technology described in Patent Document 1 is a slide show in which the user himself / herself grasps all the contents of the plurality of images, and does not take into account the browsing of others, and is therefore insufficient as a presentation effect. Summary of the invention

[0007] An embodiment of the present invention is completed in view of the above situation, and its purpose is to provide an image processing device, an image processing method, a program and a recording medium that can improve the presentation effect when displaying more than two display images.

[0008] In order to achieve the above-mentioned purpose, an image processing device involved in an embodiment of the present invention has a processor, and displays more than two display images in a display form including a manner in which the number of display images in a screen changes. The image processing device is characterized in that the processor performs the following processing: obtains at least one information among image acquisition conditions and image features for multiple images, selects multiple candidate images from multiple images based on at least one information obtained for multiple images, sets two or more candidate images that meet specified conditions among the multiple candidate images as two or more display images, and sets a display method for the two or more display images.

[0009] Also, the plurality of images may include photographed images.

[0010] Furthermore, the image acquisition conditions may include photographing conditions of the photographed image.

[0011] Furthermore, the photographing conditions may be conditions set based on at least one of the photographing date and time, the photographing interval, the photographing activity, the photographing mode, the camera type, and the photographer.

[0012] Furthermore, the processor may perform the following processing: selecting a plurality of candidate images from the plurality of images based on image features obtained by analyzing the plurality of images.

[0013] Furthermore, the processor may perform the following processing: selecting two or more candidate images from the plurality of candidate images based on image features obtained by analyzing the plurality of candidate images.

[0014] Furthermore, the processor may perform the following processing: calculating scores based on the image features for the two or more display images; and setting display methods for the two or more display images according to the scores.

[0015] Furthermore, the display method is at least one of a display position of a display image within a display area, a display angle of the display image relative to the display area, and a display time during which the entire image area of ​​the display image is displayed.

[0016] Furthermore, the processor may perform processing to set at least one of a synthesized image obtained by synthesizing a first region different from the image region with respect to an image region including the subject and a synthesized image obtained by synthesizing two or more images as the display image.

[0017] Furthermore, the processor may perform a process of setting an image with a framed area obtained by synthesizing the frame area of ​​the first area outside the image area as the display image.

[0018] Furthermore, the processor may perform the following processing: selecting the type of the frame area according to at least one of a preset selection criterion and input information from the user.

[0019] Furthermore, the processor may perform the following processing: according to the type of the frame area, a display method of the images with two or more frame areas is set.

[0020] Furthermore, the processor may perform the following processing: compared with the display position of the image with a framed area having a framed area having a first outer size, the display position of the image with a framed area having a second outer size smaller than the first outer size is set to a position away from the reference position within the display area.

[0021] Furthermore, the processor may perform the following processing: according to the type of the frame area, the display order of the images with two or more frame areas is set.

[0022] Furthermore, the processor may perform the following processing: according to the number of images that can be arranged on the same print surface during printing, the number of the displayed images is set to more than two.

[0023] Furthermore, in order to solve the above-mentioned problems, an image processing method involved in one embodiment of the present invention displays two or more display images in a display form including a manner in which the number of display images in a screen changes, and the image processing method is characterized in that the following processing is performed by a processor: for multiple images, at least one information among image acquisition conditions and image features is acquired; based on at least one information acquired for multiple images, multiple candidate images are selected from multiple images; and among the multiple candidate images, two or more candidate images that meet specified conditions are set as two or more display images, and a display method for the two or more display images is set.

[0024] Furthermore, according to one embodiment of the present invention, a program can be realized for causing a computer to execute each process included in the above-mentioned image processing method.

[0025] Furthermore, according to one embodiment of the present invention, a recording medium can be realized, which is a computer-readable recording medium having a program recorded thereon for causing a computer to execute each process included in the above-mentioned image processing method.

[0026] Effects of the Invention

[0027] According to one embodiment of the present invention, there can be provided an image processing device, an image processing method, a program and a recording medium capable of improving the presentation effect when displaying two or more display images. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 FIG. 1 is a diagram showing a state in which a display image is displayed in image processing according to an embodiment of the present invention.

[0029] Figure 2 This is a diagram (part 2) showing a state in which a display image is displayed in the image processing involved in one embodiment of the present invention.

[0030] Figure 3 FIG. 3 is a diagram showing a state in which a display image is displayed in the image processing according to one embodiment of the present invention.

[0031] Figure 4 FIG. 4 is a diagram showing a state in which a display image is displayed in the image processing according to one embodiment of the present invention.

[0032] Figure 5It is a diagram showing a configuration example of an image processing system including an image processing device according to an embodiment of the present invention.

[0033] Figure 6 This is a diagram for explaining the functions of an image processing device according to one embodiment of the present invention.

[0034] Figure 7 This is a diagram showing a selection screen for selecting a plurality of images in image processing according to one embodiment of the present invention.

[0035] Figure 8 This is a diagram showing a selection screen for selecting whether to print a displayed image in image processing according to an embodiment of the present invention.

[0036] Fig. 9 This is a diagram showing a selection screen for displaying an arrangement pattern of images when printing is selected in the image processing according to the embodiment of the present invention.

[0037] Fig.10 This is a diagram showing a selection screen for selecting a display image for setting a frame region in image processing according to one embodiment of the present invention.

[0038] Fig.11 This is a diagram showing a selection screen for selecting a frame region type in image processing according to an embodiment of the present invention.

[0039] Fig.12 This is a diagram showing the sequence of an image processing flow involved in one embodiment of the present invention.

[0040] Explanation of symbols

[0041] 10-image processing device, 21-processor, 22-memory, 23-communication interface, 24-storage device, 31-image acquisition unit, 32-information acquisition unit, 33-candidate image selection unit, 34-display image number setting unit, 35-display image setting unit, 36-border area setting unit, 37-display order setting unit, 38-display method setting unit, 39-display data generation unit, 40-image display unit, 41-print data generation unit, 100-user terminal, A-display area, C-reference position, H1, H2, H3, V1, V2, V3-distance, N-network, P1, P2, P3, P4-display image, S-image processing system. DETAILED DESCRIPTION

[0042] A specific embodiment of the present invention is described with reference to the accompanying drawings. However, the embodiment described below is only an example for easy understanding of the present invention and does not limit the present invention. In addition, the present invention can be changed or improved from the following embodiments as long as it does not deviate from its purpose. In addition, the present invention includes its equivalents.

[0043] In this specification, the concept of "device" includes a single device that performs a specific function, and also includes a combination of multiple devices that are dispersed and exist independently of each other and cooperate (coordinate) to perform the specific function.

[0044] Furthermore, in this specification, an “image” is a display object, and more specifically, an image represented by digital image data (hereinafter referred to as image data).

[0045] The image data may include, for example, image data in irreversible compression such as JPEG (Joint Photographic Experts Group) format, and image data in lossy compression such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format.

[0046] The image data may include supplementary information such as image acquisition conditions described later.

[0047] The “image” may include, for example, images photographed by a user terminal with a camera function or a photographing device such as a camera (photographed images), and images other than photographed images (created images) such as CG (Computer Graphics) images drawn using a computer.

[0048] Furthermore, the “image” may be a composite image obtained by synthesizing a first region different from the image region with respect to an image region including the subject. Specifically, it may be an image with a frame region obtained by synthesizing a frame region (equivalent to the first region) outside the image region, or a decorative image obtained by synthesizing a decorative region (equivalent to the first region) such as a stamp inside or outside the image region.

[0049] Furthermore, the “image” may be an image having no frame region and consisting only of an image region.

[0050] Here, the "frame area" is a blank area in the shape of a rectangular or square frame formed along the outer edge of a rectangular or square image, and more specifically, is an area surrounding the outside of an image area including a subject, i.e., a person, etc. An image with a frame area can be obtained, for example, when photographing with a camera or other photographic equipment having a frame area setting function, or can be obtained by synthesizing a frame area on the outside of an image area for a photographed image using a computer or the like.

[0051] Furthermore, the “image” may be a composite image obtained by combining two or more images, and more specifically, may be a collage image in which a plurality of images are arranged in a template or the like.

[0052] In this specification, a "user" is a user who utilizes the image processing apparatus of the present invention. Utilizing the image processing apparatus means utilizing the functions of the image processing apparatus, and in addition to directly operating the image processing apparatus, it also includes utilizing the functions of the image processing apparatus through a device that can communicate with the image processing apparatus (e.g., a user terminal).

[0053] <<About Image Processing>>

[0054] refer to Figure 1 to Figure 4 , a series of processes related to image processing (hereinafter referred to as “image processing”) implemented using an image processing device and an image processing method involved in one embodiment of the present invention are described. Figure 1 to Figure 4 Shows the transition of the user terminal screen from Figures 1 to 4 Arranged in chronological order.

[0055] Figure 1 to Figure 4 Each of the four display images P1 , P2 , P3 , and P4 shown is an image with a framed area.

[0056] The "user terminal" is a computer used by the user, and specifically, is a smart device such as a smartphone, a tablet terminal, or a notebook PC (Personal Computer), etc. An application for selecting a plurality of images for image processing (hereinafter referred to as an image selection application) is installed in the user terminal.

[0057] In image processing, a plurality of candidate images are selected from a plurality of images, and then two or more candidate images are selected from the plurality of candidate images as two or more display images, and a display method is set for the selected two or more display images. Then, in the set display method, the two or more display images are displayed in a display form including a method in which the number of display images in a screen is changed.

[0058] Therefore, if Figure 1 to Figure 4As shown, multiple (four) display images P1, P2, P3, and P4 are displayed in stages from the depth side to the front side in the depth direction of the screen. In more detail, the four display images P1, P2, P3, and P4 are displayed in the order of display image P1, display image P2, display image P3, and display image P4, for example, in a display area A having a quadrilateral shape. The newly added display image is configured to hide a part or all of the display image already configured in the display area A. Alternatively, the newly added display image is configured at a position that does not hide the display image already configured in the display area A.

[0059] If the "method in which the number of displayed images changes" is described in more detail, when displaying a display image, a part or all of the display image on the depth side may be hidden due to the display image on the near side, and sometimes it may not be hidden. Even when a part of the display image on the depth side is displayed, the number of displayed images in the screen changes. That is, the number of displayed images increases by one according to the amount of the added display image on the near side. This method is the "method in which the number of displayed images changes."

[0060] On the contrary, when all the images displayed on the depth side are hidden by the images displayed on the near side, the number of images displayed on the screen does not change. That is, even if the images displayed on the near side are added, the number of images displayed on the screen does not increase or decrease as a result of not displaying the images displayed on the depth side. This method is a method in which the number of images displayed does not change.

[0061] In the image processing, two or more display images are displayed in a display mode including a mode in which the number of display images in a screen is changed.

[0062] Next, the “display method” is described in more detail. The display method refers to at least one of the display position of the display image in the display area, the display angle of the display image relative to the display area, and the display time for which the entire image area of ​​the display image is displayed.

[0063] exist Figure 1 to Figure 4 In the example shown, the display angle of each of the four display images P1, P2, P3, and P4 is randomly set within a predetermined angle range (e.g., -5° to +5°) relative to the display area A. The display time of each of the four display images P1, P2, P3, and P4 is set to a constant time. On the other hand, the display position of each of the four display images P1, P2, P3, and P4 is set according to the type of the frame area.

[0064] The “type of frame area” refers to the outer dimensions of the frame area and the aspect ratio of the frame area. The “outer dimensions of the frame area” refers to the area of ​​the inner area surrounded by the outer shape of the frame area, that is, the area of ​​the entire area of ​​the image with the frame area. The “aspect ratio of the frame area” refers to the aspect ratio of the outer shape of the frame area, that is, the aspect ratio of the image with the frame area.

[0065] exist Figure 1 to Figure 4 In the example shown, a display image having a larger frame area is displayed closer to the reference position C located at the center of the display area A. On the other hand, a display image having a smaller frame area is displayed farther from the reference position C.

[0066] More specifically, in the four display images P1, P2, P3, and P4, Figure 1 and Figure 4 As shown, the outer dimensions of the frame areas of the two display images P1 and P4 are the same, and the outer dimensions of the frame area are the smallest. Figure 2 As shown, the outer dimensions of the frame area of ​​the display image P2 are larger than those of the two display images P1 and P4, and the outer dimensions of the frame area are smaller than those of the display image P3. Figure 3 As shown, among the four display images P1, P2, P3, and P4, the outer dimensions of the frame area of ​​the display image P3 are the largest.

[0067] Therefore, when the long side direction of the user terminal screen is assumed to be the up-down direction and the short side direction of the screen is assumed to be the left-right direction, as shown in FIG. Figure 1 and Figure 4 As shown, the center position of each of the two display images P1 and P4 in the vertical direction and the horizontal direction is displayed at a position away from the reference position C of the display area A by a distance V1 in the vertical direction and by a distance H1 in the horizontal direction. Figure 2 As shown, the center position of the display image P2 is displayed at a position which is a distance V2 shorter than the distance V1 in the vertical direction and a distance H2 shorter than the distance H1 in the horizontal direction from the reference position C of the display area A. Figure 3 As shown, the display image P3 is displayed with its center position at a position which is separated from the reference position C of the display area A by a distance V3 shorter than the distance V2 in the vertical direction and by a distance H3 shorter than the distance H2 in the horizontal direction.

[0068] Furthermore, as long as the above-mentioned distances V1, V2, V3, H1, H2, and H3 are set, the display positions of the four display images can be set at any positions around the reference position C, and are randomly set in this example.

[0069] According to the display method set in this way, the four display images P1, P2, P3, and P4 are displayed in the display area A of the screen while the number of display images is changed stepwise.

[0070] In addition, the display method described above is only an example, and other display methods may be used. For example, together with the display position, at least one of the display angle and the display time may be set according to the type of the frame area.

[0071] Furthermore, in the above example, a display image having a larger frame area is displayed at a position closer to the reference position C located at the center of the display area A, and a display image having a smaller frame area is displayed at a position farther from the reference position C. However, the above example is only one example and is not limited thereto. For example, a display image having a larger frame area is displayed at a position farther from the reference position C located at the center of the display area A, and a display image having a smaller frame area is displayed at a position closer to the reference position C.

[0072] After the display method is set, display data (dynamic image) is generated to display two or more display images by the set display method. The display data is disclosed to acquaintances or the public, for example, through SNS, etc. The display data may be, for example, a slide show or a GIF (Graphics Interchange Format) dynamic image.

[0073] As described above, in an image processing device according to an embodiment of the present invention, multiple candidate images are selected from multiple images, and then two or more candidate images are selected from the multiple candidate images as two or more display images. That is, since the image selection is performed in two stages, a display image that conforms to a determined theme, etc. is appropriately selected. In this way, by using two or more display images that conform to a theme, etc., the presentation effect when displaying two or more display images can be improved.

[0074] Furthermore, in an image processing device according to an embodiment of the present invention, two or more display images are displayed in a display mode including a change in the number of display images in the screen. Thus, not only the display content of the display image is displayed, but also the state in which the relationship between the two or more display images is gradually changed is displayed, thereby improving the presentation effect when displaying the two or more display images.

[0075] In particular, in the image processing device according to one embodiment of the present invention, since the display method of two or more display images is set, the relationship between the two or more display images can be further changed, and as a result, the presentation effect can be further improved.

[0076] Furthermore, in the image processing device according to one embodiment of the present invention, when the photographed image is included in two or more display images, it is possible to improve the presentation effect when the photographed image is displayed.

[0077] Furthermore, in an image processing device according to an embodiment of the present invention, the display method is at least one of a display position, a display angle, and a display time. Therefore, when displaying two or more display images, the demonstration effect of at least one of the display position, the display angle, and the display time can be improved.

[0078] <<Configuration example of an image processing system according to an embodiment of the present invention>>

[0079] refer to Figure 5 , the structure of an image processing system (hereinafter referred to as “image processing system S”) including an image processing device involved in one embodiment of the present invention is described.

[0080] like Figure 5 As shown, the image processing system S is composed of an image processing apparatus 10 and a plurality of user terminals 100. The image processing apparatus 10 and the plurality of user terminals 100 are connected via a network N so as to be communicable.

[0081] The image processing device 10 is constituted by a server computer. The image processing device 10 may be constituted by one server computer, or may be constituted by a plurality of server computers arranged in parallel and dispersed. Furthermore, the server computer constituting the image processing device 10 may also be a server computer for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service) or IaaS (Infrastructure as a Service). In this case, if necessary information is input into the user terminal 100, the server computer performs various processes (calculations) according to the input information, and the calculation results are output on the user terminal 100 side. That is, the function of the server computer as the image processing device 10 can be utilized on the user terminal 100 side.

[0082] like Figure 5 As shown, the computer constituting the image processing apparatus 10 includes a processor 21 , a memory 22 , a communication interface 23 , and a storage device 24 .

[0083] The processor 21 can be composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a TPU (Tensor Processing Unit) or an ASIC (Application Specific Integrated Circuit).

[0084] The memory 22 is composed of semiconductor memories such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 22 stores a program for executing image processing (hereinafter referred to as an image processing program). The image processing program is a program for causing the processor 21 to perform each process of the image processing method described later. The image processing program can be obtained by reading from a computer-readable recording medium, or by downloading via a communication network such as the Internet or an intranet.

[0085] The communication interface 23 may be constituted by, for example, a network interface card or a communication interface board, etc. The computer constituting the image processing apparatus 10 can communicate with other devices connected to the Internet or a mobile communication line, etc., via the communication interface 23 .

[0086] The storage device 24 is composed of, for example, a flash memory, a HDD (Hard Disc Drive), an SSD (Solid State Drive), an FD (Flexible Disc), an MO disk (Magneto-Optical Disc), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD card (Secure Digital card), or a USB memory (Universal Serial Bus memory). The storage device 24 may be built into a computer main body constituting the image processing device 10, or may be installed in the computer main body in an external form.

[0087] The user terminal 100 includes a processor, a memory, a communication interface, a recording device, a camera, an input device, and an output device.

[0088] <<Functions of the Image Processing Device According to One Embodiment of the Present Invention>>

[0089] Next, the configuration of the image processing device 10 according to one embodiment of the present invention will be described again from the functional aspect.

[0090] like Figure 6 As shown, the image processing device 10 has an image acquisition unit 31, an information acquisition unit 32, a candidate image selection unit 33, a display image number setting unit 34, a display image setting unit 35, a border area setting unit 36, a display order setting unit 37, a display method setting unit 38, a display data generation unit 39, an image display unit 40 and a print data generation unit 41.

[0091] These functional units are implemented by cooperation between hardware devices included in a computer constituting the image processing apparatus 10 and a program (ie, software) installed in the computer.

[0092] Each functional unit will be described below.

[0093] <Image acquisition unit>

[0094] The image acquisition unit 31 acquires a plurality of images transmitted from the user terminal 100. More specifically, the user selects a plurality of images to be processed on the screen of the user terminal 100, and the plurality of images selected from the user terminal 100 are transmitted from the user terminal 100, whereby the image acquisition unit 31 receives and acquires the plurality of images via the network N. Figure 7 In the example shown, after the user starts the image selection application installed in the user terminal 100, the screen of the user terminal 100 is switched to a screen displaying a plurality of images to be selected through a predetermined operation. If a plurality of images are selected through the displayed selection screen, the selected plurality of images are sent from the user terminal 100 to the image processing device 10, and the image acquisition unit 31 acquires the plurality of images.

[0095] The multiple images to be selected may be the above-mentioned photographic images, created images, images with framed areas, images without framed areas, decorative images, collage images, etc., and they may also exist in a mixed state. Figure 7 In the example shown, a situation where both images with framed regions and images without framed regions exist in a mixed state is shown.

[0096] The plurality of images to be selected may be images stored in a storage device of the user terminal 100 or may be images stored in an external device that can communicate via the network N, such as an online storage or a database server.

[0097] <Information Acquisition Department>

[0098] The information acquisition unit 32 acquires at least one of the image acquisition conditions and the image characteristics for the plurality of images. Alternatively, the information acquisition unit 32 may acquire the above information for each of the plurality of images or for at least one of the plurality of images.

[0099] (Image acquisition conditions)

[0100] The “image acquisition conditions” include photographing conditions for photographed images and creation conditions for created images such as CG images.

[0101] The "photography conditions" are conditions set based on at least one of the photography date and time, photography interval, photography activity, photography mode, camera type, photographer, and photography location.

[0102] The “photographing date and time information” includes information such as the date and time when the photograph was taken.

[0103] The “photographing interval information” is information on the time interval between photographing dates and times of each of two photographed images related to each other.

[0104] The “photographic event information” is information related to the photographed event, and includes, for example, information such as the event name.

[0105] The "photography mode information" includes information related to the continuous shooting mode or the automatic shooting mode (interval mode). Both the "continuous shooting mode" and the "automatic shooting mode" are used when a plurality of photographic images are continuously acquired within a specified time. The "continuous shooting mode" is used, for example, when a plurality of photographic images are acquired while changing the angle of view and the position of the photographic device during photography, and the "automatic shooting mode" is used, for example, when a plurality of photographic images are acquired without changing the angle of view and the position during photography.

[0106] The “camera model information” includes, for example, the model name and manufacturer name of photographic equipment such as a camera.

[0107] “Photographer information” includes the photographer’s personal information within the scope of permitted disclosure.

[0108] The “photography location information” includes information such as the latitude, longitude, and altitude of the photography location based on, for example, GPS (Global Positioning System).

[0109] On the other hand, the "creation condition" is a condition set based on at least one information of the creation date and the creator.

[0110] The “creation date and time information” includes information on the date and time at which the image was created.

[0111] “Creator information” includes the creator’s personal information within the scope permitted for disclosure.

[0112] (Image Features)

[0113] "Image features" are obtained by analyzing multiple images. Image features are information related to the image quality of each area of ​​the image, the grayscale values ​​of the pixels included in each area, and the information of the subject estimated based on this information. The subject information may include the type of subject, the state of the subject, the position of the subject in the image, the facial expression when the subject is a person, etc. When the subject is an object, for example, it may include information related to determining objects related to activities such as sports games and weddings, more specifically, information such as wedding dresses. When the subject is a landscape, for example, it may include information related to determining landscapes such as mountains, seas, and skies. In addition, even if it is the same landscape (the same place), it may be a different subject if the shooting range (field of view, etc.) is different.

[0114] Furthermore, the image feature is preferably a feature that can be quantified, vectorized, or tensorized. In this case, the image analysis result becomes a quantified, vectorized, or tensorized image feature, that is, a feature amount.

[0115] In this specification, "image features" refer to features in the image area, and are distinguished from features of a frame area surrounding the outside of the image area. The "frame area features" will be described later.

[0116] <Candidate image selection section>

[0117] The candidate image selection unit 33 selects a plurality of candidate images from the plurality of images based on the at least one information obtained for the plurality of images. In addition, the candidate images may include an image obtained by synthesizing a first region different from the image region (e.g., a frame region and a decorative region, etc.) with respect to an image region including a subject (e.g., an image with a frame region and a decorative image, etc.), and an image obtained by synthesizing two or more images (image regions) (e.g., a collage image, etc.).

[0118] Two selection methods are given below as examples.

[0119] The first selection method is a selection method based on image acquisition conditions. For example, based on information related to the continuous shooting mode or automatic shooting mode in the shooting conditions, multiple images continuously shot in the continuous shooting mode or automatic shooting mode are selected as multiple candidate images.

[0120] In this way, in the image processing device 10, multiple candidate images can be selected from multiple photographed images based on the photographic conditions of the photographed images, more specifically, the photographic date and time, the photographic interval, the photographic activity, the photographic mode, the camera model, and the conditions set by at least one of the photographer.

[0121] As a second selection method, a selection method based on image features is used. The candidate image selection unit 33 selects a plurality of candidate images from a plurality of images based on image features obtained by analyzing the plurality of images.

[0122] As an example, the candidate image selection unit 33 may compare the feature quantities of the subject included in a plurality of images to calculate the similarity of the subject, and set a determined number of images as candidate images in order from images with high similarity of the subject. Alternatively, conversely, a determined number of images may be set as candidate images in order from images with low similarity of the subject.

[0123] In another example, the candidate image selection unit 33 calculates scores of the plurality of images based on the feature quantity of the subject, and sets a predetermined number of images as candidate images in order from images with higher scores. For example, when the subject is a person, the scores of the plurality of images may be calculated based on feature quantities such as smile degree or number of people, and a predetermined number of images may be set as candidate images in order from images with higher scores.

[0124] In this manner, the image processing device 10 can select a plurality of candidate images from a plurality of images based on the image features obtained by analyzing the plurality of images.

[0125] <Display image quantity setting section>

[0126] The display image number setting unit 34 sets the number of display images to be displayed in the display area of ​​the screen.

[0127] Hereinafter, the method of setting the number of display images will be described by taking four setting methods as examples.

[0128] As a first setting method, the total number of the selected candidate images is set as the number of display images.

[0129] As a second setting method, an upper limit is set for the number of images to be displayed. When the number of multiple candidate images is greater than the upper limit, the upper limit is set as the number of images to be displayed; when the number of multiple candidate images is less than the upper limit, the number is set as the number of images to be displayed.

[0130] As a third setting method, the number of images input by the user is set as the number of displayed images.

[0131] As a fourth setting method, when the user prints the display images, the number of two or more display images is set according to the number of images that can be arranged on the same print surface when printing. Thus, when displaying two or more display images, the number of display images corresponding to the number of images when printing can be displayed.

[0132] If the fourth setting method is described in more detail, the screen of the user terminal 100 is switched to Figure 8 The screen shown displays a screen requesting selection of whether to subsequently print two or more displayed images.

[0133] When the user selects not to print the displayed images, the number of displayed images is set by any one of the first to third setting methods described above.

[0134] On the other hand, when the user selects to print the displayed images, the number of displayed images is set by the fourth setting method.

[0135] More specifically, the screen of the user terminal 100 is switched to Fig. 9 The screen shown displays a plurality of candidates for arrangement patterns of two or more display images arranged on the same print surface, and displays a screen requesting selection of any one arrangement pattern from among the candidates for the arrangement patterns.

[0136] The "arrangement pattern" may be a pattern in which a plurality of display images of the same outer dimensions are arranged on the same printing surface, or a pattern in which a plurality of display images of different outer dimensions are arranged on the same printing surface. The number of display images arranged on the same printing surface may be set to, for example, a square value of n (n is a natural number greater than or equal to 2), or may be set to a number based on the square value of n.

[0137] In addition, in the above example, when the user selects to print the display images, the number of display images is set by the fourth setting method, but the present invention is not limited thereto. For example, even when the user selects not to print the display images, the number of display images can be set by any of the first to third setting methods. That is, the number of display images included in the display data (dynamic image) can also be set to a value different from the number of images arranged on the same print surface when printing.

[0138] <Display image setting section>

[0139] The display image setting unit 35 sets two or more candidate images satisfying a predetermined condition from among a plurality of candidate images as two or more display images.

[0140] The "prescribed condition" is a condition set based on the number of display images set by the display image number setting unit 34 and at least one of the image acquisition conditions and image features. In other words, the display image setting unit 35 selects two or more candidate images corresponding to the number of display images from a plurality of candidate images based on the number of display images and at least one of the image acquisition conditions and image features, and sets them as two or more display images.

[0141] More specifically, the display image setting unit 35 may select two or more candidate images from the plurality of candidate images based on the image features obtained by analyzing the plurality of candidate images, and set the selected candidate images as the two or more display images.

[0142] As an example, the display image setting unit 35 may compare the feature quantities of the subject included in a plurality of candidate images to calculate the similarity of the subject, and set the number of images set by the display image number setting unit 34 as display images in order from the candidate images with high similarity of the subject. Alternatively, conversely, the number of images set by the display image number setting unit 34 may be set as display images in order from the candidate images with low similarity of the subject.

[0143] As another example, the display image setting unit 35 may calculate scores of a plurality of candidate images based on the feature amount of the subject, and set the number of images set by the display image number setting unit 34 as display images in order from the candidate images with the highest scores.

[0144] As described above, in the image processing device 10 , two or more candidate images selected based on the analysis results of a plurality of candidate images can be displayed as two or more display images.

[0145] In addition, the display image setting unit 35 may also set as a display image at least one of an image (hereinafter referred to as a synthesized image) obtained by synthesizing an image area including a subject with a first area different from the image area (for example, a frame area and a decorative area, etc.) and an image obtained by synthesizing two or more images (image areas) (for example, a collage image, etc.).

[0146] Therefore, in the image processing device 10, when the displayed image is a composite image, the presentation effect when the composite image is displayed can be improved. Also, when the displayed image is an image with a framed area, the presentation effect when the image with a framed area is improved.

[0147] <Border area setting section>

[0148] The frame region setting unit 36 ​​sets a frame region for the set two or more display images.

[0149] More specifically, the screen of the user terminal 100 is switched to Fig.10 The screen shown is a screen requesting the setting of a frame area for the two or more display images set as described above.

[0150] exist Fig.10 In the example shown, two of the four display images are display images with frame areas already set. Fig.10 The screen shown in FIG. 1 is a screen requesting to set a frame area for the remaining two display images. If a display image without a frame area is selected, the screen of the user terminal 100 is switched to Fig.11 The screen shown shows multiple ( Fig.11 In addition, the plurality of candidates for border regions also include candidates for borderless regions. If the user selects any type of border region, the type of border region is selected based on the input information from the user, and an image having the selected type of border region (an image with a border region) is set as a display image.

[0151] In addition, Fig.11 In the example shown, when a candidate for a frameless region is selected, an image of a frameless region is set as a display image.

[0152] And, in Fig.11 In the example shown, for an image that already has a border area set, no request is made to set a border area, but this is not limited to this. For example, a request may be made to select whether to change the type of border area for an image that already has a border area set, and the type of border area may be selected (changed) based on input information from the user.

[0153] And, in Fig.11 In the example shown, the type of the frame area is selected based on the input information from the user, but the present invention is not limited thereto, and the type of the frame area may be selected based on a pre-set selection criterion. For example, the frame area may be selected based on a selection criterion related to the aspect ratio, and specifically, a frame area having an aspect ratio closer to the aspect ratio of the target display image may be selected.

[0154] As described above, the frame area setting unit 36 ​​selects the type of frame area for two or more display images based on at least one of the preset selection criteria and the input information from the user. Thus, the image with the frame area corresponding to the selected frame area type can be set as the display image.

[0155] <Display order setting section>

[0156] The display order setting unit 37 sets the display order of two or more display images based on the image acquisition conditions, image (image area) characteristics, and frame area characteristics.

[0157] The "frame area feature" includes the presence or absence of a frame area in the display image, the type of frame area, the presence or absence of text in the frame area, and the presence or absence of decoration (pictures, patterns, etc.) in the frame area.

[0158] Hereinafter, the method of setting the display order of display images will be described by taking three setting methods as examples.

[0159] As a first setting method, which is a setting method based on the image acquisition condition, the display order setting unit 37 may set the display order of two or more display images in ascending or descending order of the shooting date and time based on the shooting date and time information, for example.

[0160] As a second setting method, which is a setting method based on image features, the display order setting unit 37 can set the display order of two or more display images in the order of the number of people in the image from the largest to the smallest according to the number of people in the image, for example, when the subject includes people. Alternatively, the display order can be set in a manner that the display images including the same subject are continuous, or conversely, the display order can be set in a manner that the display images including the same subject are discontinuous.

[0161] As a third setting method, which is a setting method based on the characteristics of the frame area, the display order setting unit 37 can set the display order of two or more images with frame areas according to the type of the frame area. For example, the display order setting unit 37 can set the display order in a manner that the display images including the same type of frame area are continuous, and conversely, the display order can be set in a manner that the display images including the same type of frame area are discontinuous.

[0162] As described above, in the image processing device 10 , by displaying display images in a display order corresponding to frame region features, more specifically, the types of frame regions, it is possible to improve the presentation effect when displaying two or more display images.

[0163] <Display method setting section>

[0164] The display method setting unit 38 sets the display method for the two or more display images. The display method setting unit 38 may set the display method for each of the two or more display images, or may set the display method for at least one of the two or more display images.

[0165] More specifically, the display method setting unit 38 sets the display methods of two or more display images, that is, at least one of the display position, display angle, and display time, based on at least one of the image (image area) feature and the frame area feature.

[0166] The display method setting unit 38 may calculate scores based on image features for two or more display images and set the display methods of the two or more display images based on the scores. This allows presentations corresponding to the scores to be performed when the two or more display images are displayed.

[0167] Furthermore, as described above, the display method setting unit 38 may also set the display method of two or more images with frame areas according to the type of frame area. Figure 1 to Figure 4 As shown, the display method setting unit 38 may also set the display position of the image with a framed area having a second outer size smaller than the first outer size to a position away from the reference position in the display area, compared to the display position of the image with a framed area having a framed area having a first outer size. Thus, when displaying two or more display images, it is possible to perform a presentation corresponding to the type of the framed area, specifically, the outer size of the image with the framed area.

[0168] <Display data generation unit>

[0169] The display data generating unit 39 generates display data for displaying two or more display images in the set display method. The generated display data may be stored in a storage device of the user terminal 100 or in an external device that can communicate with the image processing device 10 via the network N, such as an online storage or a database server.

[0170] <Image display unit>

[0171] The image display unit 40 transmits display data to the user terminal 100 and displays two or more display images on the screen of the user terminal 100 in a display mode including a mode in which the number of display images on the screen changes in the display method set above.

[0172] <Print data generation unit>

[0173] The print data generating unit 41 generates print data in which two or more display images are arranged on the same print surface in the selected arrangement pattern.

[0174] In addition, a code area such as a QR code (registered trademark) is attached to the printed surface, and information indicating the storage location of the above-mentioned display data is encoded and included in the code area. Therefore, by reading the code area, the storage location of the display data is determined based on the information included in the code area, and the display data stored in the determined storage location can be accessed. Then, by accessing the display data, two or more display images represented by the display data can be displayed in a set display method.

[0175] <<Regarding the image processing flow involved in one embodiment of the present invention>>

[0176] Next, the data processing flow, that is, the image processing flow using the image processing device 10 will be described. The image processing flow adopts the image processing method of the present invention, according to Fig.12 That is, Fig.12 Each step in the flowchart shown in FIG. 1 corresponds to each element constituting the image processing method of the present invention.

[0177] in addition, Fig.12 The illustrated flow is merely an example, and unnecessary steps may be deleted or new steps may be added without departing from the gist of the present invention.

[0178] First, the user starts up an image selection application installed in the user terminal 100, and a signal generated in conjunction with the start of the application is transmitted to the server computer constituting the image processing device 10. The image processing flow is started upon the transmission of the signal.

[0179] When a plurality of target images are transmitted from the user terminal 100, the processor 21 acquires the plurality of target images (S001). Next, the processor 21 acquires at least one information of image acquisition conditions and image features for the plurality of images (S002).

[0180] Next, the processor 21 selects a plurality of candidate images from the plurality of images based on the at least one information acquired with respect to the plurality of images ( S003 ).

[0181] Next, the processor 21 requests selection of whether to print two or more display images afterwards ( S004 ), and when receiving user input information that the two or more display images are not to be printed, the processor 21 skips step S005 and proceeds to step S006 .

[0182] On the other hand, when the processor 21 receives input information from a user to print two or more display images, it requests to select any one of the configuration patterns from a plurality of candidates of configuration patterns of the two or more display images configured on the same print surface. Then, the processor 21 sets the configuration pattern of the display image on the print surface according to the user's selection information (S005).

[0183] Next, the processor 21 sets the number of display images displayed in the display area of ​​the screen of the user terminal 100 (S006). More specifically, when the processor 21 receives input information from the user not to print more than two display images in step S004, for example, the processor 21 sets an upper limit on the number of display images, and when the number of multiple candidate images is greater than the upper limit, the upper limit is set to the number of display images, and when the number of multiple candidate images is less than the upper limit, the number is set to the number of display images. On the other hand, when the processor 21 receives input information from the user to print more than two display images in step S004, the processor 21 sets the number of images based on the configuration pattern set in step S005 to the number of display images.

[0184] Next, the processor 21 sets two or more candidate images satisfying a predetermined condition from among the plurality of candidate images as two or more display images ( S007 ).

[0185] Next, the processor 21 determines whether there is a display image without a border area among the two or more display images (S008). If there is no display image without a border area, that is, if all the two or more display images are images with border areas, step S009 is skipped and the process proceeds to step S010.

[0186] On the other hand, if there is a display image without a frame area, processor 21 sets a frame area for the display image without a frame area (S009). More specifically, processor 21 sets a frame area for the image without a frame area based on user input information regarding the type of the frame area.

[0187] In addition, the processor 21 may set an image without a frame area as a display image based on input information from the user.

[0188] Furthermore, the processor 21 may request a selection of whether to change the type of the frame area for the framed area image in which the frame area has been set, and change the type of the frame area based on input information from the user.

[0189] Next, the processor 21 sets the display order of the two or more display images (S010). Then, the processor 21 sets the display method of the two or more display images (S011).

[0190] Next, the processor 21 generates display data for displaying two or more display images in the set display method (S012). The processor 21 sends the display data to the user terminal 100, so that the two or more display images are displayed on the screen of the user terminal 100 in a display form including a change in the number of display images in the screen in the set display method (S013).

[0191] Then, the processor 21 generates print data in which two or more display images are arranged on the same print surface according to the arrangement pattern set in step S005 ( S014 ).

[0192] When the series of processing so far is completed, the image processing flow ends.

[0193] <<Other Implementation Methods>>

[0194] In the above-mentioned image processing system, the image processing device 10 is constituted by a server computer, but the present invention is not limited thereto. For example, the image processing device of the present invention may also be constituted by a user terminal.

[0195] <About the structure of the processor>

[0196] The image processing device of the present invention and the processor included in the image processing device include various processors. Among the various processors, for example, there is a general-purpose processor, ie, a CPU, which executes software (program) and functions as various processing units.

[0197] Among various processors, there is a processor called PLD (Programmable Logic Device) whose circuit configuration can be changed after manufacturing, such as FPGA (Field Programmable Gate Array).

[0198] In addition, various processors include processors such as ASIC (Application Specific Integrated Circuit) which have a circuit structure specially designed to perform specific processing, that is, dedicated circuits.

[0199] Furthermore, the image processing device of the present invention and a processing unit possessed by the image processing device may be constructed by one of the various processors mentioned above, or may be constructed by a combination of two or more processors of the same type or different types, such as a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc.

[0200] Furthermore, the plurality of functional units included in the image processing apparatus of the present invention may be configured by one of various processors, or two or more of the plurality of functional units may be configured by one processor.

[0201] Furthermore, as described above, it may be in the form of forming a processor by combining more than one CPU and software, which functions as a plurality of functional units.

[0202] Furthermore, the following method may be adopted: for example, a processor that realizes the overall system functions including the image processing device of the present invention and multiple functional units in the image processing device by using a single IC (Integrated Circuit) chip, represented by SoC (System on Chip). In addition, more specifically, the hardware structure of the various processors described above may be a circuit (Circuitry) composed of a combination of circuit elements such as semiconductor elements.

Claims

1. An image processing device comprising a processor, and displaying two or more display images in a display method including a method in which the number of display images in a screen changes, wherein: The processor performs the following processing: For a plurality of images, obtaining at least one of image acquisition conditions and image features; selecting a plurality of candidate images from the plurality of images according to the at least one information obtained for the plurality of images; Two or more candidate images satisfying a predetermined condition among the plurality of candidate images are used as the two or more display images, and a display method for the two or more display images is set.

2. The image processing device according to claim 1, wherein: The plurality of images includes photographed images.

3. The image processing device according to claim 2, wherein: The image acquisition conditions include photographing conditions of the photographed image.

4. The image processing device according to claim 3, wherein: The photographing conditions are conditions set based on at least one of the photographing date and time, the photographing interval, the photographing activity, the photographing mode, the camera type, and the photographer.

5. The image processing device according to claim 1, wherein: The processor performs the following processing: The plurality of candidate images are selected from the plurality of images based on image features obtained through analysis of the plurality of images.

6. The image processing device according to claim 1, wherein: The processor performs the following processing: The two or more candidate images are selected from the plurality of candidate images according to image features obtained through analysis of the plurality of candidate images.

7. The image processing device according to claim 1, wherein: The processor performs the following processing: For the two or more displayed images, calculating scores based on image features; and The display method of the two or more display images is set according to the score.

8. The image processing device according to claim 1, wherein: The display method is at least one of a display position of the display image in a display area, a display angle of the display image relative to the display area, and a display time during which the entire image area of ​​the display image is displayed.

9. The image processing apparatus according to claim 1, wherein: The processor performs the following processing: At least one of a composite image obtained by combining an image region including a subject with a first region different from the image region and a composite image obtained by combining two or more images is set as the display image.

10. The image processing device according to claim 9, wherein: The processor performs the following processing: An image of a framed region obtained by combining the frame region of the first region with the outside of the image region is set as the display image.

11. The image processing apparatus according to claim 10, wherein: The processor performs the following processing: The type of the frame area is selected based on at least one of a preset selection criterion and input information from a user.

12. The image processing apparatus according to claim 10, wherein: The processor performs the following processing: The display method of the images in two or more framed regions is set according to the type of the framed region.

13. The image processing apparatus according to claim 10, wherein: The processor performs the following processing: Compared with the display position of the image of the framed area of ​​the framed area having the first outer size, the display position of the image of the framed area having the second outer size smaller than the first outer size is set at a position farther away from the reference position in the display area.

14. The image processing apparatus according to claim 10, wherein: The processor performs the following processing: The display order of the images in the two or more framed areas is set according to the type of the framed area.

15. The image processing apparatus according to claim 1, wherein: The processor performs the following processing: The number of the two or more display images is set according to the number of images that can be arranged on the same print surface during printing.

16. An image processing method, which displays two or more display images in a display manner including a manner in which the number of display images in a screen changes, the image processing method performing the following processing by a processor: For a plurality of images, obtaining at least one of image acquisition conditions and image features; selecting a plurality of candidate images from the plurality of images according to the at least one information obtained for the plurality of images; and Two or more candidate images satisfying a predetermined condition among the plurality of candidate images are used as the two or more display images, and a display method for the two or more display images is set.

17. A recording medium which is a computer-readable recording medium, The recording medium records a program for causing a computer to execute each process included in the image processing method according to claim 16 .

Images