Information processing device, processing method, and program

Abstract

To enable appropriate image selection when choosing some images from multiple images acquired by multiple users.SOLUTION: An information processing device includes management means for associating and managing information identifying images with information identifying users, setting means for setting conditions for selecting images for each user, calculation means for calculating an evaluation value for each image managed by the management means, and selection means for selecting images using the evaluation values calculated by the calculation means and the conditions set for each user by the setting means.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a processing method, and a program. [Background technology]

[0002] Methods for selecting and outputting images from a plurality of images owned by a user are known. For example, Patent Document 1 proposes a method for automatically selecting better shots from a large number of images. This method performs an analysis process on a plurality of photographic images and stores them in a manner that distinguishes between images that meet a predetermined judgment criterion (acceptable images) and other images (unacceptable images). This allows the method to automatically select better shots from other photographic images on behalf of the user. This reduces the user's effort in selecting images from a group of images to create a photo album. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2004-236120 Summary of the Invention [Problem to be solved by the invention]

[0004] Consider a use case where multiple participants at an event each take a photo, and the event organizer collects the photos, selects the photos, creates an album, and provides it to the participants. In this case, if only better shots are selected as in Patent Document 1, there is a possibility that the images taken by a specific participant will be concentrated. This can cause inconvenience when it is desired to select images taken by each of the multiple participants evenly. Therefore, an object of the present invention is to enable appropriate image selection when selecting a portion of images from multiple images taken by multiple users. [Means for solving the problem]

[0005] As a means for achieving the above object, an information processing device of the present invention comprises the following arrangement.

[0006] That is, the system is characterized by comprising a management means for managing information identifying an image in association with information identifying a user, a setting means for setting conditions for selecting an image for each user, a calculation means for calculating an evaluation value for each image managed by the management means, and a selection means for selecting an image using the evaluation value calculated by the calculation means and the conditions set for each user by the setting means. [Effects of the Invention]

[0007] According to the present invention, when selecting some images from a plurality of images acquired by a plurality of users, it is possible to more appropriately select the images. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device. [Figure 2] FIG. 1 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 3] FIG. 10 is a diagram showing an example of image information managed by an image information management unit. [Figure 4] 10 is a flowchart showing an example of a series of processes for selecting an image by an information processing device. [Figure 5] FIG. 10 is a diagram showing an example of a selection condition setting screen displayed on the display unit. [Figure 6] FIG. 10 is a diagram showing an example of evaluation information output by an evaluation unit; [Figure 7] Flowchart showing an example of selection processing [Figure 8] 10 is a flowchart showing an example of image generation processing by a generation unit; [Figure 9] FIG. 10 is a diagram showing an example of a composite image generated by a generation unit. [Figure 10] 10 is a flowchart showing an example of a selection process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below based on preferred embodiments thereof with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0010] [Embodiment 1] An example of the hardware configuration of an information processing device according to this embodiment will be described with reference to the block diagram of FIG.

[0011] <Hardware configuration> The information processing device includes at least a CPU 101 , a RAM 102 , a ROM 103 , a network interface 104 , an external storage device 105 , a display device 106 , and an input device 107 .

[0012] The CPU 101 controls the operation of each component of the information processing device and is responsible for executing various processes (described later) performed by the information processing device. The RAM 102 is a memory for temporarily storing data and control information and serves as a work area used by the CPU 101 when executing various processes. The ROM 103 stores fixed operational settings and operating programs for the information processing device. The network interface 104 is a wired or wireless communication unit that provides the function of connecting to and communicating over a network. The information processing device can exchange various data with external devices via the network interface 104. The external storage device 105 is a device for storing data and has an interface that accepts I / O commands for reading and writing data. The external storage device 105 may be a hard disk drive (HDD), a solid-state drive (SSD), an optical disk drive, a semiconductor storage device, or other storage device. The external storage device 105 stores computer programs and data for causing the CPU 101 to execute various processes (described later) performed by the information processing device. The display device 106 is, for example, an LCD (Liquid Crystal Display) and displays information required by the user. The input device 107 is, for example, a keyboard, a mouse, a touch panel, or the like, and receives necessary input from the user.

[0013] <Functional configuration> Next, an example of the functional configuration of the information processing device according to this embodiment will be described with reference to the block diagram of Fig. 2. Each functional unit described below is implemented by hardware or software. In the case of software, each function is executed by the CPU 101 executing a program corresponding to each functional unit.

[0014] As shown in FIG. 2, the information processing device includes an image management unit 201, an image information management unit 202, a selection unit 203, an evaluation unit 204, a display unit 205, a selection condition setting unit 206, a generation unit 207, a correction unit 208, and a determination unit 209.

[0015] The image management unit 201 manages images owned by multiple users. The image information management unit 202 manages images by associating them with the users who own the images. An example of image information managed by the image information management unit 202 is shown in FIG. 3. Image information 301 is information in which an image ID for uniquely identifying an image, a user ID for identifying a user, and information indicating the date and time of shooting are associated with each other. This information is an example, and other information, such as information about the shooting location, may also be associated with the image.

[0016] The selection unit 203 selects images based on the set conditions according to the flow shown in FIG. 4 (described later). The evaluation unit 204 acquires evaluation information from an evaluation engine, for example, a convolutional neural network. The evaluation engine analyzes images and outputs evaluation information indicating image characteristics, such as "out of focus," "camera shake," and "blink," based on the analysis results including image feature values. "Out of focus" indicates that the focus is not on the subject, "camera shake" indicates that the camera moved when the image was captured, and "blink" indicates that the image contains a face with closed eyes. The display unit 205 displays images and selection conditions via the display device 106. The selection condition setting unit 206 sets image selection conditions for each user. Here, the selection conditions are the number of images or the ratio of the number of images. The generation unit 207 generates a composite image by combining multiple images according to the flow shown in FIG. 8 (described later). The correction unit 208 performs image correction processing using image processing algorithms, such as sharpening, camera shake correction, and cropping. The determining unit 209 calculates an evaluation value based on the evaluation information acquired by the evaluating unit, and determines an image whose evaluation value meets the criteria as a passing image.

[0017] <Processing flow> Next, a series of processing steps for selecting an image by the information processing device in this embodiment will be described with reference to the flowchart of FIG. 4. Each step will be described below with an S added to the beginning of the reference numeral. Each step of this flowchart is executed by CPU 101 executing a program stored in a memory such as external storage device 105.

[0018] In S401, the display unit 205 displays a selection condition setting screen, and the user sets the selection conditions. The selection conditions set by the user are accepted by the selection condition setting unit 206. FIG. 5 shows an example of the selection condition setting screen displayed by the display unit 205. The display unit 205 acquires the user IDs associated with each image to be selected from the image information management unit 202 and displays user information on the screen based on this information. FIG. 5 shows an example in which user IDs are listed, such as User001 and User002. The user information may be any user identification information. For example, if the user name corresponding to User001 is User A, it may be displayed as User A. Furthermore, the selection conditions can be adjusted for each user. FIG. 5 shows an example in which the number of images can be reduced by selecting the - button and increased by selecting the + button. Note that the number of images to be selected in S401 does not necessarily have to be set by the user on a setting screen; instead, the selection condition setting unit 206 may determine a uniform setting value for all users.

[0019] In S402, the evaluation unit 204 evaluates the image. FIG. 6 shows an example of evaluation information output by the evaluation unit 204. The image ID is an ID that identifies the image. The focus consistency is the result of evaluating the degree to which the subject in the image is in focus. The focus consistency is indicated by a value from 0 to 100, with a larger value indicating a better focus. The subject size is the result of evaluating the size of the subject in the image. The subject size is indicated by a value from 0 to 100, with a larger value indicating that the size is closer to the appropriate size. The blink detection is the result of analyzing how many faces with closed eyes are included in the image. The face direction is the result of evaluating the face direction of the subject. The face direction is indicated by a value from 0 to 100, with a larger value indicating that the face is facing more forward. The subject type is the result of analyzing the type of subject. If the subject is a person, the value is "person," and if the subject is a cake, the value is "cake." These evaluation information items are examples, and other evaluation information may be used. For example, information such as the degree of camera shake, face size, cropping, and over / under exposure may be used.

[0020] In S403, the determination unit 209 determines whether an image is acceptable based on the evaluation information. Specifically, an image whose evaluation value calculated based on the evaluation information satisfies a standard is determined to be acceptable. The evaluation value is calculated based on at least one of the evaluation information shown in FIG. 6. For example, if the evaluation value is the sum of the focus matching degree and the subject size, the evaluation value for Image001 would be 140, for Image002 90, and for Image003 80. If a threshold value of 100 is set as the standard for the evaluation value, Image001, whose evaluation value is 100 or greater, is determined to be acceptable. Note that the evaluation value may be calculated not only by addition, but also by average or maximum value. Furthermore, the subject type may be used as the evaluation value. For example, if the subject type is other than a person, the focus matching degree and the subject size are used, and if the subject type is a person, the evaluation value may be calculated by additionally using person-specific evaluation information such as blink detection and face direction.

[0021] Alternatively, a plurality of similar images may be grouped into a single image group, and images from each image group whose evaluation value meets a standard may be determined as acceptable images. For example, the grouping may be performed by grouping images whose difference in shooting date and time between images is equal to or less than a predetermined threshold, or whose color difference between images obtained using color information for each pixel is equal to or less than a threshold. Similar images may also be grouped using other methods. Images determined as acceptable images in S403 are classified into the accepted image group, and other images are classified into the rejected image group.

[0022] In S404, the selection unit 203 selects images according to the flow shown in Fig. 7, which will be described later. As a result, a group of adopted images is selected that satisfies the selection conditions set in S401. In this embodiment, a case where the number of selected images is set as the selection condition will be described.

[0023] 7 is a flow showing details of the selection process performed by the selection unit 203. In S701, the selection unit 203 acquires a user list linked to each image to be selected based on the image information of the image information management unit 202. The process from S702 to S705 is performed for each user included in the acquired user list information. The process from S702 to S705 for user i included in the user list will be described.

[0024] In S702, the selection unit 203 checks whether the selected number of images for user i have already been adopted in the group of adopted images. If the selected number of images have already been adopted, the processes from S703 to S705 are skipped and the process proceeds to the next user. On the other hand, if the selected number of images have not been adopted, in S703 the selection unit 203 selects the missing number of images from the images associated with user i included in the group of unadopted images in descending order of evaluation value, and classifies them as candidate images for adoption.

[0025] In S704, it is checked whether any of the images selected in S703 has an evaluation value equal to or less than a threshold. The evaluation value may be the same as the evaluation value used in S402 and S403, or a value calculated by a different method may be used. Note that if the same evaluation value as the evaluation value used in S402 and S403 is used as the evaluation value, it is preferable that the threshold be smaller than the threshold used in S403.

[0026] In S705, the correction unit 208 performs correction processing on the image determined to be below the threshold in S704. The correction processing is an appropriate correction processing for items with low evaluations in the image evaluation information. For example, if the degree of focus is low, processing such as sharpening processing or super-resolution processing is performed, and if the subject size is small, processing such as increasing the subject size by cropping is performed. By performing the correction processing in S705, images with low evaluations are corrected to increase their evaluations before being added to the group of adopted images, so it is possible to some extent to prevent low-evaluation images from being adopted as they are and from including images with low aesthetic quality in the group of adopted images. Note that the processing of S704 and S705 may not be performed.

[0027] In S706, an image generation process is performed according to the flow shown in Fig. 8, which will be described later. As a result, an image is generated in which the image included in the group of candidate images selected in S703 is synthesized with other images from the same group and an image from the group of adopted images.

[0028] FIG. 8 is a flow chart showing the flow of image generation processing by the generation unit 207. In S801, each image included in the adoption candidate image group is grouped with an image from the adoption image group. Specifically, images in the adoption image group that are taken at similar times to each image in the adoption candidate image group are grouped into the same group. In S802, the generation unit 207 generates a composite image from images in the same group. FIG. 9 shows an example of a composite image generated by the generation unit 207. FIG. 9(a) shows an example in which images included in the same group are arranged with their sizes changed according to the evaluation value of each image. FIG. 9(b) shows an example in which images included in the same group are displayed overlapping each other, with images with higher evaluation values ​​being closer to the foreground and images with lower evaluation values ​​being further back. This makes images with lower evaluation values ​​less noticeable, thereby preventing aesthetic loss. In S803, the composite image generated by the generation unit 207 is added to the adoption image group.

[0029] According to the present embodiment described above, it is possible to select an image of each user from an image group including images of multiple users based on the set selection conditions.

[0030] [Embodiment 2] In the first embodiment, in the selection process, an image with a high evaluation value is selected for each user from a group of rejected images, and an image is generated by combining the image with an adopted image taken at a similar date and time. However, when images taken at the same event by multiple users are selected as selection targets, there is a possibility that similar images taken at the same scene will be combined into one image when the adopted image and an image taken at a similar date and time are combined.

[0031] In this embodiment, an example is shown in which an image captured from a different viewpoint of the same scene as an image in a group of adopted images is selected from a group of unadopted images in the selection process. Note that components and steps having the same functions as those in other embodiments are given the same reference numerals, and descriptions of components that are structurally and functionally unchanged will be omitted.

[0032] <Processing flow> The selection process in this embodiment will be described with reference to the flow of the process shown in Fig. 10. Fig. 10 differs from Fig. 7 in the processes of S1001 and S1002.

[0033] In S1001, the selection unit 203 classifies, from among the images associated with user i included in the unaccepted image group, images that were taken at a time close to the time of the accepted image and from a viewpoint as different as possible from that of the accepted image as candidates for acceptance. Specifically, the process is as follows: First, from each image associated with user i included in the unaccepted image group, images whose difference in time of taking photos from the images included in the accepted image group is as small as possible are extracted.

[0034] In S1002, it is determined whether each image extracted in S1001 was taken from a different viewpoint from the adopted image, and it is determined to be a different viewpoint image. Specifically, it is determined based on conditions such as whether the subject type is different, or whether the subject type is a person and the difference in facial direction is as large as possible. The missing number of images are classified as candidate images for adoption in descending order of the degree of difference in subject type, or, in the case of the same subject, the degree of difference in facial direction.

[0035] In addition, when the subject type is the same, if the difference in the face direction is equal to or greater than a threshold, the image may be determined as a different viewpoint image. If the number of images determined as different viewpoint images is less than the required number, the required number may be classified as candidate images in descending order of evaluation value in step S703.

[0036] Of the images determined as different viewpoint images in S1002, only those with evaluation values ​​equal to or greater than a certain threshold may be adopted as candidate images for adoption. In this case, if the number of images determined as different viewpoint images is less than the required number, the required number may be classified as candidate images for adoption in order of highest evaluation value in step S703.

[0037] Alternatively, the selection condition may be set to a ratio of the number of sheets rather than the number of sheets.

[0038] According to the present embodiment described above, in the selection process, an image captured from a different viewpoint of the same scene as an image in the group of adopted images can be selected from the group of rejected images. This ensures that images from as many different viewpoints as possible are included when generating a composite image, thereby preventing the image from becoming monotonous.

[0039] [Other embodiments] Although the exemplary embodiments have been described above in detail, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), etc. Specifically, the present invention may be applied to a system made up of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus made up of a single device.

[0040] Needless to say, the object of the present invention can be achieved by the following: A recording medium (or storage medium) on which is recorded software program code (computer program) that realizes the functions of the above-described embodiments is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. The computer (or CPU or MPU) of the system or device then reads and executes the program code stored on the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

Claims

1. a management means for managing information for identifying an image and information for identifying a user in association with each other; a setting means for setting conditions for selecting an image for each user; a calculation means for calculating an evaluation value for each of the images managed by the management means; a selection means for selecting an image using the evaluation value calculated by the calculation means and the conditions set for each user by the setting means; An information processing device comprising:

2. the selection means selects images based on the conditions in order of the evaluation values ​​from among the images associated with each user by the management means; 2. The information processing device according to claim 1,

3. a determining means for determining an image whose evaluation value calculated by the calculating means satisfies a predetermined standard as a passing image; 2. The information processing device according to claim 1,

4. the selection means extracts images that have a photographing date and time close to that of the passed image from among the images that are not determined as the passed image, and selects images from the extracted images in descending order of the degree of difference from the passed image; 4. The information processing device according to claim 3,

5. a generating means for generating a composite image from the plurality of images selected by the selecting means, the generating means generates a composite image using at least one image not determined as a pass image by the determining means and at least one pass image having a photographing date and time close to that of the image not determined as a pass image by the determining means; 4. The information processing device according to claim 3,

6. a correction means for correcting an image, the correction means performs correction when the evaluation value of the image selected by the selection means satisfies a predetermined standard; 2. The information processing device according to claim 1,

7. a management step of managing information for identifying an image and information for identifying a user in association with each other; a setting step of setting conditions for selecting an image for each user; a calculation step of calculating an evaluation value for each of the images managed by the management step; a selection step of selecting an image using the evaluation value calculated in the calculation step and the conditions set for each user by the setting means; A processing method comprising:

8. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 6.

Images