Image candidate determination device, image candidate determination method, program for controlling the image candidate determination device, and recording medium storing the program

Abstract

To provide an image candidate determination device, a method thereof, a program thereof, and a storage medium storing the program, by which it assists which images to select for equalizing, as much as possible, the number of images in which each person is captured, as to images to be disclosed.SOLUTION: Upon inputting a plurality of images, the images including an identical person are grouped out. If the identical person is included in images equal to or greater than a maximum number of images to be disclosed (Step 61;YES), comprehensive image assessment values for the images including the identical person are calculated (Step 62). The images with the small comprehensive image assessment values are determined to undisclosed image candidates so as to become images with less than the maximum number of images (Step 63). An image extraction device and a method thereof for finding similar images intended by a user are provided.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an image candidate determination device, an image candidate determination method, a program for controlling the image candidate determination device, and a recording medium storing the program. [Background technology]

[0002] At events such as graduation ceremonies and athletic meets, many children are photographed and albums are created. For this reason, a school photo management server that can centrally manage all photos taken at schools and student works, and enable viewing and ordering, is being considered (Patent Document 1). In addition, when creating an album, if there are an extremely large number of photos taken of a specific individual, It is also considered to prevent the number of images from increasing or decreasing. For this purpose, an image is selected from a list of images, a person is extracted from the selected image, the number of times the extracted person appears is counted, and a warning is displayed if the number of times exceeds five. When the warning is displayed, the user selects images to delete from among the selected images (Patent Document 2). Furthermore, an image publishing device is also being considered that can automatically determine whether to make an image public or private according to a predetermined rule, even if the user does not set whether to make the image public or private (Patent Document 3). It is also being considered to photograph multiple subjects in a well-balanced manner (Patent Document 4). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-102715 A [Patent Document 2] JP 2010-57073 A [Patent Document 3] JP 2013-191035 A [Patent Document 4] Patent No. 5616819 Summary of the Invention [Problem to be solved by the invention]

[0004] While energetic or conspicuous children appear in many photos, shy or quiet children do not appear in many photos. In the case of the system described in Patent Document 1, selling photos in such a situation may result in complaints from parents that it is unfair. In the system described in Patent Document 2, when selecting images to delete from the selected images, it may be difficult to decide which images to select. In the system described in Patent Document 3, the decision to make an image public or private is made based on the results of image analysis, and certain people may appear more or less frequently. Furthermore, the system described in Patent Document 4 merely takes a balanced picture of multiple subjects when shooting.

[0005] The present invention aims to assist in determining which images to select in order to equalize as much as possible the number of images in which each person appears in the images to be made public. [Means for solving the problem]

[0006] The image candidate determination device of the present invention is characterized by comprising an image grouping means for receiving a plurality of images and grouping one or more images containing the same person, and an image candidate determination means for determining extracted image candidates or non-extracted image candidates from among the images grouped by the image grouping means based on an overall image evaluation value and a limit value for the number of images containing the same person to be extracted.

[0007] The grouping means may be any means that allows the image candidate determination device to recognize one or more images that include the same person. For example, it may be storing the images in a folder or setting a flag.

[0008] In addition, the image candidate determination device according to the present invention may be equipped with a processor that accepts a plurality of images, groups one or more images containing the same person, and determines extracted image candidates or non-extracted image candidates from among the grouped images based on an overall image evaluation value and a limit value for the number of images containing the same person to be extracted.

[0009] The present invention also provides an image candidate determination method suitable for an image candidate determination device. That is, this method is characterized in that an image grouping means receives a plurality of images and groups one or more images including the same person, and an image candidate determination means determines an extracted image candidate or a non-extracted image candidate from among the images grouped by the image grouping means based on a total image evaluation value and a limit value for the number of images including the same person to be extracted. do.

[0010] The present invention also provides a computer-readable program for controlling the computer of the image candidate determination device, and a recording medium storing the program.

[0011] The image processing device may further comprise an image candidate display control means for displaying the extracted image candidates or non-extracted image candidates determined by the image candidate determining means on a display screen.

[0012] The overall image evaluation value is calculated, for example, based on the person image evaluation value calculated for each person appearing in the image.

[0013] The person image evaluation value is calculated, for example, based on at least the evaluation value related to the face. In this case, a facial image part extraction means for extracting a facial image part from each of the multiple images is further provided, and the image grouping means groups the facial image parts extracted by the facial image part extraction means into groups of facial image parts of the same person, and the image candidate determination means determines extracted image candidates or non-extracted image candidates from the images grouped by the image grouping means based, for example, on the overall image evaluation value and a limit value for the number of images containing the same person to be extracted.

[0014] The image capturing apparatus may further comprise a correction means for correcting the person image evaluation value using a correction coefficient that increases the person image evaluation value as the number of images including the same person decreases.

[0015] The image processing device may further include an image determination means for determining an extracted image or a non-extracted image from among the extracted image candidates displayed under the control of the image candidate display control means, or for determining a non-extracted image or an extracted image from among the non-extracted image candidates displayed under the control of the image candidate display control means.

[0016] The image capturing apparatus may further comprise a notification means for notifying the number of extracted images and non-extracted images determined by the image determination means for each person included in the images.

[0017] In the case where a plurality of images are defined in advance as extracted images or non-extracted images for each person included in the images, the image determination means may further include a notification means for updating and notifying the number of extracted images and non-extracted images for each person included in the images in response to the image determination means determining that an image is extracted or non-extracted.

[0018] The image candidate determining means may further include an image candidate stopping means for removing the non-extracted image candidate or the extracted image candidate from the non-extracted image candidates or the extracted image candidates when the number of extracted images containing a specific person falls outside a limit value in response to the non-extracted image candidate determined by the image candidate determining means being determined as a non-extracted image or the extracted image candidate being determined as an extracted image.

[0019] The image grouping means, for example, for adults, groups images of non-identical people by regarding them as the same person.

[0020] The image capturing apparatus may further comprise a first evaluation value calculation means for calculating a lower person image evaluation value as the number of people included in the image increases.

[0021] The image candidate determination means may, for example, determine extracted image candidates or non-extracted image candidates from among the images grouped by the image grouping means based on a total image evaluation value and a limit value for the number of images containing the same person to be extracted, so that the number of images containing the same person to be extracted falls within the limit value.

[0022] The image processing device may further include a person detection means for detecting a person in which the number of identical persons contained in one or more images grouped by the image grouping means is below a threshold, and a photography instruction means for issuing instructions to photograph a person detected by the person detection means. Effect of the Invention

[0023] According to this invention, a plurality of images are accepted, and one or more images containing the same person are grouped. Based on the overall image evaluation value and the limit value of the number of images containing the same person to be extracted (the limit value means an upper limit value, a lower limit value, etc.), an extracted image candidate (an extracted image candidate means an image candidate suitable for the user to extract from among the images) or a non-extracted image candidate (a non-extracted image candidate means an image candidate suitable for the user not to extract from among the images) is determined from among the grouped images. Since an extracted image candidate or a non-extracted image candidate is determined, the user knows which image is preferable to extract, and which image is preferable not to extract. Therefore, the user can specify an image to be extracted from among images considered preferable to be extracted, and specify an image not to be extracted from among images considered undesirable to be extracted. [Brief description of the drawings]

[0024] [Figure 1] 2 is a block diagram showing an electrical configuration of the image candidate determination device; [Diagram 2] 13 is a flowchart showing a processing procedure of the image candidate determination device. [Diagram 3] 13 is a flowchart showing a processing procedure of the image candidate determination device. [Figure 4]13 is a flowchart showing a processing procedure of the image candidate determination device. [Diagram 5] 13 is a flowchart showing a processing procedure of the image candidate determination device. [Figure 6] 13 is a flowchart showing a processing procedure of the image candidate determination device. [Figure 7] 13 is a flowchart showing a processing procedure of the image candidate determination device. [Figure 8] 2 is an example of an image display window. [Figure 9] 2 is an example of an image display window. [Figure 10] 2 is an example of an image display window. [Figure 11] 2 is an example of an image display window. [Figure 12] 2 is an example of an image display window. [Figure 13] 2 is an example of an image display window. [Figure 14] 2 is an example of an image display window. [Figure 15] 2 is an example of an image display window. [Figure 16] 2 is an example of an image display window. [Figure 17] 2 is an example of an image display window. [Figure 18] 2 is an example of an image display window. [Figure 19] 13 is an example of an extraction number input window. [Figure 20] 13 is an example of a shooting instruction window. [Figure 21] 2 is an example of an image display window. [Figure 22] 2 is an example of an image display window. [Figure 23] 2 is an example of an image display window. [Figure 24] 2 is an example of an image display window. [Diagram 25] 2 is an example of an image display window. [Figure 26] 1 is an example of an image. [Figure 27] 2 is an example of an image display window. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] FIG. 1 is a block diagram showing the electrical configuration of an image candidate determination device 1 according to an embodiment of the present invention.

[0026] The entire operation of the image candidate determination device 1 is controlled by a CPU (Central Processing Unit) 2. It is summarized.

[0027] The image candidate determination device 1 includes a display device 3 that displays images and other information on a display screen, and a communication device 4 that connects to the Internet or other networks and communicates with devices other than the image candidate determination device 1. The image candidate determination device 1 also includes a hard disk 5, a hard disk drive 6 that accesses the hard disk 5, a memory 7 that stores data, etc., and a keyboard 8 and a mouse 9 for inputting commands, etc. The image candidate determination device 1 also includes a compact disk drive 10 that accesses a compact disk 11, and a memory card reader / writer 12 that writes data to a memory card 13 and reads data recorded on the memory card 13.

[0028] An operation program of the image candidate determination device 1, which will be described later, is received by the communication device 4 via the Internet. The received operation program is installed in the image candidate determination device 1. The operation program may be received by the image candidate determination device 1 via a network such as the Internet and not installed in the image candidate determination device 1, but recorded on a portable recording medium such as a compact disc 11 and read from the portable recording medium. In that case, the operation program read from the portable recording medium is installed in the image candidate determination device 1. It goes without saying that the operation program is readable by the computer (CPU 2) of the image candidate determination device 1.

[0029] 2 to 7 are flowcharts showing the processing steps of the image candidate determination device 1. FIG.

[0030] This embodiment aims to eliminate as much as possible the imbalance that occurs when a large number of images including people are taken and made public, such as when there are more or less images of a particular person.

[0031] In this embodiment, the name of the person to be photographed is registered in the image candidate determination device 1 by a user of the image candidate determination device 1. In the embodiment described below, a large number of images are taken at a kindergarten graduation ceremony. In this embodiment, the names of the children to be photographed are registered in advance in the image candidate determination device 1 by a user (e.g., a kindergarten teacher).

[0032] When the memory card 13 storing image data representing a plurality of images is loaded into the image candidate determination device 1, the image data representing a plurality of images stored in the memory card 13 is read by the memory card reader / writer 12 (step 21 in FIG. 2). The face image portion included in the image represented by the read image data is extracted by the CPU 2 (face image portion extraction means) (step 22 in FIG. 2). Furthermore, based on the extracted face image portion, one or more images including the same person are grouped by the CPU 2 (image grouping means) (step 23 in FIG. 2). The same image may belong to different groups, or the same image may belong to only one group. Furthermore, the extracted face image portion is also grouped by the CPU 2 (image grouping means) into face image portions of the same person (step 24 in FIG. 2).

[0033] An image display window 80 shown in Fig. 8 is displayed on the display screen of the display device 3. The image display window 80 includes an image management tab 91, a face classification tab 101, and a public settings tab 102, and initially, the image management tab 91 is in the clicked state. In the image display window 80, a number of images (thumbnail images) represented by image data read from the memory card 13 are displayed in a list in the image display area 90 (step 25 in Fig. 3).

[0034] 8, an "Edit" character string 81 is displayed in an image display window 80, and an event management area 82 is formed below this "Edit" character string 81. In this event management area 82, the character string "Graduation Ceremony" is displayed below the character string "Kindergarten." Formed below the event management area 82 are an event addition button 83 displaying the character string "Add Event" and an event deletion button 84 displaying the character string "Delete Event."

[0035] An event called "graduation ceremony" has already been generated in the event management area 82, and the image represented by the image data read from the memory card 13 will be managed as the event called "graduation ceremony." Specifically, when the character string "kindergarten" is clicked and then the event addition button 83 is clicked, an event input window appears on the image display window 80, and the user inputs a new event using the event input window. If the user then inputs the character string "graduation ceremony" using the keyboard 8, the character string "graduation ceremony" will be displayed below the character string "kindergarten."

[0036] When the character string "graduation ceremony" is clicked, an image represented by image data read from memory card 13 is dragged and dropped into image display area 90, and the dragged and dropped image is managed by the event "graduation ceremony" (this state is shown in FIG. 8). In FIG. 8, the number "360" is displayed in parentheses to the right of the character string "graduation ceremony" displayed in event management area 82, and 360 images are managed as images for the "graduation ceremony" event. When the character string displayed in event management area 82 is clicked and then event delete button 84 is clicked, the event identified by the clicked character string and the images managed by that event are deleted.

[0037] Below the image management tab 91, a left arrow button 92 and a right arrow button 93 are formed. To the right of the right arrow button 93, an image identification number 94 is displayed to identify the specified image. The number represented by this image identification number 94 represents the specified image. In the example shown in FIG. 8, the image identification number 94 displays the character string "11 / 360", indicating that the 11th image out of 360 images managed as the "graduation ceremony" event is specified. The image identification number 94 is expressed as a fraction, with the denominator indicating the number of images managed by the event and the numerator identifying the specified image in numerical order. When the left arrow button 92 is clicked once, the numerator of the image identification number 94 decreases by one, and when the right arrow button 93 is clicked once, the numerator of the image identification number 94 increases by one. Preferably, the image identified by the image identification number 94 is surrounded by a frame, for example, so that the user can see the image that is identified.

[0038] The character string "list display" 95 is displayed to the right of the image identification number 94. When the check box to the left of the character string "list display" 95 is checked (it is checked by default), multiple images are displayed in a list in the image display area 90, as shown in Figure 8. When the check box to the left of the character string "list display" 95 is unchecked, a single image identified by the image identification number 94 is displayed in the image display area 90 (see Figure 9).

[0039] A publication status and an image file name are displayed beneath each of the multiple images displayed in the image display area 90. In this embodiment, there are three types of publication status: "public", "private" and "pending". Among the publication statuses, "public" indicates a state in which an image is public, "private" indicates a state in which an image is private, and "pending" indicates a state in which neither "public" nor "private" has been determined. For example, a user may create an album using only "public" images, or print only "public" images.

[0040] An up arrow button 97, a slide bar 98, and a down arrow button 99 are formed at the right end of the image display area 90. When the up arrow button 97 is clicked or the slide bar 98 is dragged upward, the image display area 90 is scrolled upward, and when the down arrow button 99 is clicked or the slide bar 98 is dragged downward, the image display area 90 is scrolled downward.

[0041] When any of the images displayed in the image display area 90 is clicked (YES in step 26 in FIG. 3), the public status of the clicked image is changed in response to the click (step 27 in FIG. 3). In the initial state, the public status of all images is "pending", and when an image with a public status of "pending" is clicked once, the public status becomes "public", and when an image with a public status of "public" is clicked once more, the public status becomes "private". When an image with a public status of "private" is clicked once more, the public status becomes "pending". In this way, the public status changes in sequence each time an image is clicked. However, it is also possible to remove "pending" from the public status and set it to "public" and "private". In that case, in the initial state, the public status of all images is "public" or "private" (predetermined as an extracted image or a non-extracted image for each person included in the image), and the public status switches to "public" or "private" in response to clicking the image (determination of whether it is an extracted image or a non-extracted image).

[0042] When the left arrow button 92 or the right arrow button 93 is clicked (YES in step 28 of Figure 3), if the left arrow button 92 is clicked, the numerator of the image identification number 94 is decreased by one, and if the right arrow button 93 is clicked, the numerator of the image identification number 94 is increased by one, and the image specification changes (step 29 of Figure 3).

[0043] When the check box of the list display character string 95 is unchecked (YES in step 30 in FIG. 3), the image identified by the image identification number 94 is enlarged and displayed in the image display area 90 (step 31 in FIG. 3).

[0044] FIG. 9 shows an image being displayed in an enlarged manner in an image display area 90. In FIG.

[0045] In FIG. 9, the same components as those shown in FIG. 8 are denoted by the same reference numerals and the description thereof will be omitted.

[0046] A character string 96 for "display face information" is displayed to the right of the character string 95 for "list display." A check box is also formed to the left of the character string 96 for "display face information."

[0047] In the example shown in Fig. 9, it is assumed that the character string "11 / 360" is displayed in image identification number 94 in Fig. 8, and the check box for "List display" character string 95 is unchecked. Then, the image with the image file name "0011.jpg" identified by the character string "11 / 360" is enlarged and displayed in image display area 90 shown in Fig. 9. Under the image enlarged and displayed in image display area 90, the publication status (pending) and the image file name (0011.jpg) are displayed.

[0048] When the check box to the left of the character string 96 of "display face information" is checked (YES in step 32 in FIG. 3), a face frame that surrounds the face image portion included in the image displayed in the image display area 90 is displayed (step 33 in FIG. 3). The display of this face frame is performed based on the extraction process of the face image portion that has already been performed (step 22 in FIG. 2). The name of the subject (in this case, a kindergarten teacher) of the image candidate determination device 1 is input, and the user specifies who the face image corresponds to, as will be described in detail later (see FIG. 11, FIG. 12, etc.). The image of the face of the same person as the face with the name specified by the user is found in the image candidate determination device 1, and the image of the face that has been found is also associated with the name (for example, the name of the face is stored in the header of the image file in correspondence with the information on the position of the face). If the image of the face has already been associated with the name by such user processing, the name of the person with that face is also displayed in correspondence with the face frame.

[0049] FIG. 10 shows that the check box to the left of the face information display string 96 is checked, and the face image portion included in the image displayed in the image display area 90 is surrounded by a face frame.

[0050] The faces surrounded by face frames F1, F4, F3, and F2 are people (kindergarteners) with the names "Sato Taro," "Tanaka Jiro," "Ito Hanako," and "Suzuki Haruko," respectively, and their names are displayed near (at the top of) the face frames F1, F4, F3, and F2, respectively. The face surrounded by face frame F has not been associated with a name by the user (unclassified), so the name is not displayed. By checking the check box to the left of the face information display string 96, the names of the people included in the image are displayed, allowing the user to confirm the names of the people included in the image.

[0051] When the check box to the left of the face information display string 96 is unchecked (YES in step 34 in Figure 3), the faces in the image are surrounded by a frame as shown in Figure 10, and for faces that have names associated with them, the name is displayed, but the state is changed back to one in which the name is erased as shown in Figure 9 (step 35 in Figure 3).

[0052] When the face classification tab 101 is clicked by the user (YES in step 36 in FIG. 3), the image display window 80 displays a list of the face image portions as shown in FIG 11 (step 38 in FIG. 4). When the public setting tab 102 is clicked by the user (YES in step 37 in FIG. 3), the image display window 80 displays a public / private setting image for setting whether the image is public or private as shown in FIG 13 (step 49 in FIG. 5).

[0053] When the face classification tab 101 is clicked by the user, an image display window 80 shown in Fig. 11 is displayed on the display screen of the display device 3. In Fig. 11, the same components as those shown in Figs. 8 to 10 are given the same reference numerals and their explanations are omitted.

[0054] In the image display area 90, a list of the facial image portions extracted in the processing of step 22 in Fig. 2 is displayed. The facial image portions are displayed in the image display area 90 in the order of the file names of the images in which the facial image portions are included, but this is not necessarily required.

[0055] A subject list area 110 is formed on the left side of the image display area 90. In the subject list area 110, a list of names that have been registered in advance in the image candidate determination device 1 by the user is displayed, as described above. In addition, the number of images in which a person with that name appears is also displayed corresponding to that name. The user can select (click) the names displayed in the subject list area 110.

[0056] At the top of the image display area 90, there are displayed a face classification button 111 displaying the character string "Face Classification", a character string "Show Selected Children" 112, a character string "Show Non-Selected Children" 113, and a character string "Show Unclassified Faces" 114. A check box is formed to the left of each of the character string "Show Selected Children" 112, the character string "Show Non-Selected Children" 113, and the character string "Show Unclassified Faces" 114.

[0057] Face When the similar button 111 is clicked, image portions of the same person's face are displayed together (adjacent) in the image display area 90. When the check box to the left of the character string 112 "Display selected child" is checked, the facial image of the person with the selected name from among the names displayed in the subject list area 110 is displayed in the image display area 90. For example, when "Sato Taro" is selected from among the names displayed in the subject list area 110 and the check box to the left of the character string 112 "Display selected child" is checked, the facial image of "Sato Taro" is displayed in the image display area 90.

[0058] When the check box to the left of the character string 113 "Show non-selected children" is checked, facial images of people other than the selected name among the names displayed in the subject list area 110 are displayed in the image display area 90. For example, when "Sato Taro" is selected among the names displayed in the subject list area 110 and the check box to the left of the character string 113 "Show non-selected children" is checked, facial images other than "Sato Taro" are displayed in the image display area 90. When the check box to the left of the character string 114 "Show unclassified faces" is checked, facial images of unclassified people (whose names and facial images have not been associated by the user as described above) are displayed in the image display area 90.

[0059] When face classification tab 101 is clicked and the image portion of a face displayed in image display area 90 is displayed, a name can be associated with that face. The face is associated with a name by storing name information in association with position information of the image portion of the face in the header of the image file representing the image including the image portion of the face. For face image portions displayed in image display area 90 that are not associated with a name, only the file name of the image including that image portion of the face is displayed. For face image portions associated with a name, the file name of the image including that image portion of the face and the name are displayed.

[0060] When associating a name with the face image portion displayed in image display area 90, the user clicks on one of the names displayed in subject list area 110 (YES in step 39 in FIG. 4), and then clicks on the face image portion of that name (YES in step 40 in FIG. 4). Then, as described above, name information is stored in the header of the image file that contains that face image portion along with position information of the face image portion. The selected name is displayed above the clicked face image portion along with the file name of the image that contains that face image portion (step 41 in FIG. 4).

[0061] Next, the first image evaluation value associated with the name is calculated by the CPU 2 (step 42 in FIG. 4, the first image evaluation value corresponds to the person image evaluation value in the present invention). This first image evaluation value is an evaluation value for the person with the associated name. When an image contains two people, the first image evaluation value for one person is different from the first image evaluation value for the other person, even if they are the same image.

[0062] The first image evaluation value is calculated according to evaluation parameters such as the number of people included in the image, the size of the faces included in the image, etc. Table 1 is an example of a number of people score table according to the number of people included in the image.

[0063] [Table 1]

[0064] The more people in an image, not too few and not too many, the more balanced the image is, and the more prominent the people in the image are, and the higher the value of the image for that person is. For this reason, the number of people score is calculated by CPU2 (first image evaluation value calculation means) so that if the number of people in an image is about 5, the number of people score is high, and if the number of people in an image is small or large, the number of people score is low. However, it is also possible to set the number of people score higher the fewer the number of people in an image, and the higher the score the more the number of people in an image.

[0065] Table 2 is an example of a face size score table based on the size of the face contained in the image. In Table 2, the face size indicates the size of the face relative to the image. For example, a face size of "1 / 15" indicates that the face size is 1 / 15 of the image size. When the face is surrounded by a rectangular frame, the size may be determined by using the number of horizontal pixels of the image and the number of horizontal pixels of the face image, or the number of vertical pixels of the image and the number of vertical pixels of the face image, or the number of pixels in the entire image area and the number of pixels in the face area.

[0066] [Table 2]

[0067] If the size of the face of a person in an image is too large or too small, the image will not be well-balanced, so the face size score is set to be highest at an appropriate ratio (1 / 6 in this case), and the face size score is set to be lower the further away from that ratio.

[0068] The facial score is calculated using evaluation parameters such as the number of people included in the image, the size of the faces included in the image, the degree of focus of the image parts of the faces included in the image, the degree of opening of the eyes in the faces included in the image, the facial expressions included in the image, etc. The facial score is calculated using these parameters. The facial expression score is set in accordance with the evaluation parameters so that the more in-focus the image parts of the face are, the higher the facial expression score will be, the more the eyes are properly open, and the more smiling the facial expression is, the higher the facial expression score will be.

[0069] In this embodiment, the raw score of the first image evaluation value is calculated by adding up the number of people score, the face size score, and the facial expression score.

[0070] In addition, an image quality score may be defined such that the better the overall image quality of an image (for example, the better the focus), the higher the score will be, and this may be used to calculate the raw score.

[0071] Furthermore, in this embodiment, the first image evaluation value is corrected according to the number of images that a specific person appears in. For this reason, weighting is performed so that the fewer the number of images that a specific person appears in, the higher the first image evaluation value for that person, and the more the number of images that a specific person appears in, the lower the first image evaluation value for that person.

[0072] Table 3 is an example of a weighting table. The maximum value in Table 3 is the maximum number of images of the same person among the images to be published, for example "6".

[0073] [Table 3]

[0074] A first image evaluation value for each person is calculated for each image.

[0075] Table 4 shows the first image evaluation value for each image of "Taro Sato."

[0076] [Table 4]

[0077] Table 4 shows the first image evaluation values ​​obtained from "Sato Taro" for six images with the file names "0022.jpg", "0019.jpg", "0021.jpg", "0013.jpg", "0020.jpg", and "0018.jpg". "Sato Taro" appears in these six images. For "Sato Taro", for example, the raw score of the first image evaluation value for the file name "0022.jpg" is "40", and the raw scores of the other first image evaluation values ​​are also shown. These raw scores are normalized, and the normalized raw scores are weighted to obtain the respective first image evaluation values ​​for "Sato Taro". Since "Sato Taro" appears in six images (the same number of images as the maximum value specified in the weighting table), the weighting coefficient is the maximum value "×1", and the first image evaluation value after correction using the weighting coefficient is the same as the normalized raw score. Of course, if the weighting coefficient is not "×1", the first image evaluation value after correction using the weighting coefficient will differ from the normalized raw score. In this way, it is preferable to correct the first image evaluation value by CPU2 (correction means) using a weighting coefficient (correction coefficient) that increases the first image evaluation value the fewer the number of images that include the same person, and decreases the first image evaluation value the more the number of images that include the same person.

[0078] An image evaluation value for a person is calculated for each image, not only for "Taro Sato" but also for people with other names.

[0079] Table 5 shows the first image evaluation value for the person appearing in six images featuring "Taro Sato," and also shows the second image evaluation value, which is the overall image evaluation value for an image (the second image evaluation value corresponds to the overall image evaluation value of the present invention).

[0080] [Table 5]

[0081] It can be seen that the image with the file name "0022.jpg" contains "Suzuki Haruko" in addition to "Sato Taro", the image with the file name "0019.jpg" contains "Ito Hanako" in addition to "Sato Taro", the image with the file name "0021.jpg" contains "Suzuki Haruko" and "Tanaka Jiro" in addition to "Sato Taro", the image with the file name "0013.jpg" contains "Ito Hanako" in addition to "Sato Taro", the image with the file name "0020.jpg" contains "Suzuki Haruko", "Ito Hanako", "Tanaka Jiro" and "Watanabe Ichiro" in addition to "Sato Taro", and the image with the file name "0018.jpg" contains "Watanabe Ichiro" in addition to "Sato Taro".

[0082] For example, the first image evaluation value of the image with the image file name "0022.jpg" is "20" for "Sato Taro" but "200" for "Suzuki Haruko," and the second image evaluation value, which is the overall image evaluation value of the image with the image file name "0022.jpg," is "220," which is the sum of the first image evaluation value "20" for "Sato Taro" and the first image evaluation value "200" for "Suzuki Haruko." Similarly, for other images, it can be seen that the second image evaluation value of the image with the image file name "0019.jpg" is "62.5," the second image evaluation value of the image with the image file name "0021.jpg" is "367," the second image evaluation value of the image with the image file name "0013.jpg" is "63," the second image evaluation value of the image with the image file name "0020.jpg" is "1023," and the second image evaluation value of the image with the image file name "0018.jpg" is "414.5." From these, it can be seen that, of these six images, the image with the image file name "0020.jpg" having the highest second image evaluation value has the highest importance as an image, and the image with the image file name "0019.jpg" having the lowest second image evaluation value has the lowest importance as an image. In this way, it is preferable to add up the first image evaluation value, which is different for each person included in the image, for each image, calculate the sum obtained by adding up by CPU2 (second image evaluation value calculation means), and use the sum as the second image evaluation value. Note that the second image evaluation value may be calculated by any method as long as the second image evaluation value is higher as the first image evaluation value is higher based on the first image evaluation value calculated for each person appearing in the image, and may be calculated by summing up the first image evaluation values ​​after weighting. For example, for a group of images from an event in which all the children participate, the first image evaluation value may be weighted more heavily for children in higher grades (and therefore lighter for children in lower grades) in consideration of the number of future photo opportunities, and then the second image evaluation value may be calculated by taking the sum.

[0083] In this embodiment, when deciding which of these six images should not be made public, the images with the lowest second image evaluation value are decided as the private image candidates (non-extracted image candidates), and when deciding which images should be made public, the images with the highest second image evaluation value are decided as the public image candidates (extracted image candidates).When an image is changed to public or private, the first image evaluation value and the second image evaluation value are calculated again.

[0084] In the above embodiment, when an image contains multiple facial image portions, the sum of the evaluation values ​​of those facial image portions may be calculated, and the calculated sum may be used as the overall image evaluation value. For example, with reference to Table 5, assume that an image with the image file name "0022.jpg" contains a facial image portion with the name "Sato Taro" and the evaluation value of that facial image portion is "20." Also assume that the image with the image file name "0022.jpg" contains a facial image portion with the name "Suzuki Haruko" and the evaluation value of that facial image portion is "200." The sum of the evaluation values ​​of those facial image portions may be calculated, and the resulting sum of the evaluation values, "220," may be used as the overall image evaluation value.

[0085] When the left arrow button 92 or the right arrow button 93 is clicked (YES in step 43 in FIG. 4), the image specification changes in response to the click of the left arrow button 92 or the right arrow button 93 (step 44 in FIG. 4). The number in the image identification number 94 also changes.

[0086] When the face classification button 111 is clicked (YES in step 45 in FIG. 4), the face image portions are displayed classified for each person as shown in FIG. 12 (step 46 in FIG. 4).

[0087] 12, facial image group G1 includes a facial image portion included in the image with the image file name "0021.jpg", a facial image portion included in the image with the image file name "0019.jpg", a facial image portion included in the image with the image file name "0020.jpg", and a facial image portion included in the image with the image file name "0018.jpg". Facial image group G1 also includes a facial image portion included in the image with the image file name "0022.jpg" and a facial image portion included in the image with the image file name "0013.jpg", but these are not visible in FIG. 12. Facial image group G1 includes the facial image portion of "Sato Taro".

[0088] Furthermore, facial image group G2 includes a facial image portion included in the image with the image file name "0027.jpg", a facial image portion included in the image with the image file name "0014.jpg", and a facial image portion included in the image with the image file name "0028.jpg". The facial image portions included in facial image group G2 also represent the same person. Similarly, facial image group G3 includes a facial image portion included in the image with the image file name "0027.jpg", and a facial image portion included in the image with the image file name "0014.jpg". The facial image portions included in facial image group G3 also represent the same person.

[0089] In the image display window 80 in the state shown in Fig. 12, similarly to the image display window 80 in the state shown in Fig. 11, when a face image portion is clicked (step 40 in Fig. 4) after a name displayed in the subject list area 110 is clicked (YES in step 39 in Fig. 4) in the image display window 80 in the state shown in Fig. 12, the clicked face image portion is associated with the name (step 41 in Fig. 4). In addition, in the case of classification into face image groups as shown in Fig. 12, when a face image portion included in any face image group or an area other than a face image portion within the area of ​​any face image group is clicked after a name displayed in the subject list area 110 is clicked (YES in step 39 in Fig. 4), all face image portions included in the face image group including the clicked portion may be associated with the name. It becomes relatively easy to associate all face image portions included in the face image group with names.

[0090] Moreover, when the image management tab 91 is clicked (YES in step 47 in FIG. 4), the process proceeds to the process shown in FIG. 3. Furthermore, when the public setting tab 102 is clicked (YES in step 48 in FIG. 4), a public / private setting image as shown in FIG. 13 is displayed in the image display window 80 (step 49 in FIG. 5).

[0091] In FIG. 13, the same components as those shown in FIGS. 8 to 12 are designated by the same reference numerals and their description will be omitted.

[0092] The image display area 90 displays the image that was identified by the image identification number 94 before the public settings tab 102 was clicked. In FIG. 13, it is assumed that an image with the image file name "0022.jpg" was identified by the image identification number. The image with the image file name "0022.jpg" is displayed in the image display area 90. The image with the image file name "0022.jpg" includes "Taro Sato" and "Haruko Suzuki". The face of "Taro Sato" is surrounded by a face frame F1, and the name of "Taro Sato" is displayed above the face frame F1. The face of "Haruko Suzuki" is surrounded by a face frame F2, and the name of "Haruko Suzuki" is displayed above the face frame F2.

[0093] A public / private information display area 120 is formed on the left side of the image display area 90. In the public / private information display area 120, the name of the subject is displayed. Check boxes 121-126, 129, etc. are formed to the left of the subject's name. To the right of the check box 129, the name of the person is displayed as "Teacher / Parent." In the case of a kindergarten graduation ceremony, the children are the main characters, and the teachers and parents are often supporting characters, so they are treated as the same person without being separated by person. In this way, adults such as teachers and parents may be considered as the same person and grouped by the CPU 2 (image grouping means) for each image. Of course, teachers and parents may not be treated as the same person and may be treated separately by person. The check box of the name corresponding to the person (the person whose name is displayed) included in the image displayed in the image display area 90 is checked. In the example shown in FIG. 13, the image displayed in image display area 90 includes "Taro Sato" and "Haruko Suzuki," and the check box 121 to the left of the name "Taro Sato" and the check box 122 to the left of the name "Haruko Suzuki" are checked.

[0094] In addition, to the right of the subject's name, the number of images containing that subject that are set to "private" (number of non-extracted images), the number of images that are set to "pending," and the number that are set to "public" (number of extracted images) are displayed for each person in the subject list area 110 of the image display window 80 that is displayed on the display device 3 (notification means) under the control of the CPU 2 (display control means).

[0095] When an image displayed in the image display area 90 is clicked (step 50 in FIG. 5: YES), the public status of the clicked image changes in response to the click, similar to the process in step 27 in FIG. 3 (step 51 in FIG. 5). When the image is clicked once while its public status is "pending", its public status becomes "public". The number of "public" images for each of the names "Taro Sato" and "Haruko Suzuki" in the public / private information display area 120 is changed from "0" to "1". In addition, the number of "pending" images for the name "Taro Sato" becomes from "6" to "5", and the number of "pending" images for the name "Haruko Suzuki" becomes from "3" to "2". When the image is clicked once more, its public status becomes "private". The number of "public" images for each of the names "Taro Sato" and "Haruko Suzuki" in the public / private information display area 120 is returned from "1" to "0", and the number of "private" images for each of them is changed from "0" to "1".

[0096] If the public status does not include "pending" and there are two types, "private" and "public," the public status switches between "private" and "public" each time the image displayed in image display area 90 is clicked once, and the number of "private" images and the number of "public" images displayed in public / private information display area 120 are updated by CPU 2 for each person included in the image, and are displayed (notified) in public / private information display area 120 of image display window 80 displayed on display device 3 (notification means).

[0097] When either the left arrow button 92 or the right arrow button 93 is clicked (YES in step 52 in FIG. 5), the image identification number 94 decreases or increases, and the image designation changes (step 53 in FIG. 5). In FIG. 13, one of the six images including "Sato Taro" is displayed in the image display window 80, and the image identification number 94 is "1 / 6". This is because the public setting tab 102 was clicked while "Sato Taro" was selected in the subject list area 110 included in the image display window 80 shown in FIG. 12. The images displayed in the image display area 90 include "Sato Taro" and "Suzuki Haruko". The face of "Sato Taro" is surrounded by a face frame F1, and the name "Sato Taro" is displayed above the face frame F1. Similarly, the face of "Suzuki Haruko" is surrounded by a face frame F2, and the name "Suzuki Haruko" is displayed above the face frame F2.

[0098] When the right arrow button 93 is clicked while the image display window 80 shown in FIG. 13 is displayed, the image display window 80 shown in FIG.

[0099] Referring to FIG. 14, the image displayed in the image display area 90 includes "Sato Taro" and "Ito Hanako", as well as a person whose name has not been determined (the face image portion has not been associated with a name using FIG. 11, FIG. 12, etc.). The face of "Sato Taro" is surrounded by a face frame F1, and the name "Sato Taro" is displayed at the top of the face frame F1. Similarly, the face of "Ito Hanako" is surrounded by a face frame F3, and the name "Ito Hanako" is displayed at the top of the face frame F3. The face of an unclassified person whose name has not been determined is surrounded by a face frame F. The character string "Unclassified" is displayed at the top of the unclassified face frame F so that it is clear that the face image portion has not been associated with a name. Also, the image identification number 94 has changed from "1 / 6" to "2 / 6" so that it is clear that this is the second image out of six images that include "Sato Taro".

[0100] When the right arrow button 93 is clicked while the image display window 80 shown in FIG. 14 is displayed, the image display window 80 shown in FIG.

[0101] Referring to FIG. 15, the image displayed in the image display area 90 includes "Sato Taro", "Suzuki Haruko", "Tanaka Jiro", and a person whose name has not been determined (the image portion of the face has not been associated with a name using FIG. 11, FIG. 12, etc.). "Sato Taro"'s face is surrounded by a face frame F1, and the name "Sato Taro" is displayed above the face frame F1. Similarly, "Suzuki Haruko"'s face is surrounded by a face frame F2, and the name "Suzuki Haruko" is displayed above the face frame F2, and "Tanaka Jiro"'s face is surrounded by a face frame F4, and the name "Tanaka Jiro" is displayed above the face frame F4. The face of an unclassified person whose name has not been determined is surrounded by a face frame F. The character string "Unclassified" is displayed above the unclassified face frame F. Additionally, the image identification number 94 has changed from "2 / 6" to "3 / 6," indicating that this is the third image out of six images that include "Taro Sato."

[0102] When the right arrow button 93 is clicked while the image display window 80 shown in FIG. 15 is displayed, the image display window 80 shown in FIG.

[0103] Referring to FIG. 16, the image displayed in the image display area 90 includes "Sato Taro" and "Ito Hanako", as well as a person whose name has not been determined (the image portion of the face has not been associated with a name using FIG. 11, FIG. 12, etc.). "Sato Taro"'s face is surrounded by a face frame F1, and the name "Sato Taro" is displayed above the face frame F1. Similarly, "Ito Hanako"'s face is surrounded by a face frame F3, and the face of an unclassified person whose name has not been determined is surrounded by a face frame F. The character string "Unclassified" is displayed above the unclassified face frame F. Also, the image identification number 94 has changed from "3 / 6" to "4 / 6" to show that this is the fourth image out of six images that include "Sato Taro".

[0104] When the right arrow button 93 is clicked while the image display window 80 shown in FIG. 16 is displayed, the image display window 80 shown in FIG.

[0105] Referring to FIG. 17, the image displayed in the image display area 90 includes "Sato Taro", "Suzuki Haruko", "Ito Hanako", "Tanaka Jiro", and "Watanabe Ichiro", as well as a person whose name has not been determined (the face image portion has not been associated with a name using FIG. 11, FIG. 12, etc.). The face of "Sato Taro" is surrounded by a face frame F1, and the name "Sato Taro" is displayed above the face frame F1. Similarly, the face of "Suzuki Haruko" is surrounded by a face frame F2, the face of "Ito Hanako" is surrounded by a face frame F3, the face of "Tanaka Jiro" is surrounded by a face frame F4, and the face of "Watanabe Ichiro" is surrounded by a face frame F5. The face of an unclassified person whose name has not been determined is surrounded by a face frame F, and the character string "Unclassified" is displayed above the unclassified face frame F. Additionally, the image identification number 94 has changed from "4 / 6" to "5 / 6," indicating that this is the fifth image out of six images that include "Taro Sato."

[0106] When the right arrow button 93 is clicked while the image display window 80 shown in FIG. 17 is displayed, the image display window 80 shown in FIG.

[0107] Referring to FIG. 18, the images displayed in the image display area 90 include "Sato Taro", "Watanabe Ichiro", and an adult or guardian. The images displayed in the image display area 90 also include a person whose name has not been determined (the image portion of the face has not been associated with a name using FIG. 11, FIG. 12, etc.). The face of "Sato Taro" is surrounded by a face frame F1, and the name "Sato Taro" is displayed above the face frame F1. Similarly, the face of "Watanabe Ichiro" is surrounded by a face frame F5, and the face of the adult or guardian is surrounded by a face frame F6. The face of an unclassified person whose name has not been determined is surrounded by a face frame F, and the character string "Unclassified" is displayed above the unclassified face frame F. Also, the image identification number 94 has changed from "5 / 6" to "6 / 6" so that it can be seen that this is the sixth image out of six images that include "Sato Taro".

[0108] When the left arrow button 92 is clicked while the image display window 80 shown in Figure 18 is displayed, the image that was displayed before the right arrow button 93 was clicked is displayed in the image display area 90 of the image display window 80 each time it is clicked.

[0109] As shown in Figures 13 to 18, each time the right arrow button 93 or the left arrow button 92 is clicked, people whose name includes "Taro Sato" can be displayed in sequence in the image display area 90. Each time any image displayed in the image display area 90 of the image display window 80 shown in Figures 13 to 18 is clicked, the publication status changes in sequence to "public", "private" or "pending", allowing the user to specify whether the image displayed in the image display area 90 should be public or private. Similarly, for images that do not include a person whose name includes "Taro Sato", the user can specify whether the image should be public or private.

[0110] For example, as shown in Figs. 13 to 18, when the public setting tab 102 is clicked, if a name displayed in the subject list area 110 is clicked (selected), an image including the person with the clicked name is displayed in the image display area 90. For example, when "Suzuki Haruko" is clicked among the names displayed in the subject list area 110, the first image among the images including "Suzuki Haruko" is displayed in the image display area 90. Since there are three images including "Suzuki Haruko" (images in which the name "Suzuki Haruko" is associated with the image) (the total number of images is three, since the number of private images, the number of reserved images, and the number of public images displayed corresponding to the name "Suzuki Haruko" in the subject list area 110 is three), when the right arrow button 93 is clicked while the image of "Suzuki Haruko" is displayed in the image display area 90, the three images are displayed in order in the image display area 90.

[0111] When the number of sheets input button 130 is clicked (YES in step 54 in FIG. 5), while the image display window 80 is displayed (the image display window 80 may disappear temporarily), the number of sheets to be extracted input window 140 shown in FIG. 19 is displayed on the display screen of the display device 3 (step 55 in FIG. 5).

[0112] Referring to FIG. 19, the extraction number input window 140 is for inputting the minimum and maximum numbers of images containing the same person to be made public (the minimum and maximum numbers of images to be made public are limit values ​​for the number of images to be extracted). The extraction number input window 140 displays a character string saying "Please input the number of images to be extracted," and has a minimum number input area 141 and a maximum number input area 142. By clicking the minimum number input area 141, the minimum number of images containing the same person to be made public can be input from the keyboard 8 in the minimum number input area 141, and by clicking the maximum number input area 142, the maximum number of images containing the same person to be made public can be input from the keyboard 8 in the maximum number input area 142. A decision button 143 on which the character string "decide" is displayed is formed at the bottom right of the extraction number input window 140.

[0113] When the minimum number is input in the minimum number input area 141 and the maximum number is input in the maximum number input area 142 (either the minimum number or the maximum number may be input) (step 56 in FIG. 5), and the decision button 143 is clicked (YES in step 57 in FIG. 5), the input numbers of pieces to be extracted (minimum number to be extracted, maximum number to be extracted) are stored in the memory 7 (step 58 in FIG. 5). The number to be extracted input window 140 disappears from the display screen of the display device 3.

[0114] The CPU 2 checks whether there are images that include the same person and the number of images that are publicly available is greater than the maximum number that was input (step 61 in FIG. 6). If there are images that are greater than the maximum number that was input (YES in step 61 in FIG. 6), the CPU 2 calculates the first image evaluation value and the second image evaluation value for the images that include the same person (step 62 in FIG. 6), and the CPU 2 (image candidate determination means) determines non-public image candidates from among the public images so that the number of images is within the maximum number (within the limit value) of images that include the same person (step 63 in FIG. 6).

[0115] For example, suppose that the images containing "Sato Taro" are six images with file names "0022.jpg", "0019.jpg", "0021.jpg", "0013.jpg", "0020.jpg" and "0018.jpg". Then, the second image evaluation values ​​of those six images are, for example, the values ​​shown in Table 5. Also, suppose that the maximum number of images input is five. Then, one of the six images is determined by the CPU2 as a private image candidate. As can be seen from Table 5, the image with the lowest second image evaluation value among the six images is the image with the image file name "0019.jpg", so the image with the image file name "0019.jpg" is determined by the CPU2 (image candidate determination means) as a private image. In this way, a private image candidate (non-extracted image candidate) is determined from the grouped images based on the second image evaluation value and the limit value for the number of images to be extracted that contain the same person.

[0116] Conversely, the CPU2 (image candidate determination means) may determine five images with the highest second image evaluation values ​​as public image candidates (extracted image candidates) among the six images. For example, a window including a character string stating that it is recommended that the image with the image file name "0019.jpg" be kept private may be displayed on the display screen together with the image with the image file name "0019.jpg" (non-extracted image candidate) under the control of the CPU2 (image candidate display control means). Conversely, among the six images with file names "0022.jpg", "0019.jpg", "0021.jpg", "0013.jpg", "0020.jpg" and "0018.jpg", images with image file names other than "0019.jpg" (extracted image candidates) may be displayed on the display screen together with a character string stating that it is recommended that the image be made public under the control of the CPU2 (image candidate display control means). By clicking the left arrow button 92 or the right arrow button 93, the user can cause an image display window 80 to be displayed on the display screen of the display device 3, in which an image having the image file name "0019.jpg" is displayed in the image display area 90, as shown in FIG. 14.

[0117] The user checks the image displayed in the image display area 90, and if the image is to be made private, the user clicks on the image to have the public status determined as private by the CPU2 (image determination means). This designates the image as private (YES in step 64 of FIG. 6, step 65). If no end command is given (NO in step 66), the process from step 61 is repeated.

[0118] In the above embodiment, a private image candidate is determined, but instead of determining a private image candidate, a public image candidate may be determined. For example, images with the image file names "0022.jpg", "0021.jpg", "0013.jpg", "0020.jpg" and "0018.jpg" may be determined as public image candidates. When determining an image to be made public from the public image candidates, the image is clicked and the CPU2 (image determination means) determines the public status as public.

[0119] Furthermore, when determining a private image candidate or a public image candidate, if the number of extracted images containing a specific person exceeds the maximum number or falls below the minimum number as a result of the determined private image candidate being made a private image, or the determined public image candidate being made a public image (if the number of extracted images containing a specific person falls outside the limit as a result of a determined non-extracted image candidate being made a non-extracted image, or an extracted image candidate being made an extracted image), it is preferable that the CPU2 (image candidate stopping means) be controlled to remove such images from the private image candidates (non-extracted image candidates) or public image candidates (extracted image candidates). For example, in Table 5, if the image with the image file name "0019.jpg" is determined to be a private image as described above, and the number of public images of a particular person (e.g., "Ito Hanako") falls below the minimum number, it is preferable not to determine the image with the image file name "0019.jpg" as a private image candidate, but to determine another image as a private image candidate, or to determine the image with the image file name "0019.jpg" as a public image candidate.

[0120] If a program end command is not given to the image candidate determination device 1 (NO in step 66 in FIG. 6), the process is repeated from step 61 in FIG. 6. If there are still images that are publicly available and contain more than the maximum number of images that include the same person (YES in step 61 in FIG. 6), the image evaluation values ​​for those images are calculated again (step 62 in FIG. 6).

[0121] If there are no images that are publicly available and contain the same person in more than the maximum number of images (NO in step 61 in FIG. 6), the process ends.

[0122] In the above embodiment, it is assumed that the images that can be used for public display are determined after the event is over. However, the above process may be performed in real time during the event to capture people who appear in a small number of images. In this case, in the process of step 61 in Fig. 6, if there are no images that are more than the maximum number entered (NO in step 61), the process proceeds to step 67 shown in Fig. 7.

[0123] In the process of step 67 shown in Fig. 7, it is confirmed whether there are images that are publicly available and contain the same person and the number of images that is less than the minimum number (step 67 in Fig. 7). If there are images that are less than the minimum number (YES in step 67 in Fig. 7), a shooting instruction window 150 shown in Fig. 20 is displayed on the display screen of the display device 3 (step 68 in Fig. 7). In other words, a person in which the number of identical people contained in one or more grouped images is below a threshold is detected by CPU2 (person detection means), and a photographing instruction is issued by CPU2 (photography instruction means) to photograph that person.

[0124] Referring to FIG. 20, a shooting instruction window 150 prompts the user to shoot people who appear in fewer images than the minimum number.

[0125] The shooting instruction window 150 displays the character string "There are not enough images of XX" (XX is replaced with the name of the person who is lacking in images) to inform the user of the name of the person who is lacking in the number of images, and the character string "Please take 1 to 5 more images" to inform the user of the number of images that are lacking (the number of images that is lacking varies depending on the situation, but preferably the number of images after shooting is equal to or greater than the minimum number and equal to or less than the maximum number entered as described above). A confirmation button 151 with the word "Confirm" displayed is formed at the bottom right of the shooting instruction window 150. After the user confirms who is lacking in images and how many of them they are lacking, they click the confirmation button 151. The shooting instruction window 150 then disappears from the display screen of the display device 3.

[0126] If there are no images that are less than the minimum number of images to be captured (NO in step 67 in FIG. 7), the display of the shooting instruction window 150 shown in FIG. 20 is skipped.

[0127] If an end command for the program is given to the image candidate determination device 1 (YES in step 69 in FIG. 7), the program ends (YES in step 69 in FIG. 7), and if an end command for the program is not given (NO in step 69 in FIG. 7), the process from step 61 in FIG. 6 is repeated. However, if an end command is not given (NO in step 66 in FIG. 6 or NO in step 69 in FIG. 7), the process may return to step 49 in FIG. 5.

[0128] When an image set to public / private is displayed (step 49 in FIG. 5), if the image management tab 91 is clicked (YES in step 59 in FIG. 5), the processing starts from step 25 in FIG. 3. Also, when an image set to public / private is displayed (step 49 in FIG. 5), if the face classification tab 101 is clicked (YES in step 60 in FIG. 5), the processing starts from step 38 in FIG. 4.

[0129] In the above embodiment, the image candidate determination program is terminated when the program termination determination process in step 66 in Figure 6 and the program termination determination process in Figure 7 are performed, but if a termination command is given to the image candidate determination device 1 in other processes, the image candidate determination program is terminated by the termination command being interrupted.

[0130] Furthermore, in the above-described embodiment, image data representing multiple images stored on the memory card 13 is read by the memory card reader / writer 12, but it is also possible to store image data representing multiple images in a database server separate from the image candidate determination device 1, and have the database server communicate with the image candidate determination device 1, so that the images represented by the image data sent from the database server are displayed on the display screen of the display device 3 of the image candidate determination device 1, as described above.

[0131] Furthermore, in the above embodiment, image data recorded on the memory card 13 is read, but if the communication device 4 of the image candidate determination device 1 can communicate with a digital camera (not shown) by wireless communication means, image data representing an image taken using the digital camera may be transmitted from the digital camera to the image candidate determination device 1, and the above-mentioned processing may be performed. Similarly, image data representing an image taken with a smartphone may be transmitted from the smartphone to the communication device 4 of the image candidate determination device 1, and the above-mentioned processing may be performed. Image data representing images taken with a digital camera or smartphone may be transmitted to the image candidate determination device 1 in real time, and the above-mentioned processing may be performed while taking the images, or image data representing images taken with a digital camera or smartphone may be transmitted to the image candidate determination device 1 after all the taking of images is completed.

[0132] Furthermore, in the above-described embodiment, when calculating the raw score of the first image evaluation value, evaluation values ​​related to faces such as face size score and facial expression score and evaluation values ​​related to non-faces such as number of people score and image quality score are used, but the raw score of the first image evaluation value may be calculated using only the evaluation values ​​related to faces, or only the evaluation values ​​related to non-faces.

[0133] Furthermore, in the above embodiment, the image portion of the face is associated with the name while the user is looking at the image portion of the face, but the image of the face may be associated in advance with the name of the person whose face it is, and the image portion of the face may be associated with the name by automatically determining whose face it is by comparing the image of the face with the image of the face associated with the name. Furthermore, in the above embodiment, the image portion of the face is associated with the name, but if the person is wearing a name tag instead of the image portion of the face, the user may check the name tag and associate the image portion of the face with the name, or the name written on the name tag may be automatically analyzed and associated the image portion of the face with the name.

[0134] 21 to 26 show modified examples.

[0135] Figures 21 to 23 are examples of an image display window 80. Figures 21 to 23 correspond to the image display window 80 in Figure 11 etc., and the same components are given the same reference numerals and the description will be omitted. In the image display window 80 shown in Figures 21 to 23, multiple face images and names of the same person are associated together.

[0136] The user uses the mouse 9 to select the images to which the user wishes to associate a name all at once. Specifically, the user uses the mouse 9 to drag the cursor 171 over the face images to which the user wishes to associate a name all at once. The frame of the selected images then becomes a thick frame 172. In FIG. 21, it is assumed that of the images displayed in three rows in the image display area 90, the three images in the second row (the middle row) have been selected.

[0137] Next, the name of the selected face image is selected by the user from the names displayed in the subject list area 110. Here, it is assumed that the name of the selected face image is "Tanaka Jiro," and that "Tanaka Jiro" is selected. Then, the color around the selected name "Tanaka Jiro" changes.

[0138] FIG. 22 shows how the area around the selected name of "Tanaka Jiro" has changed. In FIG. 22, the area around the name of "Tanaka Jiro" is indicated by hatching 173, and the area around the selected name of "Tanaka Jiro" has changed. In addition, a representative face image 175 (pre-registered) of the selected "Tanaka Jiro" is displayed above the subject list area 110. The user can confirm whether or not they are the same person by comparing the representative face image 175 with the face image selected from the face images displayed in the image display area 90. Next, the user moves the cursor 171 onto the selected name of "Tanaka Jiro" and drags it onto the selected face image. Then, the cursor 171 temporarily changes to a name display area 174 that shows the selected name of "Tanaka Jiro". After that, as shown in FIG. 23, a name display area 176 is displayed on each of the selected images.

[0139] 21 to 23, among the names displayed in the subject list area 110, the names of people who are prohibited from being made public are marked with a prohibition mark 170. By seeing the prohibition mark 170, it is possible to prevent the names from being made public by mistake.

[0140] 21 to 23, for images that are recognized as face images but in which the face is turned too sideways or the image of the face is blurred, flags such as "too sideways," "too blurred," and "unsuitable for publication" may be added to the images. When selecting an image to be made public, such flags can be referenced to determine whether to make the image public or private.

[0141] Furthermore, the images displayed in the image display area 90 in Figures 21 to 23 may be displayed in order of the number of times a subject appears or in the order in which they were photographed, so that the more frequently a subject appears, the higher it is displayed, and the earlier the photographed time, the further to the left it is displayed.

[0142] Fig. 24 shows another example of the image display window 80. In Fig. 24, the same components as those shown in Fig. 8 etc. are given the same reference numerals and the description thereof will be omitted.

[0143] In the event management area 82, the sub-events "presentation of graduation certificate" and "words from the principal" are generated under the event "graduation ceremony." Images managed as the event "graduation ceremony" are managed as sub-events such as "presentation of graduation certificate" and "words from the principal." In addition, to the right of the event addition button 83, a sub-event addition button 85 with the text "add sub-event" is formed.

[0144] For example, when the character string "graduation ceremony" is clicked and then the add sub-event button 85 is clicked, a sub-event input window appears on the image display window 80, and the user uses the sub-event input window to input a new sub-event. If the user then inputs the character string "presentation of graduation certificate" using the keyboard 8, the character string "presentation of graduation certificate" will be displayed below the character string "graduation ceremony". Similarly, the character string "words from the principal" will be displayed below the character string "graduation ceremony".

[0145] When the character string "presentation of graduation certificate" is clicked, an image managed as the "graduation ceremony" event is dragged and dropped into image display area 90, whereby the dragged and dropped image is managed by the event "presentation of graduation certificate." When the character string indicating a sub-event displayed in event management area 82 is clicked and then event delete button 84 is clicked, the sub-event identified by the clicked character string and the image managed by that sub-event are deleted.

[0146] In the image display window 80 shown in FIG. 24, the public settings tab 102 has been clicked, and a large number of images 165 managed by an event named "graduation ceremony" are displayed in the image display area 90.

[0147] A display switching pull-down 160 is formed in the upper right of the image display area 90. The display switching pull-down 160 is used when displaying images in the image display area 90 divided by the shooting time or the like. By clicking the pull-down button 161, a multiple time and date display switching menu such as "1 hour interval" and "2 hour interval" appears, from which the user can select the display interval. Images are displayed in the image display area 90 according to the display interval specified by the display switching pull-down 160.

[0148] Assume that one-hour intervals are selected by the display switching pull-down 160. Then, the image display area 90 is divided into areas every hour. In FIG. 24, a first image display area 90A and a second image display area 90B appear. The first image display area 90A and the second image display area 90B are separated by a dashed line, but they may be separated by other methods. Areas other than these image display areas 90A and 90B also appear in the image display area 90 by moving the slide bar 98.

[0149] The display time 163 displayed in the first image display area 90A is displayed in the upper left of the first image display area 90A. In Fig. 24, the characters "11:00" are displayed as the display time 163. Since the display interval is an hour interval, the first image display area 90A displays images taken between 11:00 and the like among the images managed by events, etc. displayed in the event management area 82.

[0150] A check box 162 is formed on the left side of the display time 163, and a check box 164 is also formed in the upper right of the image 165 displayed in the first image display area 90A. Both check boxes 162 and 164 specify the publication of the image. When the check box 162 formed on the left side of the display time 163 is checked, the check box 164 formed in the upper right of the image 165 specified by that display time 163 (all images 165 displayed in the first image display area 90A) is checked, and the publication of those images 165 is specified. When the check box 164 formed in the upper right of the image 165 is checked, the checked image is specified to be published. When the check box 164 formed in the upper right of the image 165 is unchecked, that image becomes private. When the check box 164 is checked, the image 165 is surrounded by a thick frame 166. This is to make it easier to distinguish between images with and without the check box 164 checked, and the color of the frame may be changed.

[0151] The character string "12:00" is displayed in the display time 163 of the second image display area 90B, and an image 165 taken around 12 o'clock is displayed in the second image display area 90B. A check box 162 is formed to the left of the display time 163, and a check box 164 is also formed in the upper right of the image 165. By checking these check boxes 162 or 164, the checked image is made public, in the same manner as described for the image displayed in the first image display area 90A.

[0152] Fig. 25 shows another example of the image display window 80. In Fig. 25, the same components as those in Fig. 8 etc. are given the same reference numerals and the description thereof will be omitted.

[0153] Check boxes 195 are also formed in the upper right corner of the images displayed in the image display area 90 of the image display window 80 shown in Fig. 25, and by checking these check boxes 195, the corresponding images are set to be public. Images with a checked check box 195 are surrounded by a thick frame 196. Images are not public if the check box 195 is not checked.

[0154] A minimum number of images to be released setting area 180 is formed above the left side of image display area 90. Minimum number of images to be released setting area 180 is used to set the minimum number of images to be released. An up arrow button 181 and a down arrow button 182 are formed on the right side of minimum number of images to be released setting area 180. Each time up arrow button 181 is clicked, the number displayed in minimum number of images to be released setting area 180 increases by one, and each time down arrow button 182 is clicked, the number displayed in minimum number of images to be released setting area 180 decreases by one.

[0155] A published number display area 190 is formed on the left side of the image display area 90. The published number display area 190 displays the number of published images in correspondence with the names of people included in the published images.

[0156] The published number display area 190 displays the name of the subject included in the image (for example, Taro Sato), the number of published images (for example, 6), and a representative facial image 194 of the subject. The published number display area 190 also displays a mark 191 and a line 193 that serve as a guide to the minimum published number set in the minimum published number setting area 180. Furthermore, the number of published images for each subject name is shown by a bar graph (shown by hatching) 192. The fewer the published number of images, the shorter the bar graph 192, and the more the published number of images, the longer the bar graph 192. The further to the right the bar graph 192 extends, the greater the number of published images. If the right end of bar graph 192 does not exceed line 193, the number of images of the subject represented by that bar graph 192 that have been made public has not reached the minimum number of images made public, and if the right end of bar graph 192 matches line 193, the number of images of the subject represented by that bar graph 192 that have been made public matches the minimum number of images made public, and if the right end of bar graph 192 exceeds line 193, the number of images of the subject represented by bar graph 192 that have been made public exceeds the minimum number of images made public. By checking whether the right end of bar graph 192 exceeds line 193, it is possible to check for each subject whether or not more than the minimum number of images have been set to be made public.

[0157] Furthermore, a mark 197 is displayed on the name of the subject corresponding to an image that does not meet the minimum number of images to be released in order to alert the user. By looking at the mark 197, the user can select images to be released so as to reach the minimum number of images to be released. The color of the name of the subject corresponding to the image that does not meet the minimum number of images to be released may be different from the color of the name of the subject corresponding to the image that meets the minimum number of images to be released. Furthermore, a prohibition mark 170 is also displayed in correspondence with the name displayed in the number of images to be released display area 190. This makes it possible to prevent an image that includes a non-disclosed subject from being mistakenly set as a target for release.

[0158] FIG. 26 is an example of an image that has been set as a publicly accessible object by checking the check box 195, etc.

[0159] It is assumed that an undisclosed subject 198 is included in the image set as a subject to be made public. In such a case, the undisclosed subject 198 is trimmed from the image. The undisclosed subject 198 is removed from the image to be made public.

[0160] FIG. 27 shows another example of the image display window 80 .

[0161] An image 200 is displayed in the image display area 90. This image 200 includes three people, subjects 201, 202, and 203. If the image 200 includes subjects 201, 202, and 203, the facial images of the subjects 201, 202, and 203 should be recognized, and facial images corresponding to the subjects 201, 202, and 203 should be detected. However, there are cases where facial images corresponding to the number of subjects are not detected. For example, there are cases where the facial image of subject 201 is detected and a facial frame F1 is displayed, and the facial image of subject 203 is detected and a facial frame F3 is displayed, but the facial image of subject 202 is not displayed.

[0162] Above the image 200, the number of subjects detected 210 is displayed. Two people are displayed as the number of subjects detected from the image 200. Below the image, a confirmation message 211 is displayed to confirm whether the number of people detected from the image 200 is correct. By looking at the confirmation message, the user can see that an undetected face is included in the image 200. For example, by selecting a name corresponding to the name of the subject 202 with an undetected face from among the names displayed in the public / private information display area 120 and clicking the corresponding face, the selected name (name) is associated with the image 200.

[0163] The processing units that execute the above-mentioned processes include the CPU 2, which executes software and functions as various processing units, as well as programmable logic devices such as FPGAs (field-programmable gate arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (application specific integrated circuits), which are processors with a circuit configuration specially designed to execute specific processes.

[0164] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). As an example of configuring multiple processing units with one processor, first, as represented by computers such as client computers and servers, one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by systems on chips, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with a single IC (integrated circuit) chip. In this way, the various processing units are configured using one or more processors of various types as a hardware structure.

[0165] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0166] The image candidate determination device may be constituted by a personal computer, or may be constituted by a so-called smart device such as a smartphone or a tablet device, or may be constituted by a mobile phone such as a feature phone. [Explanation of symbols]

[0167] 1: Image candidate determination device, 2: CPU, 3: Display device, 4: Communication device, 5: Hard disk, 6: Hard disk drive, 7: Memory, 8: Keyboard, 9: Mouse, 10: Compact disk drive, 11: Compact disk, 12: Memory card reader / writer, 13: Memory card, 80: Image display window, 81: Character string, 82: Event management area, 83: Event addition button, 84: Event deletion button, 90: Image display area, 91: Image management tab, 92: Left arrow button, 93: Right arrow button, 94: Image identification number, 95: Character string, 96: Character string, 97: Up arrow button, 98: Slide bar, 99: Down arrow button, 101: Face classification tab, 102: Public setting tab, 110: Subject list area, 111: Face classification button, 112: Character string, 113: Character string, 114: Character string, 120: Public / private information display area, 121-1 26: Check box, 129: Check box, 130: Number of images input button, 140: Extraction number input window, 141: Minimum number of images input area, 142: Maximum number of images input area, 143: Confirm button, 150: Shooting instruction window, 151: Confirm button, 160: Display switch pull-down, 161: Pull-down button, 162: Check box, 163: Display time, 164: Check box, 165: Image, 166: Bold frame, 170: Prohibition mark, 171: Cursor, 172: Bold frame, 173: Hatching, 174: Name display area, 175: Face image, 176: Name display area, 180: Minimum number of images to be released setting area, 181: Up arrow button, 182: Down arrow button, 190: Number of images to be released display area, 191: Mark, 192: Bar graph, 193: Line, 194: Face image, 195: Check box, 196: Bold frame, 197: Mark, 198: Subject, 200: Image, 201-203: Subject, 210: Number of subjects detected, 211: Confirmation message, F: Face frame, F1-F6: Face frame, G1-G3: Face image group

Claims

1. an image grouping means for accepting a plurality of images and grouping one or more images containing the same person; an image candidate determination means for determining an extracted image candidate or a non-extracted image candidate from among the images grouped by the image grouping means, based on a comprehensive image evaluation value calculated based on a person image evaluation value calculated for each person appearing in the images; and a correction means for correcting the person image evaluation value using a correction coefficient that increases the person image evaluation value as the number of images including the same person decreases; An image candidate determination device comprising:

2. An image candidate display control means for displaying the extracted image candidates or non-extracted image candidates determined by the image candidate determination means on a display screen; The image candidate determination device according to claim 1 , further comprising:

3. The person image evaluation value is calculated based on at least a face evaluation value, A facial image portion extraction means for extracting a facial image portion from each of the plurality of images, The image grouping means includes: the facial image parts extracted by the facial image part extraction means are grouped into groups of facial image parts of the same person; The image candidate determination device according to claim 1 .

4. An image determination means for determining an extracted image or a non-extracted image from the extracted image candidates displayed under the control of the image candidate display control means, or for determining a non-extracted image or an extracted image from the non-extracted image candidates displayed under the control of the image candidate display control means; The image candidate determination device according to claim 2 , further comprising:

5. an image candidate stopping means for removing the one non-extracted image candidate or the one extracted image candidate from the non-extracted image candidates or the extracted image candidates when the number of extracted images including a specific person falls outside a limit value in response to the one non-extracted image candidate determined by the image candidate determination means being determined as a non-extracted image or the one extracted image candidate being determined as an extracted image; The image candidate determination device according to claim 1 , further comprising:

6. The image grouping means includes: For adults, non-identical people are treated as the same person and grouped by image.

6. An image candidate determination device according to claim 2.

7. A first evaluation value calculation means for calculating a lower person image evaluation value as the number of people included in the image increases; The image candidate determination device according to claim 1 , further comprising:

8. A person detection means for detecting a person in which the number of identical persons included in one or more images grouped by the image grouping means is equal to or less than a threshold, and a photographing instruction means for instructing a photographing operation of a person detected by the person detection means; The image candidate determination device according to claim 1 , further comprising:

9. An image grouping means for receiving a plurality of images and grouping one or more images including the same person; an image candidate determining means for determining an extracted image candidate or a non-extracted image candidate from among the images grouped by the image grouping means, based on a comprehensive image evaluation value calculated based on a person image evaluation value calculated for each person appearing in the images; a correction means for correcting the person image evaluation value using a correction coefficient that increases the person image evaluation value as the number of images including the same person decreases; Method for determining image candidates.

10. A program for controlling a computer of an image candidate determination device and readable by said computer, comprising: Accepting multiple images and grouping one or more images containing the same person; determining candidates for extracted images or candidates for non-extracted images from among the grouped images based on a comprehensive image evaluation value calculated based on a person image evaluation value calculated for each person appearing in the images; A program for controlling a computer of an image candidate determination device so as to correct the person image evaluation value using a correction coefficient that increases the person image evaluation value as the number of images including the same person decreases.

11. A recording medium storing the program according to claim 10.

Images