Information processing system, information processing apparatus, information processing method, and program

Abstract

To provide a novel technique for identifying a person who cannot be identified based on an original image, by using image processing on other image.SOLUTION: An information processing apparatus 100 according to the present invention includes a collation unit 102 for collating first feature information indicating features of a retrieval target person with first feature information extracted from a person included in a first image 10, an extracting unit 104 for extracting second feature information from the person included in the first image if the collation unit 102 obtains a matched result, and a registration unit 106 for storing the second feature information extracted from the person included in the first image in a second feature information storing unit 110.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program for performing image recognition processing.

Background Art

[0002] In recent years, technologies for detecting a person who meets desired conditions using an image have been developed. For example, Patent Document 1 describes a system for determining whether a predetermined person such as a missing person or a wanted criminal is included in a moving image captured in a street or the like. This system generates feature data of face images included in each frame of a captured image, and classifies the feature data for each person so that the feature data of the same person forms one group by comparing the generated feature data. Then, based on the classified feature data, the system determines representative feature data for each person, and transmits at least one of the determined feature data and a face image corresponding to the feature data to a face authentication device as representative face data. The face authentication device collates the transmitted data with face data of a predetermined person registered in a face data dictionary.

[0003] Patent Document 2 describes an image processing apparatus that automatically tracks a person by identifying the same person from images captured by different cameras. The different cameras capture the person from different directions.

[0004] Patent Document 3 describes detecting a walking state of a person from a sequence of person images obtained from distant frames or different cameras, and determining whether the persons included in different image sequences are the same based on the walking state.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

[0006] The present inventor has studied a new method for identifying a person who cannot be identified in an original image by using image processing on another image. That is, an object of the present invention is to provide a new method for identifying a person who cannot be identified in an original image by using image processing on another image. [Means for Solving the Problems]

[0007] In each aspect of the present invention, in order to solve the above-described problems, the following configurations are adopted, respectively.

[0008] The first aspect relates to an information processing apparatus. The information processing apparatus according to the first aspect is an extraction means for extracting face information from an object included in the first image when a collation result of first feature information indicating features of a search target in which no face is shown and first feature information extracted from an object included in the first image matches; a registration means for storing the extracted face information in a first storage means; and has.

[0009] The second aspect relates to an information processing method executed by at least one computer. The first information processing method according to the second aspect is when an information processing apparatus extracts face information from an object included in the first image when a collation result of first feature information indicating features of a search target in which no face is shown and first feature information extracted from an object included in the first image matches, stores the extracted face information in a first storage means, and includes this.

[0010] As another aspect of the present invention, it may be a program that causes at least one computer to execute the method of the second aspect, or it may be a computer-readable recording medium that records such a program. This recording medium includes a non-transitory tangible medium. This computer program includes computer program code that causes a computer to implement the control method on an information processing device when executed by the computer.

[0011] Note that any combination of the above components, as well as those obtained by converting the expression of the present invention among a method, an apparatus, a system, a recording medium, a computer program, etc., are also effective as aspects of the present invention.

[0012] In addition, the various components of the present invention do not necessarily have to exist independently of each other. It is also possible that a plurality of components are formed as one member, one component is formed of a plurality of members, a certain component is part of another component, or a part of a certain component overlaps with a part of another component, and so on.

[0013] In addition, although a plurality of procedures are described in order in the method and computer program of the present invention, the described order does not limit the order of executing the plurality of procedures. Therefore, when implementing the method and computer program of the present invention, the order of the plurality of procedures can be changed within a range where there is no problem in content.

[0014] Furthermore, the plurality of procedures of the method and computer program of the present invention are not limited to being executed at different timings individually. Therefore, it is also possible that another procedure occurs during the execution of a certain procedure, or a part or all of the execution timing of a certain procedure overlaps with the execution timing of another procedure, and so on.

Advantages of the Invention

[0015] According to each of the above aspects, it is possible to provide a new method for identifying a person who cannot be identified in the original image by using image processing on other images.

Brief Description of the Drawings

[0016] The above-described object, as well as other objects, features, and advantages, will become more apparent from the following preferred embodiments and the accompanying drawings described below.

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Embodiments for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate. In addition, in each figure of this specification, the configuration of the part that does not relate to the essence of the present invention is omitted and not shown.

[0019] (First Embodiment) FIG. 1 is a diagram conceptually showing the configuration of an image processing system according to an embodiment of the present invention. The image processing system 1 includes an information processing apparatus 100. The information processing apparatus 100 searches for a certain person from images captured by a plurality of cameras 5a, 5b, 5c... (referred to as "camera 5" when there is no need to particularly distinguish).

[0020] For example, the person to be searched for may be shown in a back view in the original image (second image 22), and thus face authentication may not be possible. In such a case, the information processing apparatus 100 uses information such as body shape (hereinafter referred to as first feature information) to find candidates similar to the person from another image (first image 10) captured by the camera 5, and identifies the person by face authentication using an image in which the face is shown among them.

[0021] In the embodiment, "acquisition" includes at least one of the following: the self-device going to obtain data or information stored in another device or storage medium (active acquisition), and the self-device inputting data or information output from another device (passive acquisition). Examples of active acquisition include sending a request or inquiry to another device and receiving its reply, and accessing and reading from another device or storage medium. Examples of passive acquisition include receiving information distributed (or transmitted, push-notified, etc.). Furthermore, "acquisition" may mean selecting and acquiring from the received data or information, or selecting and receiving the distributed data or information.

[0022] The camera 5 is equipped with imaging elements such as a lens and a CCD (Charge Coupled Device) image sensor. For example, it is a network camera such as an IP (Internet Protocol) camera. The network camera has, for example, a wireless LAN (Local Area Network) communication function and is connected to the information processing device 100 via a communication network, that is, a relay device (not shown) such as a router. These cameras 5 may be so-called surveillance cameras installed in the street or inside and outside buildings. And the camera 5 may be provided with a mechanism that follows the movement of a specific person and controls the direction of the camera body or lens, zoom control, focusing, etc.

[0023] The camera 5 and the information processing device 100 may not be connected, may be directly connected, or may be indirectly connected via a communication network 3 or the like. When the camera 5 and the information processing device 100 are not connected, for example, image data is read and acquired from media 7a and media 7b (hereinafter referred to as "media 7" when there is no need for particular distinction) that record the images captured by the camera 5a and the camera 5b respectively.

[0024] The camera 5c in an example connected to the information processing device 100 via a communication network 3 or the like has, for example, a wireless LAN (Local Area Network) communication function and is connected to the information processing device 100 via a communication network 3, that is, a relay device (not shown) such as a router. The camera 5c is preferably a network camera such as an IP (Internet Protocol) camera.

[0025] Also, the timing of transmitting an image from the camera 5 to the information processing device 100 may be real-time distribution, for example, streaming distribution, or may transmit images for a predetermined period at a predetermined interval. This transmission timing may be appropriately selected based on the memory capacity, communication capacity, image processing ability of the camera 5 and the information processing device 100, and the communication status between the camera 5 and the information processing device 100, or may be changed according to a change in the situation.

[0026] For example, the image data captured by the camera 5 may be directly transmitted to the information processing apparatus 100, and the information processing apparatus 100 may sequentially receive this image data. Alternatively, a storage device (not shown) accessible to both the camera 5 and the information processing apparatus 100 may be provided. In this case, the image data captured by the camera 5 is stored in the storage device. Then, the information processing apparatus 100 reads the image data from this storage device.

[0027] Here, the image data may be at least either a still image or a moving image. The data format, file format, file size, image size, image resolution, frame rate of the moving image, etc. are not particularly limited, and various formats can be adopted according to the specifications, standards, performance, etc. of the camera 5 and the information processing apparatus 100, or the processing ability, accuracy, etc. of the image analysis. And at least one frame of this image data may be at least one of the first image 10 and the second image 22 described later.

[0028] FIG. 2(a) is a diagram showing an example of the data structure of the video data 40. The video data 40 is captured by a certain camera 5 and is received from the camera 5 or recorded on the medium 7. The video data 40 includes a video ID for identifying the video data 40, a media ID for identifying the medium 7 on which the video data 40 is recorded or a camera ID for identifying the camera 5 by which the video data 40 is captured, information indicating the shooting conditions of the camera 5 (for example, shooting location, shooting time, etc.), and the main body of the video data.

[0029] FIG. 2(b) is a diagram showing an example of the data structure of the first image 10. The first image 10 is a part of the video data 40 captured by a certain camera 5 and may be a certain frame of the video data 40. The first image 10 includes an image ID for identifying the image, information indicating a video ID for identifying the video data 40 that is the source of the image, and the main body of the image data. When the main body of the image data is a frame, the first image 10 further includes information indicating a frame number that can identify the frame. Also, the image ID may be a file name. Note that the first image 10 is an image to be searched for a person to be searched, unlike the second image 22. Also, the first image 10 may be captured by a camera 5 different from the second image 22.

[0030] FIG. 3 is a functional block diagram showing the logical configuration of the information processing apparatus 100 according to the present embodiment. The information processing apparatus 100 includes a collation unit 102, an extraction unit 104, and a registration unit 106. In the example shown in this figure, the information processing apparatus 100 further includes a second feature information storage unit 110. The second feature information storage unit 110 may be integrated with the main body of the information processing apparatus 100 or may be separate. The second feature information storage unit 110 may have a database structure or other data structure.

[0031] The collation unit 102 performs a collation process of collating the first feature information indicating the features of the person to be searched with the feature information of the person area extracted from the person included in the first image 10. Here, the first feature information extracted from the second image 22 is feature information indicating the features of the person area of the person to be searched and is hereinafter also referred to as the first feature information 14a. Also, the feature information of the person area extracted from the person included in the first image 10 to be searched is also referred to as the first feature information 14b. When the collation result by the collation unit 102 matches, the extraction unit 104 extracts the second feature information from the person included in the first image 10. The registration unit 106 stores the second feature information extracted from the person included in the first image 10 in the second feature information storage unit 110. In other words, the registration unit 106 registers the second feature information extracted from the person included in the first image 10 in the second feature information storage unit 110.

[0032] Specifically, the matching unit 102 acquires the first image 10 by reading it from the medium 7 or receiving it from the camera 5. Then, the matching unit 102 first identifies the human region from the acquired first image 10, and performs a matching process with the first feature information 14a for the identified human region. Here, there may be a plurality of pieces of first feature information 14a.

[0033] FIG. 4(a) shows an example of the second image 22. The second image 22 is an image in which a person is depicted but the person cannot be identified by face authentication or the like. The second image 22 is, for example, an image in which the face of a person is not depicted such as when the person is facing backward, an image in which the face is not depicted to such an extent that matching accuracy equal to or higher than a reference value can be obtained in face authentication, or a low-quality image in which face authentication cannot be performed. Note that the image in which the person cannot be identified is not limited to the case where face authentication cannot be performed. It may be an image in which the person cannot be identified by other authentication methods. The other authentication method is, for example, biometric authentication and is at least one of iris, auricle, vein, and fingerprint. Also, for the person depicted in the second image 22, the first feature information 14a may be extracted from the human region of the person without performing person identification by face authentication processing or other authentication processing. That is, it is not always necessary to perform face authentication processing or other authentication processing on the second image 22. In other words, the second image 22 may be an image in which the person depicted therein has not been identified.

[0034] In the matching process, the matching unit 102 determines that the human region having a degree of coincidence with the first feature information 14a equal to or higher than a reference is a region corresponding to the first feature information 14a. The extraction unit 104 identifies the human region including the region having a degree of coincidence with the first feature information 14a equal to or higher than a reference from the first image 10, and further identifies the face region of the human region.

[0035] Note that the degree of coincidence is a value indicating the degree of matching. In the present embodiment, the degree of coincidence is a numerical value from 0 to 1, where a perfect match is 1 and a complete mismatch is 0.

[0036] The extraction unit 104 extracts the feature information of the specified face region to generate second feature information, and the registration unit 106 stores it in the second feature information storage unit 110. As shown in FIGS. 4(b) and 4(c), the person region 16 and the face region 20 are specified from the first image 10. The person including these regions becomes a search target candidate.

[0037] Note that the extraction unit 104 may extract the first feature information 14a of a plurality of detection target persons from the person regions of persons included in a plurality of different second images 22, and the registration unit 106 may associate the plurality of first feature information 14a with each other as information of the same person and store them in the first feature information storage unit 112. The collation unit 102 may perform a collation process using the plurality of first feature information 14a.

[0038] Further, the information processing apparatus 100 does not necessarily have to include the first feature information storage unit 112. The collation unit 102 may read and acquire the first feature information 14a from the first feature information storage unit 112, or may acquire it by input from an external device. Further, the first feature information storage unit 112 may also serve as the second feature information storage unit 110. In this case, the information processing apparatus 100 uses the generated second feature information as the first feature information.

[0039] In the present embodiment, the person region includes regions corresponding to a first part and a second part. The second part is, for example, at least one of the regions of the face, iris, auricle, vein, and fingerprint. Alternatively, the second part may be not a region but a person's gait. The first part may or may not include the second part. In the present embodiment, the first part is the region of the whole person, and the second part is the face region. The first feature information (the first feature information 14a and the first feature information 14b) of the first part is feature information indicating visual features such as the back style or clothing of the person. In this specification, the first feature information is also referred to as "person region feature information", and the second feature information is also referred to as "face information".

[0040] The first feature information (the first feature information 14a and the first feature information 14b) includes information indicating features such as height, shoulder width, the ratio of body parts, clothing (shape, color, material, etc.), hairstyle (including hair color), ornaments (hats, glasses, accessories, etc.), and possessions (bags, umbrellas, canes, etc.). Further, the person region feature information may include information such as the likelihood of the feature information.

[0041] In this embodiment, the first feature information 14a of the person to be searched for is generated using the second image 22 in which the person to be searched for is shown. The second image 22 may be an image captured by the camera 5, or may be an image captured by other imaging means or an image read by a scanner.

[0042] The information processing apparatus 100 further includes a first feature information generation unit (not shown). This first feature information generation unit generates the first feature information 14a using the second image 22 in which the person to be searched for is shown, and stores it in the first feature information storage unit 112.

[0043] FIG. 5 is a flowchart showing an example of the procedure of the process in which the information processing apparatus 100 generates the first feature information 14a. The first feature information generation unit acquires an image (the second image 22) of the person to be searched for (step S10), generates the first feature information (person region feature information) from the acquired image, and stores it in the first feature information storage unit 112 (step S20).

[0044] FIG. 6(a) is a diagram showing an example of the data structure of the first feature information storage unit 112. The first feature information storage unit 112 stores in association a search target ID for identifying the person to be searched for, the first feature information 14a of the person, and the image data of the original image from which the first feature information 14a was acquired. The first feature information storage unit 112 may store the main body of the image data, or may store a path indicating the location and file name where the image data is stored.

[0045] FIG. 6(b) is a diagram showing an example of the data structure of the second feature information storage unit 110. The second feature information storage unit 110 stores, in association with each other, a candidate ID for identifying a candidate of a person to be searched for and second feature information of the candidate. The second feature information storage unit 110 stores information on at least one candidate of a person to be searched for. When there are a plurality of persons to be searched for, as shown in FIG. 6(c), the second feature information storage unit 110 may associate the candidate ID and the second feature information of the candidate with the search target ID from which the candidate is derived.

[0046] FIG. 7 is a flowchart showing an example of the operation of the information processing apparatus 100. FIG. 8 is a flowchart showing details of the search process in step S110 of FIG. 7.

[0047] First, the matching unit 102 acquires the first feature information 14a (person area feature information) of the person to be searched for (step S101). Further, the matching unit 102 acquires the first image 10 (step S103). Then, the process proceeds to the search process in step S110 of FIG. 8.

[0048] As shown in FIG. 8, in the search process in step S110, the matching unit 102 first performs a matching process between the first feature information 14a of the person to be searched for and the first feature information 14b of the person area extracted from the person included in the first image 10 (step S111). When the matching result between the first feature information 14a and the first feature information 14b matches, in other words, when a region with a degree of match with the first feature information 14a equal to or higher than the reference is found (YES in step S113), the extraction unit 104 generates second feature information (face information) from the person included in the first image 10 (step S115). For example, the extraction unit 104 may specify a person area including the region, specify a face area among the specified person areas, and extract feature information of the specified face area to generate second feature information (face information). Then, the registration unit 106 stores the generated face information in the second feature information storage unit 110 (step S117).

[0049] In this embodiment, when the matching unit 102 finds a person who matches the person area feature information (first feature information 14a) of the search target person from the first image 10, the extraction unit 104 generates the feature information of the face area of the person, and the registration unit 106 stores it in the second feature information storage unit 110. Then, the feature information of the face area stored in the second feature information storage unit 110 is used for the person identification process in image processing. According to this embodiment, the search target person can be found from the first image 10 captured by the camera 5 using the person area feature information (first feature information 14a) obtained from an image in which the face of the search target person does not appear, and the feature information of the face of the found person can be extracted and stored.

[0050] (Second Embodiment) FIG. 9 is a functional block diagram showing the logical configuration of the information processing apparatus 100 according to the second embodiment. The information processing apparatus 100 of this embodiment is the same as the above embodiment except that the second feature information stored in the second feature information storage unit 110 is compared with the pre-registered face authentication information to obtain candidate information of the search target person.

[0051] The information processing apparatus 100 in FIG. 9 further includes an authentication unit 120 and an acquisition unit 122 in addition to the configuration shown in FIG. 3. The authentication unit 120 performs a matching process on the second feature information stored in the second feature information storage unit 110 using the authentication information database 114. In the authentication information database 114, the second feature information of a specific person is registered in association with the person's information. The acquisition unit 122 acquires the person's information associated with the second feature information determined to have a matching degree equal to or higher than a reference from the authentication information database 114 by the matching process.

[0052] FIG. 10 is a diagram showing an example of the data structure of the authentication information database 114. In the authentication information database 114, an authentication ID for identifying a specific person (hereinafter referred to as a registrant) for whom authentication information is registered, face feature information (second feature information) indicating the features of the registrant's face, the registrant's face image data, the registrant's name, and the registrant's attribute information are registered in association with each other.

[0053] The authentication information database 114 may register the actual body of the face image data, or may register the path indicating the location and file name where the face image data is stored. This face image data is used for screen display as described later, and the operator can view the face photo displayed on the screen together with the name and attribute information.

[0054] In the example of FIG. 9, the authentication information database 114 is a part of the information processing apparatus 100. The authentication information database 114 may be integrated with the main body of the information processing apparatus 100 or may be separate. The authentication information database 114 may be included in the same apparatus as the second feature information storage unit 110, or may be included in different apparatuses. The authentication information database 114 has a database structure, but may have other data structures.

[0055] The function of the authentication unit 120 in the present embodiment may be possessed by the collation unit 102. Further, when the authentication result by the authentication unit 120 matches, the information processing apparatus 100 may include an output processing unit (not shown) that reads out the identification information (authentication ID, name, etc.) corresponding to the face feature information stored in the authentication information database 114 and performs output processing to the output unit. The output processing will be described in detail later.

[0056] FIG. 11 is a flowchart showing an example of the procedure of the authentication process of the information processing apparatus 100 in the present embodiment. First, the authentication unit 120 executes a collation process on the second feature information stored in the second feature information storage unit 110 using the authentication information database 114 (step S121). Then, when there is a person determined to have a degree of coincidence equal to or higher than the reference by the collation process (YES in step S123), the authentication unit 120 acquires the information (name, etc.) of the person from the authentication information database 114 (step S125).

[0057] Also, in this embodiment, the registration unit 106 of the information processing apparatus 100 associates the region corresponding to at least the first feature information 14a in the first image 10 with the second feature information and stores it in the second feature information storage unit 110.

[0058] For example, as shown in FIG. 12, after step S117 of the flowchart in FIG. 8, the registration unit 106 may associate the image data of the person region in the first image 10 with the second feature information (face information) and store it in the second feature information storage unit 110 (step S119).

[0059] FIG. 13 is a diagram showing an example of the data structure of the second feature information storage unit 110 of this embodiment. In this embodiment, the second feature information storage unit 110 may store, in addition to the information described in FIG. 6(b), the degree of match information calculated in the matching process of step S121 in FIG. 11, the image data of the person region, and the image ID (shown as the original image ID in the figure) for identifying the first image 10 from which the person region was obtained, in association with each other.

[0060] The information processing apparatus 100 can obtain the video ID from the first image 10 illustrated in FIG. 4(b) by associating the original image ID, and further obtain information such as the shooting location from the second image 22 illustrated in FIG. 4(a) in association with the second feature information. Also, the information processing apparatus 100 may store, in the second feature information storage unit 110, shooting conditions such as the shooting location, shooting direction, and shooting time of the camera 5 in association with the second feature information.

[0061] As described above, in this embodiment, the authentication unit 120 performs a matching process between the face information stored in the second feature information storage unit 110 and the face information registered in the authentication information database 114, so that the acquisition unit 122 can obtain information about similar persons. Thus, according to this embodiment, the same effects as those of the above embodiment can be achieved, and further, face authentication processing is performed on candidates similar to the search target person using the person region feature information from the first image 10 captured by the camera 5, and information about similar persons can be obtained.

[0062] (Third Embodiment) FIG. 14 is a functional block diagram showing a logical configuration of an information processing apparatus 100 according to the third embodiment. The information processing apparatus 100 of the present embodiment is the same as that of the second embodiment except for having a configuration for displaying information on detected candidates. In addition to the configuration of FIG. 9, the information processing apparatus 100 further includes a display processing unit 130. The display processing unit 130 causes a display device 132 such as a liquid crystal display or an organic EL (ElectroLuminescence) display connected to the information processing apparatus 100 to display various information.

[0063] In the present embodiment, the acquisition unit 122 acquires a plurality of candidate person region images and information (name, attributes, etc.) from the authentication information database 114. Then, the display processing unit 130 causes the image display unit 214 to display each candidate person region image, and for each candidate, causes the candidate display unit 222 to display information on at least one person detected as a candidate as a result of the face authentication process. Specifically, the display processing unit 130 causes at least two of the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 to be displayed on the same screen.

[0064] FIG. 15 is a diagram showing an example of a result list screen 200 displayed by the information processing apparatus 100 of the present embodiment. In the example of this figure, the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 are displayed on the same result list screen 200. Examples of causing at least two of the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 to be displayed on the same screen will be described later.

[0065] In this embodiment, the acquisition unit 122 acquires at least a part of each piece of information to be displayed by the display processing unit 130. Some of this information includes, for example, the second image 22 in which the first feature information 14a (person area feature information) indicating features related to areas other than the face of the person to be searched is extracted, the first image 10 (hereinafter also referred to as the candidate person area image) in which the collation result with the first feature information 14a (person area feature information) is a match and the face area is detected, and the result information. The result information shows the result of collating the face information stored in the authentication information database 114 with the second feature information extracted from the face area, and is generated by the authentication unit 120. Specifically, the result information is the data stored in the authentication result storage unit 116. That the collation result with the first feature information 14a (person area feature information) is a match means, for example, including an area where the degree of match with the first feature information 14a (person area feature information) is equal to or higher than a reference.

[0066] The second image 22 in which the person to be searched is shown is displayed on the search target display unit 202. The second image 22 displayed on the search target display unit 202 is a part of the second image 22 that at least includes the person area of the person to be searched, and may be, for example, an image generated by cutting out a rectangle including the person area of the person to be searched. The search result display unit 210 includes at least one candidate display unit 212. Each candidate display unit 212 includes an image display unit 214. In the image display unit 214, the candidate person area image of the candidate is displayed together with the imaging conditions of the image (for example, the imaging location) and the degree of match with the second image 22 displayed on the search target display unit 202. The candidate person area image is at least a part of the first image 10, and may be, for example, an image generated by cutting out a rectangle including the person area that matched in the collation. The authentication result display unit 220 displays the result of face authentication, and the candidate display unit 222 is displayed for each candidate of the candidate display unit 212. The candidate display unit 222 includes an image display unit 224. In the image display unit 224, at least the face image of a person with a high degree of match with the candidate person area image is displayed. The information displayed in the image display unit 224 is included in the above-described result information.

[0067] In the example of FIG. 15, the information of multiple candidates is displayed in descending order of the degree of match with the first feature information 14a. Here, in the candidate display unit 222, the information of multiple candidates may be displayed in descending order of the degree of match with the second feature information. Also, the number of candidates displayed in the search result display unit 210 and the authentication result display unit 220 may be settable in a setting menu (not shown) or the like, respectively. Further, all candidates whose degree of match is equal to or higher than a predetermined reference value may be displayed by a list screen or scrolling. In this case, the predetermined reference value may be set in the setting menu.

[0068] Also, as shown in FIG. 14, the information processing apparatus 100 further includes an authentication result storage unit 116. The authentication result storage unit 116 may be integrated with the main body of the information processing apparatus 100 or may be separate. The authentication result storage unit 116 may be included in the same apparatus as the second feature information storage unit 110 and the authentication information database 114, or may be included in different apparatuses from each other. The authentication result storage unit 116 may have a database structure.

[0069] FIG. 16 is a diagram showing an example of the data structure of the authentication result storage unit 116 of the present embodiment. For each candidate, the authentication result storage unit 116 registers in association with a candidate ID, an authentication result ID for identifying result information, a degree of match of face authentication, and an authentication information ID indicating the original authentication information from which the result information was obtained.

[0070] FIG. 17 is a flowchart showing an example of the operation of the information processing apparatus 100 of the present embodiment. First, after executing the authentication process (step S120) described with reference to FIG. 11, the acquisition unit 122 stores the information about the person obtained by the authentication process as result information in the authentication result storage unit 116 (step S127). Then, the display processing unit 130 causes the display device 132 to display the acquired information (result list screen 200 of FIG. 15) (step S130).

[0071] FIG. 18 is a flowchart showing an example of the detailed procedure of the display process in step S130 of FIG. 17. First, the acquisition unit 122 acquires an image of the person to be searched (second image 22) from the first feature information storage unit 112 (step S131). Further, the acquisition unit 122 acquires a candidate image (first image 10) including a region with a degree of coincidence equal to or higher than the reference and a face region from the second feature information storage unit 110 (step S133). Then, the acquisition unit 122 acquires the result information from the authentication result storage unit 116 (step S135).

[0072] Then, the display processing unit 130 displays the second image 22 acquired in step S131 on the search target display unit 202 of the result list screen 200, displays the candidate image acquired in step S133 on the image display unit 214 of the search result display unit 210, and displays the result information acquired in step S135 on the authentication result display unit 220 (step S137).

[0073] FIG. 19 is a diagram showing details of the display example of the candidate display unit 212 in FIG. 15. The candidate display unit 212 includes an image display unit 214, a candidate ranking display unit 215, a degree of coincidence display unit 216, and a shooting location display unit 218. An image including the person region 16 of the candidate is displayed on the image display unit 214, and a frame surrounding the person region 16 and a frame surrounding the face region 20 may be respectively displayed.

[0074] FIG. 20 is a diagram showing details of the display example of the candidate display unit 222 in FIG. 15. The candidate display unit 222 includes an image display unit 224, a candidate ranking display unit 225, a degree of coincidence display unit 226, a name display unit 228, a candidate attribute information display unit 229, and a detailed display button 230. The face image of the person detected by the face authentication process is displayed on the image display unit 224. The name of the person is displayed on the name display unit 228. At least one of the face image of the person detected by the face authentication process and the identification information is displayed on the candidate display unit 222. Note that the identification information of the person may be the identification information (authentication ID in FIG. 10) in the authentication information database 114, or may be at least one of the name and the attribute.

[0075] The candidate attribute information display unit 229 displays the attribute information of the person detected by the face authentication process, such as gender and age. The detailed display button 230 is an operation button for displaying further attribute information of the person. When the pressing of the detailed display button 230 is accepted, the detailed information window 240 in FIG. 21 is displayed. The detailed information displayed in the detailed information window 240 may be registered in the authentication information database 114, or may be stored in another storage device in association with the authentication ID. In the authentication result storage unit 116, information for reaching this detailed information is stored as part of the result information.

[0076] As described above, in the present embodiment, by the display processing unit 130, the image of the search target person (second image 22), the image of the person area of the candidate detected by the collation process between the first feature information 14a (person area feature information) and the first image 10 captured by the camera 5, and the candidate detected by the collation process using the authentication information database 114 for the face area included in the person area The images and information are displayed on the same result list screen 200.

[0077] Thus, according to the present embodiment, based on the image of the detection target person (second image 22) in which the face is not shown, the image of the candidate is detected and displayed from the first image 10 captured by the camera 5, and further, the face area in the image of the candidate The face image and attribute information of the person registered in the authentication information database 114 can be viewed on the same result list screen 200.

[0078] (Fourth Embodiment) In the above embodiment, the configuration in which the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 are simultaneously displayed on the result list screen 200 has been described. The information processing apparatus 100 of the present embodiment can display the images of at least one candidate on the search target display unit 202 and the candidate display unit 212, or can sequentially switch and display the candidates in order of rank, and further, display the candidate display unit 222 indicating the face authentication result for the selected candidate.

[0079] Figs. 22 to 24 are diagrams showing an example of the procedure of the display process of the information processing apparatus 100 according to the present embodiment. Further, Fig. 25 is a diagram for explaining the transition of the display screen. Hereinafter, the display process of the information processing apparatus 100 will be described with reference to these diagrams.

[0080] First, the display processing unit 130 receives the designation of the search target person from the second image 22 and displays it on the search target display unit 202 of the screen 300 in Fig. 25(a) (step S201). For example, when the second image 22 includes a plurality of persons, the operator may operate and display a frame for designating the person area 16 of the target person on the second image 22. The screen 300 further includes a search button 302 and a cancel button 304.

[0081] Also, in the example of this figure, although there is one search target person, when the collation process is performed using the first feature information 14a of a plurality of search target persons shown in a plurality of second images 22, a plurality of images of the plurality of search target persons may be displayed on the search target display unit 202. At least one candidate display unit 212 may be displayed for each search candidate, or information on a plurality of candidates obtained as a result of the collation process using the first feature information 14a of a plurality of search target persons may be displayed on the candidate display unit 212.

[0082] When the pressing of the search button 302 is received (YES in step S203), the information processing apparatus 100 executes the search process described with reference to Fig. 8 (step S110). During the search process, a message 306 indicating that the search is in progress is displayed (step S205). When the information processing apparatus 100 receives the pressing of the cancel button 304 during the search process, it aborts the detection process and returns to step S201 (not shown).

[0083] If no candidate is found as a result of the search process (NO in step S207), the screen 310 in Fig. 25(e) is displayed. This screen 310 includes a message 312 indicating that no candidate has been found. Further, the screen 310 may include a re-search button 314. When the pressing of the re-search button 314 is received, the process returns to step S201, and a message prompting re-selection of the second image 22 of the person to be searched may be displayed, and re-selection may be received (not shown).

[0084] In this embodiment, it is assumed that the display method of the candidates can be set in advance by a setting menu (not shown) or the like. In the setting menu, it is possible to select whether to display the detected candidates one by one or to display all the detected candidates (in groups of a predetermined number (for example, in groups of the top three)). When at least one candidate is found (YES in step S207), the information processing apparatus 100 switches the display method of the candidates according to the setting (step S211). For example, when all (in this example, the top three) are selected, the process proceeds to the authentication process in step S120, and face authentication processing is performed on the face regions of each candidate. Then, a result list screen 200 (Fig. 15) showing the results of the face authentication processing is displayed (step S213).

[0085] When the setting of sequentially displaying candidates one by one is selected (sequentially in step S211), the information processing apparatus 100 proceeds to step S221 (Fig. 23). First, 1 is set in the counter i (step S221). Then, when i is 1 or less, as shown in Fig. 25(b), a next button 332 for displaying the next candidate is displayed on the screen 330, and the previous button for displaying the previous candidate is not displayed (step S225). On the screen 330, a search target display section 202 and a candidate display section 212 for the first candidate are displayed (step S233). Further, a search button 334 for receiving face authentication processing of the candidate is displayed on the screen 330.

[0086] Processing is switched according to the type of operation button pressed on screen 330 (step S235). When the press of the next button 332 is accepted, i is incremented and becomes 2 (step S243), and the process returns to step S223. Since i is 2, it becomes NO in step S223, and the previous button 342 is further displayed on screen 340 (step S227). Here, assuming that the maximum value of the number of candidate display persons is set to 3, since i > the maximum value imax - 1 (2) is not true (NO in step S229), the process proceeds to step S233, and the candidate display section 212 of the second candidate is displayed on screen 340 of FIG. 25(c) (step S233).

[0087] When the previous button 342 is pressed (before step S235), i is decremented and becomes 1 (step S241), and the process returns to step S223. And since i ≤ 1, the process proceeds to step S225, the next button 332 is displayed, and the previous button 342 is made non-displayed. And the candidate display section 212 of the first candidate is displayed on screen 330 (FIG. 25(b)) (step S235).

[0088] When the search button 334 is pressed on screen 330 of FIG. 25(b) (search in step S235), face authentication processing is performed on the face region of the first candidate (step S120). And when the result information is acquired, the result screen 350 of FIG. 25(d) is displayed (step S237). On the result screen 350, a search target display section 202, a candidate display section 212, and a candidate display section 222 are displayed, and further, a change button 352 for executing face authentication processing of another candidate may be displayed.

[0089] When the change button 352 is pressed (YES in step S239), the process returns to step S223. Here, since i is 1, screen 330 of FIG. 25(b) is displayed. And when the next button 332 is selected, the second candidate is displayed (step S233), and when the search button 334 is selected (search in step S235), face authentication processing is performed on the second candidate (step S120).

[0090] Also, as another example, instead of displaying candidates one by one, a predetermined number of candidates (for example, the top three) may be displayed, and a person who performs face authentication processing may be selected from among the candidates. An example of the procedure of the display process is shown in the flowchart of FIG. 24. When candidates are detected in step S207 of FIG. 22 (YES in step S207), the screen 360 in FIG. 25(f) is displayed (step S251). The screen 360 includes an authentication result display unit 220 and a search result display unit 210 including a candidate display unit 212 for a plurality (three in this example) of candidates. A search button 362 for accepting selection of a candidate who performs face authentication processing is displayed on the candidate display unit 212.

[0091] When the search button 362 is pressed (YES in step S253), face authentication processing is executed by the authentication unit 120 for the selected candidate (step S120). Then, the result of the face authentication processing is displayed on the screen 350 in FIG. 25(d). On the screen 350, an image of the candidate selected as the target of the face authentication processing is displayed on the candidate display unit 212.

[0092] As described above, in the present embodiment, the display processing unit 130 can display the screen 320 or the screen 360 including the search target display unit 202 and the search result display unit 210. And on the screen 320 or the screen 360, an operator can select a candidate who performs face authentication processing and perform face authentication processing. Therefore, face authentication processing can be omitted for clearly different persons, improving efficiency. Also, since the display content can be changed according to the size of the display, etc., a screen display that is easy to see becomes possible.

[0093] FIG. 26 is a diagram showing an example of the configuration of a computer 80 that realizes the information processing apparatus 100 of each of the above-described embodiments. The computer 80 includes a CPU (Central Processing Unit) 82, a memory 84, a program 90 that realizes the components of each information processing device 100 in FIGS. 2, 9, and 11 loaded in the memory 84, a storage 85 that stores the program 90, an I / O (Input / Output) 86, and a network connection interface (communication I / F 87).

[0094] The CPU 82, the memory 84, the storage 85, the I / O 86, and the communication I / F 87 are connected to each other via a bus 89, and the entire information processing device is controlled by the CPU 82. However, the method of connecting the CPU 82 and the like to each other is not limited to bus connection.

[0095] The memory 84 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 85 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card.

[0096] The storage 85 may be a memory such as a RAM or a ROM. The storage 85 may be provided inside the computer 80, or may be provided outside the computer 80 and connected to the computer 80 by wire or wirelessly as long as the computer 80 can access it. Alternatively, it may be provided detachably to the computer 80.

[0097] By the CPU 82 reading out and executing the program 90 stored in the storage 85 into the memory 84, each function of each unit of the information processing device 100 of each embodiment can be realized.

[0098] I / O 86 controls the input / output of data and control signals between computer 80 and other input / output devices. The other input / output devices include, for example, input devices (not shown) such as a keyboard, a touch panel, a mouse, and a microphone connected to computer 80, output devices (not shown) such as a display, a printer, and a speaker, and an interface between these input / output devices and computer 80. Further, I / O 86 may control the input / output of data with a reading or writing device (not shown) for other recording media.

[0099] Communication I / F 87 is a network connection interface for performing communication between computer 80 and an external device. Communication I / F 87 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, computer 80 that realizes information processing apparatus 100 is connected to at least one camera 5 via a network by communication I / F 87.

[0100] Each component of information processing apparatus 100 of each embodiment is realized by an arbitrary combination of the hardware and software of computer 80 in this figure. And it is understood by those skilled in the art that there are various modifications to the realization method and apparatus. The functional block diagram showing information processing apparatus 100 of each of the above-described embodiments shows blocks of logical function units, not a configuration in hardware units.

[0101] Information processing apparatus 100 may be constituted by a plurality of computers 80 or may be realized by a virtual server.

[0102] As an example constituted by a plurality of computers 80, the present invention can be realized as an information processing system exemplified below, but is not limited thereto. (1) Extraction unit 104 and collation unit 102 are implemented by different devices (computers 80). For example, the information processing system may include a terminal device having extraction unit 104 and a server device having collation unit 102. (2) The storage means for realizing the second feature information storage unit 110 and the first feature information storage unit 112 may be realized by a plurality of devices. For example, a storage device that stores the second feature information extracted from the person included in the first image and a storage device that stores the identification information of a specific person and the second feature information may exist separately. (3) Among the extraction processes of the extraction unit 104, the extraction process of the feature information from the first image 10 and the extraction process of the feature information from the second image 22 are performed by different devices (computer 80). For example, the information processing system may include a plurality of terminal devices that analyze the video having the extraction unit 104 and a server device that integrates the information from them and performs the collation process. The plurality of terminal devices are distributed by area, and the extraction unit 104 may be realized by a plurality of physical devices. (4) The types of feature information (face feature information and person area feature information) to be the target of video analysis (extraction unit 104 and collation unit 102) may be divided into different devices (computer 80).

[0103] As described above, the embodiments of the present invention have been described with reference to the drawings, but these are examples of the present invention, and various configurations other than the above can also be adopted. For example, in the above embodiment, the first collation process between the feature information extracted from the person area of the person included in the first image 10 and the first feature information 14a of the search target person, and the second feature information extracted from the face area of the person in the first image 10 that matched in the collation process and the face feature information of the registrant were performed. In other embodiments, using the first feature information 14a and the second feature information, the collation unit 102 may cause at least one of the first collation process and the second collation process to be performed on a third image different from the first image 10 and the second image 22.

[0104] In this configuration, when a face region can be detected from the third image, instead of performing the first matching process with the first feature information 14a, the second matching process with the second feature information can be performed. Also, when a face region cannot be detected from the third image, the first matching process with the first feature information 14a may be performed. Then, the newly generated second feature information obtained by the first matching process may be stored in the second feature information storage unit 110.

[0105] According to this configuration, for example, for a third image captured at a different date and time or location from the first image 10, by performing a matching process using the already acquired first feature information 14a or second feature information, it is possible to search for the target person in the third image as well.

[0106] Also, in the above embodiment, the first feature information is the feature information extracted from the first part of the person region, and the second feature information is the feature information extracted from the second part of the person region, particularly the face region, as an example. However, other combinations of the first feature information and the second feature information are also conceivable.

[0107] As a first example, the first feature information may be information indicating at least any one of the color of a person's clothing, age group, gender, and height, and the second feature information may be the face feature information of the person.

[0108] In this example, for instance, when face authentication cannot be performed because the face of the target person is not shown in the second image 22, instead of performing the process of extracting the first feature information 14a from the person region of the second image 22, at least any one of the color of the target person's clothing, age group, gender, and height may be specified by an operator's operation and stored in the first feature information storage unit 112 as the first feature information 14a. In this example, an operation reception unit (not shown) for receiving the input of the first feature information of the target person may be provided on the screen together with the search target display unit 202 that displays the second image 22. The operator can input the first feature information of the target person while looking at the image of the target person displayed on the search target display unit 202.

[0109] As a second example, the first feature information may be feature information of the entire human region, and the second feature information may be a plurality of biometric authentication information for performing multimodal biometric authentication processing by combining a plurality of biometric information.

[0110] In this example, for example, when it is difficult to perform face authentication because the face of the person to be searched for is not shown in the second image 22, and it is also difficult to perform gait authentication or the like because the lower body is not shown, it can be applied. The first feature information 14a is extracted from the human region of the person shown in the second image 22, candidates are detected for the first image 10 using the first feature information 14a, and for the detected candidates, the person may be identified by multimodal biometric authentication processing using a plurality of biometric information as the second feature information.

[0111] As described above, the present invention has been described with reference to the embodiments and examples, but the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In addition, when acquiring and using information regarding the user in the present invention, it shall be performed legally.

[0112] Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto. 1. Matching means for matching first feature information indicating the features of a person to be searched for with first feature information extracted from a person included in a first image; Extracting means for extracting second feature information from the person included in the first image when the matching result by the matching means is a match; Registration means for storing the second feature information extracted from the person included in the first image in a storage means; An information processing apparatus comprising: 2. In the information processing apparatus according to 1., The matching means: Matches the second feature information stored in the storage means associated with the identification information of a specific person with the second feature information extracted from the person included in the first image; An information processing apparatus comprising output processing means for causing the output means to output the identification information stored in the storage means when the verification result matches. 3. In the information processing apparatus according to 1. or 2., The information processing apparatus, wherein the first feature information is generated using a second image different from the first image. 4. In the information processing apparatus according to 3., The extraction means extracts a plurality of the first feature information from a person region of a person included in a plurality of different second images, The registration means stores the plurality of first feature information in the storage means in association with each other as information of the same person, The verification means performs the verification process using the plurality of first feature information stored in the storage means. 5. In the information processing apparatus according to 3. or 4., The information processing apparatus, wherein the second image is generated by an imaging unit different from the first image. 6. In the information processing apparatus according to any one of 1. to 5., The verification means causes at least one of a verification process with the first feature information and a verification process with the second feature information to be performed on a third image. 7. In the information processing apparatus according to any one of 1. to 6., The registration means stores, in the storage means, at least a region corresponding to the first feature information in the first image in association with the second feature information. 8. In the information processing apparatus according to any one of 1. to 7., The registration means stores the imaging conditions of the first image in association with the second feature information. 9. In the information processing apparatus according to any one of 1. to 8., The information processing apparatus, wherein the second feature information is feature information of a person's face, and the first feature information is feature information including a region other than the person's face.

[0113] 10. Matching means for matching first feature information indicating the characteristics of a person to be searched for with first feature information extracted from a person included in a first image; Extracting means for extracting second feature information from the person included in the first image when the matching result by the matching means is a match; Registration means for storing the second feature information extracted from the person included in the first image in storage means; An information processing system comprising: 11. In the information processing system according to 10., The matching means: Matches the second feature information stored in the storage means associated with the identification information of a specific person with the second feature information extracted from the person included in the first image, An information processing system comprising output processing means for outputting the identification information stored in the storage means to output means when the matching result is a match. 12. In the information processing system according to 10. or 11., The first feature information is information generated using a second image different from the first image. 13. In the information processing system according to 12., The extraction means extracts a plurality of the first feature information from the human regions of the persons included in a plurality of different second images, The registration means stores the plurality of first feature information in the storage means in association with each other as information of the same person, The matching means performs the matching process using the plurality of first feature information stored in the storage means. 14. In the information processing system according to 12. or 13., The second image is generated by an imaging unit different from the first image. 15. In the information processing system according to any one of 10. to 14., The matching means is an information processing system that causes at least one of a matching process with the first feature information and a matching process with the second feature information to be performed on the third image. 16. In the information processing system according to any one of 10. to 15., The registration means is an information processing system that associates at least a region corresponding to the first feature information in the first image with the second feature information and stores it in the storage means. 17. In the information processing system according to any one of 10. to 16., The registration means is an information processing system that stores the imaging conditions of the first image in association with the second feature information. 18. In the information processing system according to any one of 10. to 17., The second feature information is feature information of a person's face, and the first feature information is feature information including a region other than the person's face.

[0114] 19. An information processing apparatus compares first feature information indicating the features of a person to be searched for with first feature information extracted from a person included in a first image, when the comparison result matches, extracts second feature information from the person included in the first image, and stores the second feature information extracted from the person included in the first image in a storage unit. Information processing method. 20. In the information processing method according to 19., the information processing apparatus compares the second feature information stored in the storage unit in association with the identification information of a specific person with the second feature information extracted from the person included in the first image, and when the comparison result matches, causes the identification information stored in the storage unit to be output to an output unit. 21. In the information processing method according to 19. or 20., the first feature information is generated using a second image different from the first image. 22. In the information processing method according to 21., wherein the information processing apparatus extracts a plurality of pieces of the first feature information from human regions of persons included in a plurality of different second images, associates the plurality of pieces of the first feature information with each other as information of the same person and stores them in a storage unit, and performs the collation process using the plurality of pieces of the first feature information stored in the storage unit, an information processing method. 23. In the information processing method according to 21. or 22., wherein the second image is generated by an imaging unit different from the first image, an information processing method. 24. In the information processing method according to any one of 19. to 23., wherein the information processing apparatus causes at least one of a collation process with the first feature information and a collation process with the second feature information to be performed on a third image, an information processing method. 25. In the information processing method according to any one of 19. to 24., wherein the information processing apparatus associates at least a region corresponding to the first feature information in the first image with the second feature information and stores them in the storage unit, an information processing method. 26. In the information processing method according to any one of 19. to 25., wherein the information processing apparatus associates imaging conditions of the first image with the second feature information and stores them, an information processing method. 27. In the information processing method according to any one of 19. to 26., wherein the second feature information is feature information of a human face, and the first feature information is feature information including a region other than the human face, an information processing method.

[0115] 28. For a computer, a procedure for collating first feature information indicating features of a person to be searched for with first feature information extracted from a person included in a first image, When the verification result matches, a procedure for extracting second feature information from the person included in the first image A program for causing a computer to execute a procedure for storing, in a storage unit, second feature information extracted from the person included in the first image. 29. In the program according to 28., A procedure for comparing the second feature information stored in the storage unit in association with the identification information of a specific person with the second feature information extracted from the person included in the first image A program for causing a computer to execute a procedure for outputting, to an output unit, the identification information stored in the storage unit when the verification result matches. 30. In the program according to 28. or 29., The first feature information is generated using a second image different from the first image. 31. In the program according to 30., A procedure for extracting a plurality of the first feature information from the person regions of the persons included in a plurality of different second images A procedure for storing, in a storage unit, the plurality of first feature information in association with each other as information of the same person A program for causing a computer to execute a procedure for performing the verification process using the plurality of first feature information stored in the storage unit. 32. In the program according to 30. or 31., The second image is generated by an imaging unit different from the first image. 33. In the program according to any one of 28. to 32., A program for causing a computer to execute a procedure for performing at least one of a verification process with the first feature information and a verification process with the second feature information on a third image. 34. In the program according to any one of 28. to 33., A program for causing a computer to execute a procedure of associating, in the memory unit, at least a region corresponding to the first feature information in the first image with the second feature information and storing the result. 35. In the program according to any one of 28. to 34., A program for causing a computer to execute a procedure of associating and storing imaging conditions of the first image with the second feature information. 36. In the program according to any one of 28. to 35., A program in which the second feature information is feature information of a person's face, and the first feature information is feature information including a region other than the person's face.

Claims

1. an extraction means for extracting face information from an object included in a first image when a match is found between first feature information indicating features of a search object not including a face and the first feature information extracted from the object included in the first image; A registration means for storing the extracted face information in a first storage means; An information processing device comprising:

2. 2. The information processing device according to claim 1, An information processing device comprising: an output means for outputting a result of matching between registered face information stored in a second storage means in association with identification information of a specific target and the face information extracted by the extraction means.

3. 3. The information processing device according to claim 2, The output means outputs the identification information stored in the second storage means when the comparison result with the registered face information is a match.

4. 4. The information processing device according to claim 1, The extraction means extracts a plurality of pieces of first feature information from objects included in a plurality of second images different from the first image; the registration means stores the plurality of pieces of first characteristic information in a third storage means in association with each other as information of the same search target; The extraction means performs a matching process using the plurality of first feature information stored in the third storage means, and if the matching process results in a match, extracts facial information from the target included in the first image.

5. 5. The information processing device according to claim 4, An information processing apparatus in which the second image is generated by an imaging unit different from that of the first image.

6. An information processing device according to any one of claims 1 to 5; At least one imaging unit connected to the information processing device, The imaging unit captures an image.

7. An information processing device, extracting face information from the target included in the first image when a match is found between first feature information indicating features of the search target not including a face and first feature information extracted from the target included in the first image; storing the extracted face information in a first storage means; Information processing methods.

8. On the computer, extracting face information from the target included in the first image when a match is found between first feature information indicating features of the search target not including a face and the first feature information extracted from the target included in the first image; a step of storing the extracted face information in a first storage means; A program for executing the above.

Images