Person search device, person search system, person search method, and person search program

Abstract

To provide a person search device capable of highly accurately searching the images photographed by a plurality of cameras for an image including a specific person.SOLUTION: In a person search system, a person search device comprises: a similarity calculation unit which acquires a total body image feature amount and a face image feature amount of a person to be searched and calculates a first similarity of the total body image feature amount and a second similarity of the face image feature amount indicative of similarities in the database; a classification unit which determines whether the first similarity exceeds a first threshold value and the second similarity exceeds a second threshold value; a person search unit which calculates a region and a time zone using the results of the classification unit and photographic information exceeding the second threshold value and performs narrow-down processing using second photographic information exceeding the first threshold value and not exceeding the second threshold value with the region and the time zone as conditions; and a total body image feature amount correction unit which replaces the total body image feature amount with the total body image feature amount associated with the face image feature amount when the difference value between the total body image feature amount associated with the face image feature amount exceeding the second threshold value and the total body image feature amount exceeds the third threshold value.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to a person search device that searches for an image including a specific person from among a plurality of images. [Background technology]

[0002] Among the techniques for identifying a person included in an image, there is a technique for identifying a person using a feature amount of the person in the image. For example, Patent Document 1 describes a method of extracting a first feature related to the person's face and a second feature related to the person's body from an image including the person captured by an imaging unit (camera), and identifying the person based on a third feature calculated by weighting the extracted first feature and second feature respectively. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-23785 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when the multiple images are images taken by multiple cameras, for example, the images include images taken under different shooting conditions, and the feature amounts of people in the images often differ depending on the shooting conditions. In such cases, conventional technology has the problem that there is variation in the accuracy of identifying people, which tends to reduce the accuracy of searching for images containing a specific person from among images taken with multiple cameras. The person identification device described in Patent Document 1 has difficulty in determining optimal weighting so as to match the shooting conditions of all of the multiple cameras, and thus is still unable to solve the above-mentioned problem.

[0005] An object of the present disclosure is to provide a person search device that searches for images including a specific person from among images captured by a plurality of cameras with high accuracy. [Means for solving the problem]

[0006] A person search device of the present disclosure is a person search device that searches for images including a search target person from among multiple images captured by multiple cameras, and includes a similarity calculation unit that, when acquiring whole-body image features and face image features of the person in a specific person image indicating the search target person, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, calculates a first similarity indicating the similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating the similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold, and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects the whole-body image features of the specific person image; and calculates a first similarity indicating the similarity between the whole-body image features of the specific person image and the whole-body image features in the database using a determination result from the classification unit. and a person search unit that acquires shooting information linked to the facial image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires from the database second shooting information linked to the whole-body image feature whose first similarity exceeds the first threshold and the facial image feature whose second similarity does not exceed the second threshold, and narrows down the acquired second shooting information using the area and time period as conditions to generate specific person shooting information. The whole-body image feature correction unit is configured to acquire from the database the whole-body image feature linked to the facial image feature whose second similarity exceeds the second threshold using the judgment result of the classification unit, and if a difference value between the acquired whole-body image feature and the whole-body image feature of the specific person image exceeds a third threshold, replaces the whole-body image feature of the specific person image with the whole-body image feature linked to the facial image feature whose second similarity exceeds the second threshold. Effect of the Invention

[0007] According to the present disclosure, it is possible to provide an effect of providing a person search device that searches for images including a specific person with high accuracy from among images captured by a plurality of cameras. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a person search system including a person search device according to the present disclosure. [Diagram 2] 1 is a diagram illustrating a configuration of a person search system including a person search device according to a first embodiment of the present disclosure. [Diagram 3] FIG. 2 is a diagram showing an example of an image used in the present disclosure. [Figure 4] 4 is a flowchart showing a data generation process in the person retrieval device according to the first embodiment of the present disclosure. [Diagram 5] 4 is a flowchart showing a person search process in the person search device according to the first embodiment of the present disclosure. [Figure 6] 8 is a flowchart showing detailed processing in ST180 in FIG. 7. [Figure 7] FIG. 2 is a diagram illustrating a first modified example of the configuration of a person search system including a person search device according to the first embodiment of the present disclosure. [Figure 8] FIG. 11 is a diagram illustrating a second modified example of the configuration of the person search system including the person search device according to the first embodiment of the present disclosure. [Figure 9] FIG. 11 is a diagram illustrating a configuration of a person search system including a person search device according to a second embodiment of the present disclosure. [Figure 10] 13 is a flowchart showing a correction value update process in the person search device according to the second embodiment of the present disclosure. [Figure 11] 13 is a flowchart showing a process of extracting an image using whole-body image features in a person retrieval device according to a second embodiment of the present disclosure. [Figure 12] FIG. 11 is a diagram illustrating a configuration of a person search system including a person search device according to a third embodiment of the present disclosure. [Figure 13] FIG. 2 is a diagram illustrating a configuration of a threshold value determining unit in the present disclosure. [Figure 14] 5 is a flowchart showing a first threshold adjustment process in the present disclosure. [Figure 15] 13 is a flowchart showing a second threshold adjustment process in the present disclosure. [Figure 16] 1 is a diagram illustrating a first example of a hardware configuration of a person retrieval device according to the present disclosure. [Figure 17] FIG. 13 is a diagram illustrating a second example of a hardware configuration of the person retrieval device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings. Embodiment 1 FIG. 1 is a diagram showing a person retrieval system 1 including a person retrieval device 100 according to the present disclosure. FIG. 2 is a diagram showing a configuration of a person search system 1 including a person search device 100 according to the first embodiment of the present disclosure.

[0010] The person retrieval system 1 shown in FIG. 1 includes a person retrieval device 100, a database 200, a monitoring terminal 300, a camera 400, and an image recording device 500. In the person retrieval system 1, a person retrieval device 100 is communicably connected to a database 200, a monitoring terminal 300, a camera 400, and an image recording device 500. Moreover, the monitoring terminal 300, the camera 400, and the image recording device 500 are connected so as to be able to communicate with each other. The person search system 1 is a system that searches for an image including a person to be searched for, from among a plurality of images taken by a plurality of cameras 400 (400-n: n is any integer equal to or greater than 1).

[0011] 2, the person search system 1 includes a person search device 100 and a database 200 connected by a communication line or the like, and the person search device 100, a monitoring terminal 300, a camera 400, and an image recording device are mutually connected by a communication network 600 such as an IP network. The database 200 may also be configured to be connected using the communication network 600.

[0012] The person search device 100 searches a plurality of images captured by a plurality of cameras 400 for an image including a person to be searched for. Furthermore, person retrieval device 100 extracts whole-body image features and face image features of a person included in an image captured by each camera 400, and stores the extracted features in database 200 in association with shooting information indicating the shooting conditions of the image.

[0013] The person search device 100 receives, for example, a specific person image including a person to be searched for and a search request from the monitoring terminal 300 via the communication network 600. Furthermore, when the person search device 100 acquires the whole body image feature amount and the face image feature amount of the person in the specific person image showing the person to be searched, the person search device 100 refers to the database 200 to calculate a first similarity indicating the similarity between the whole body image feature amount of the specific person image and the whole body image feature amount in the database 200, and calculates a second similarity indicating the similarity between the face image feature amount of the specific person image and the face image feature amount in the database 200. Furthermore, the person search device 100 determines whether the first similarity exceeds a first threshold value, and determines whether the second similarity exceeds a second threshold value. Furthermore, the person search device 100 acquires, from the database 200, the shooting information linked to the face image feature amount whose second similarity exceeds the second threshold value, using the judgment result, and calculates the area and time period in which the person shown in the specific person image may have been photographed, using the acquired shooting information, and generates specific person shooting information including the camera identification information in the area and the time period. The person search device 100 outputs, for example, the specific person photographed information as a search result to the monitoring terminal 300 via the communication network 600. The search result may include an image identified in the specific person photographed information. The person retrieval device 100 will be described in detail later.

[0014] In the database 200, for each image output from the multiple cameras 400, whole body image features, face image features, and shooting information 270 including camera identification information, shooting position, and shooting date and time are stored in association with each other. More specifically, the database 200 has a program 210, whole-body information 220, face information 230, a correction value 240, a whole-body similarity determination threshold (first threshold) 250, a face similarity determination threshold (second threshold) 260, and shooting information 270. The program 210 is a program for causing a computer to operate as the person retrieval device 100 . The whole-body information 220 includes a whole-body image feature amount for each image and a person's identification number. The whole-body image feature amount is a value that quantifies features such as clothing, physique, and camera angle of view, and indicates the whole-body features of each person. The face information 230 includes face image features for each image and a person's identification number. The face image features are values ​​obtained by digitizing the face features for each person. The correction value 240 is set according to the photographing conditions for each camera, and is used when the whole-body image feature amount extraction unit 140 described later extracts the whole-body image feature amount. The whole-body similarity determination threshold (first threshold) 250 is used in the classification unit 180 described later when determining classification using a first similarity indicating the similarity between the whole-body image feature amount of a specific person image and the whole-body image feature amount of the database 200. The facial similarity determination threshold (second threshold) 260 is used in the classification unit 180 described later when determining classification using a second similarity indicating the similarity between the facial image feature amount of a specific person image and the facial image feature amount of the database 200. The shooting information 270 indicates the shooting conditions for each camera, and includes at least the camera identification information, the shooting position, and the shooting date and time. The camera identification information is, for example, a camera number that is different for each camera. In the database 200, when a command to store information is received, new information is added or the above information is updated. Furthermore, the database 200 presents the stored data in response to a request from the person retrieval device 100. The database 200 may be configured to be included inside the person retrieval device 100.

[0015] The monitoring terminal 300 requests the person search device 100 to search for an image in which the same person as the person shown in the specific person image indicating the person to be searched for is captured. Furthermore, the monitoring terminal 300 is configured to be able to acquire images captured by the camera 400 via the communication network 600 . Furthermore, the monitoring terminal 300 is configured to be able to acquire the images recorded in the image recording device 500 via the communication network 600 . Furthermore, the monitoring terminal 300 is configured so as to be able to acquire images from an external source other than the camera 400 and the image recording device 500 . The specific person image is an image designated from among the images acquired by the monitoring terminal 300.

[0016] After making a request for an image search, the monitoring terminal 300 receives the search results from the person search device 100 . The search result is, for example, specific person photographing information generated in the person search device 100. In this case, the monitoring terminal 300 acquires an image from the camera 400 or the image recording device 500 using the specific person photographing information. However, when the search result includes an image, the monitoring terminal 300 does not need to perform processing using the specific person photographing information.

[0017] The camera 400 indicates a plurality of cameras 400-1, 400-2, 400-3, . . . , 400-n, each of which captures an image and outputs the image. The multiple cameras 400 are, for example, surveillance cameras that are installed so as to capture different target areas. Images captured by multiple cameras are assigned a camera identification number that identifies each camera 400 and a capture date and time. Furthermore, information indicating the capture location may also be added.

[0018] Image recording device 500 records the image, camera identification information, and shooting date and time output from camera 400. If the image includes information indicating the shooting location, the information indicating the shooting location may be recorded.

[0019] The person retrieval device 100 will now be described in detail. The person search device 100 includes a camera image acquisition unit 110, a whole-body image extraction unit 120, a facial image extraction unit 130, a whole-body image feature extraction unit 140, a facial image feature extraction unit 150, a person feature storage unit 160, a similarity calculation unit 170, a classification unit 180, a person search unit 190, and a control unit (not shown).

[0020] The camera image acquisition unit 110 acquires images (camera images) captured by a plurality of cameras 400 and capture information. Specifically, the camera image acquisition section 110 receives camera images distributed from a plurality of cameras 400 at a constant frame rate, and outputs the received camera images to the whole-body image extraction section 120 and the face image extraction section .

[0021] Images used in this disclosure will be described. FIG. 3 is a diagram showing an example of an image used in the present disclosure. An image 700 shown in FIG. 3 includes a whole-body image 701 (701a, 701b) for each person, and a face image 702 (702a, 702b) for each person. However, while image 700 includes two people, the number of people included in each image varies from image to image. In the image 700, the whole body image 701 shows the whole body from head to toe, but is not limited to this. For example, the whole body image 701 may be an image from which bodily features can be extracted, as compared with the face image 702, and may be an image including a part of the whole body other than the face.

[0022] The whole-body image extraction unit 120 receives an image and extracts a whole-body image 701 of a person from the received image. The whole-body image extraction unit 120 extracts a whole-body image 701 of a person from the camera image acquired by the camera image acquisition unit 110, for example. Further, the whole-body image extraction unit 120 receives a specific person image from the monitoring terminal 300, for example, and extracts a whole-body image 701 of a person from the received specific person image. For example, when the whole-body image extraction unit 120 receives an image, it extracts the whole-body area of ​​the person based on an image of the person that has been learned in advance using machine learning such as deep learning, and outputs an image of the whole-body area (whole-body image) to the whole-body image feature extraction unit 140.

[0023] The face image extraction unit 130 receives an image and extracts a face image 702 from the received image. The face image extraction unit 130 receives, for example, a camera image from the camera image acquisition unit 110, and extracts a face image 702 of a person from the received image. Furthermore, the face image extraction unit 130 receives a specific person image from the monitoring terminal 300, for example, and extracts a person's face image 702 from the received specific person image. For example, when the facial image extraction unit 130 receives an image, it extracts a person's facial area based on a facial image that has been learned in advance using machine learning such as deep learning, and outputs an image of the facial area (facial image) to the facial image feature extraction unit 150. The face image extraction unit 130 may be configured to receive the whole-body image 701 extracted by the whole-body image extraction unit 120. In this case, the face image extraction unit 130 extracts a face image 702 from the whole-body image 701.

[0024] The whole-body image feature amount extraction section 140 extracts whole-body image feature amounts from the whole-body image extracted by the whole-body image extraction section 120 . Specifically, the whole-body image feature extraction unit 140 extracts whole-body image feature amounts from the whole-body image of the camera image acquired by the camera image acquisition unit 110, and outputs the whole-body image feature amounts to the person feature storage unit 160. In this case, when the whole-body image feature extraction unit 140 receives the whole-body image, it extracts whole-body image feature amounts (X1) that digitize features due to clothing, physique, camera angle of view, and the like, based on an image of a person that has been learned in advance by machine learning such as deep learning, and outputs the whole-body image feature amount of the camera image to the person feature storage unit 160. Specifically, the whole-body image feature extraction unit 140 extracts whole-body image feature values ​​from the whole-body image of the specific person image and outputs the whole-body image feature values ​​to the similarity calculation unit 170. In this case, when the whole-body image of the specific person image is received, the whole-body image feature extraction unit 140 extracts whole-body image feature values ​​(X1) that digitize features such as clothing, physique, and camera angle of view based on an image of a person that has been learned in advance by machine learning such as deep learning, and outputs the whole-body image feature values ​​of the specific person image to the similarity calculation unit 170.

[0025] The facial image feature amount extraction unit 150 extracts facial image feature amounts from the facial image extracted by the facial image extraction unit 130 . Specifically, the facial image feature extraction unit 150 extracts facial image feature amounts from the facial image of the camera image acquired by the camera image acquisition unit 110, and outputs the facial image feature amounts to the person feature storage unit 160. In this case, when the facial image is received, the facial image feature extraction unit 150 extracts facial image feature amounts (Y1) that quantifies the facial features based on a facial image that has been learned in advance by machine learning such as deep learning, and outputs the facial image feature amount of the camera image to the person feature storage unit 160. Specifically, the facial image feature extraction unit 150 extracts facial image feature amounts from the facial image of the specific person image and outputs the facial image feature amounts to the similarity calculation unit 170. In this case, when the facial image is received, the facial image feature extraction unit 150 extracts facial image feature amounts (Y1) that quantifies facial features based on a facial image that has been learned in advance by machine learning such as deep learning, and outputs the facial image feature amounts of the specific person image to the similarity calculation unit 170.

[0026] The person feature storage unit 160 acquires the whole-body image features and facial image features of the images captured by each camera, as well as the shooting information of the images, and stores the acquired whole-body image features, facial image features, and shooting information in the database 200 in association with each other. Specifically, the person feature storage unit 160 links the whole-body image features and facial image features extracted from the same image, and also stores the camera number, shooting position, shooting date and time, thumbnail images cut out from the whole-body image and facial image, etc.

[0027] When the similarity calculation unit 170 acquires the whole-body image features and face image features of a person in a specific person image indicating the person to be searched for, it refers to the database 200 to calculate a first similarity indicating the similarity between the whole-body image features of the specific person image and the whole-body image features in the database 200 (similarity of the whole-body images), and calculates a second similarity indicating the similarity between the face image features of the specific person image and the face image features in the database 200 (similarity of the face images). The similarity indicates the degree to which the compared images are similar, and can take values ​​ranging from 0 to 1, for example, with the closer to 1 the similarity is, the higher the similarity.

[0028] The classification unit 180 determines whether the first similarity exceeds a first threshold, and determines whether the second similarity exceeds a second threshold. The classification unit 180 classifies the set of the whole body image feature amount, the face image feature amount, the photographing information, and the thumbnail image using the judgment result, and outputs the set information (set information) together with the classification information indicating the classification according to the classification result. The set information includes the whole body image feature amount, the face image feature amount, the photographing information, and the thumbnail image, as well as the classification information. Specifically, the classification unit 180 uses the determination result to classify the images into the following classifications 1 to 4. (Class 1) Both the whole-body image similarity and the face image similarity exceed a threshold. (Class 2) Only the facial image similarity exceeds the threshold (Class 3) Only the similarity of the whole-body image exceeds the threshold (Class 4) Both the whole-body image similarity and the face image similarity are below the threshold.

[0029] Using the judgment result of the classification unit 180, the person search unit 190 acquires from the database 200 shooting information linked to the facial image feature whose second similarity exceeds the second threshold, and uses the acquired shooting information to calculate the area and time period during which the person shown in the specific person image may have been photographed, and generates specific person shooting information including camera identification information within the area and the time period. Specifically, the person search section 190 receives a specific person image together with a search request from the monitoring terminal 300, and outputs the specific person image to the whole-body image extraction section 120 and the face image extraction section . After outputting to the whole-body image extraction unit 120 and the face image extraction unit 130, the person search unit 190 receives the information output from the classification unit 180 and notifies the monitoring terminal 300 of the search results using the classification information and set information.

[0030] More specifically, the person search unit 190 selects a certain number of set information items in order of the similarity (second similarity) of the face images of the sets of information items in the categories 1 and 2, for example. The certain number may be any number necessary as a search result, and is not particularly limited. In addition, the person search unit 190 uses the set information of categories 1 and 2 to obtain information such as camera positions, camera adjacency, and relative distances, and calculates the area and time period in which the person to be searched for, shown in the specific person image, may have been photographed. From among set information of category 3 other than categories 1 and 2, person search section 190 uses the calculated area and time period to select set information including camera identification information within the area and shooting date and time of the time period. The person search unit 190 selects a certain number of sets of selected information in descending order of similarity (first similarity) of the whole-body images. The person search section 190 uses the set information of the categories 1 and 2 and the selected set information of the category 3 to generate specific person photographing information including the camera identification information and the time period. The person search unit 190 outputs the specific person photographic information as a search result.

[0031] Here, person search section 190 may perform thinning so as not to select multiple images from the same camera within a certain period of time, in order to prevent search results from being biased toward a particular camera or time period. In this case, when generating specific person shooting information, if a plurality of pieces of camera identification information indicating the same camera are included, the person search section 190 thins out the camera identification information indicating the same camera at predetermined time intervals.

[0032] The person search unit 190 may further output, as a search result, the specific person shooting information and an image captured by the camera identified by the camera identification information of the specific person shooting information to the monitoring terminal 300 that requested the search.

[0033] The processing of the person retrieval device 100 will be described. First, a process of generating and storing data in the database 200 referred to by the person retrieval device 100 will be described. FIG. 4 is a flowchart showing the data generation process in the person retrieval device 100 according to the first embodiment of the present disclosure.

[0034] The person search device 100 starts processing when, for example, a control unit (not shown) receives a command to start processing. The camera image acquisition unit 110 acquires an image (step ST10). Specifically, the camera image acquisition unit 110 acquires images and shooting information captured by a plurality of cameras.

[0035] The whole-body image extraction section 120 receives the camera image from the camera image acquisition section 110, and extracts a whole-body image 701 of a person from the image acquired by the camera image acquisition section 110 (step ST20). The whole-body image extraction unit 120 outputs the whole-body image 701 to the whole-body image feature amount extraction unit 140 .

[0036] The whole-body image feature amount extraction section 140 extracts whole-body image feature amounts from the whole-body image extracted by the whole-body image extraction section 120 (step ST30). Specifically, the whole-body image feature extraction unit 140 extracts whole-body image feature amounts from the whole-body image of the camera image acquired by the camera image acquisition unit 110, and outputs the whole-body image feature amounts to the person feature storage unit 160. Upon receiving the whole-body image, the whole-body image feature extraction unit 140 extracts whole-body image feature amounts (X1) and outputs the whole-body image feature amounts of the camera image to the person feature storage unit 160. At this time, the whole-body image feature extraction unit 140 refers to the correction value for each camera in the database 200, and corrects the whole-body image feature by multiplying the whole-body image feature by the correction value (× correction value). Note that this correction may be performed on the similarity of the whole-body image feature, as described later.

[0037] The face image extraction unit 130 receives the camera image from the camera image acquisition unit 110, and extracts a face image 702 of a person from the received image (step ST40). In step ST40, the face image extraction unit 130 may receive a whole-body image extracted by the whole-body image extraction unit 120. The facial image extraction unit 130 outputs the facial image 702 to the facial image feature amount extraction unit 150 .

[0038] The facial image feature amount extraction unit 150 extracts facial image feature amounts (step ST50). The facial image feature amount extraction unit 150 extracts facial image feature amounts from the facial image extracted by the facial image extraction unit 130 . Specifically, facial image feature amount extraction unit 150 extracts facial image feature amounts from the facial image of the camera image acquired by camera image acquisition unit 110, and outputs the facial image feature amount to personal feature storage unit 160. Upon receiving a facial image, facial image feature amount extraction unit 150 extracts a facial image feature amount (Y1) and outputs the facial image feature amount to personal feature storage unit 160.

[0039] The personal feature storage unit 160 stores the feature amounts of the whole body image and the face image as a pair in the database 200 (step ST60). Specifically, the person feature storage unit 160 acquires the whole-body image features and facial image features of the images captured by each camera, as well as the shooting information of the images, and stores in the database 200 set information linking the acquired whole-body image features, facial image features, and shooting information.

[0040] Once the set information is stored in the database 200, the process waits until the next processing start.

[0041] Next, the person search process will be described. FIG. 5 is a flowchart showing the person search process in the person search device 100 according to the first embodiment of the present disclosure. The person search device 100 starts processing when, for example, a control unit (not shown) receives a command to start processing. The person search unit 190 receives a search request from the monitoring terminal 300, and acquires a specific person image including the person to be searched for (step ST110). The person search section 190 outputs the specific person image to the whole-body image extraction section 120 and the face image extraction section .

[0042] The whole-body image extraction unit 120 receives a specific person image from the monitoring terminal 300, for example, and extracts a whole-body image 701 of a person from the received specific person image (step ST120).

[0043] The whole-body image feature amount extraction unit 140 extracts whole-body image feature amounts (step ST130). Specifically, when the whole-body image feature extraction section 140 receives the whole-body image of the specific person image, it extracts the whole-body image feature (X1) and outputs the whole-body image feature of the specific person image to the similarity calculation section 170.

[0044] The similarity calculation unit 170 calculates the similarity between the whole-body image of the person to be searched and the whole-body images in the database 200 (step ST140). Specifically, when the similarity calculation unit 170 acquires the whole-body image features of a person in a specific person image indicating the person to be searched for, it refers to the database 200 and calculates a first similarity indicating the similarity between the whole-body image features of the specific person image and the whole-body image features stored in the database 200. At this time, the similarity calculation unit 170 refers to the correction value for each camera in the database 200, and corrects the first similarity by multiplying the first similarity by the correction value (×correction value). Note that this correction may be performed on the whole-body image feature amount described above.

[0045] The face image extraction unit 130 receives a specific person image from the monitoring terminal 300, for example, and extracts a person's face image 702 from the received specific person image (step ST150).

[0046] The facial image feature amount extraction unit 150 extracts facial image feature amounts (step ST160). Specifically, the facial image feature amount extraction unit 150 extracts facial image feature amounts from the facial image of the specific person image, and outputs the facial image feature amount to the similarity calculation unit 170. In this case, when the facial image is received, the facial image feature amount extraction unit 150 extracts, for example, a facial image feature amount (Y1), and outputs the facial image feature amount of the specific person image to the similarity calculation unit 170.

[0047] The similarity calculation section 170 calculates the similarity between the face image of the person to be searched and the face images in the database 200 (step ST170). Specifically, when the similarity calculation unit 170 acquires the whole-body image feature amount and the facial image feature amount of a person in a specific person image indicating the person to be searched, the similarity calculation unit 170 refers to the database 200 and calculates a second similarity indicating the similarity between the facial image feature amount of the specific person image and the facial image feature amount stored in the database 200. The similarity calculation unit 170 outputs to the classification unit 180 the set information used in the calculation together with the first similarity and the second similarity.

[0048] The classification unit 180 compares the similarity with a threshold and performs classification (step ST180). Specifically, the classification section 180 determines whether the first similarity exceeds a first threshold value, and determines whether the second similarity exceeds a second threshold value. The classification unit 180 uses the judgment result to classify the set of whole-body image features, facial image features, shooting information, and thumbnail images, and outputs set information (set information) together with classification information indicating classification 1 to 4 according to the classification result.

[0049] The person search section 190 uses the determination result of the classification section 180 to select search result 1 from category 1 and category 2 (step ST190).

[0050] The person search unit 190 selects images and shooting information in which the person to be searched for exists (step ST200). Specifically, the person search unit 190 acquires from the database 200 shooting information linked to the facial image feature whose second similarity exceeds the second threshold, and uses the acquired shooting information to calculate the area and time period in which the person shown in the specific person image may have been photographed, and generates specific person shooting information including the camera identification information within that area and the time period. This narrows down the cameras and time ranges where the person you're searching for may be present.

[0051] Person search section 190 further selects search result 2 from class 3 that satisfies the conditions of the camera and time range in which the person to be searched for may exist (step ST210).

[0052] The person search section 190 outputs the search results (step ST220). Specifically, the person search unit 190 generates specific person shooting information including the camera identification information and the time period, using the set information of categories 1 and 2 and the selected set information of category 3. The person search unit 190 outputs the specific person shooting information as a search result.

[0053] Here, the process of step ST180 will be described in more detail. FIG. 6 is a flowchart showing the detailed process in ST180 in FIG. The classification section 180 starts the process, for example, when the similarity calculation section 170 calculates the first similarity and the second similarity.

[0054] The classification section 180 judges whether the second similarity exceeds the second threshold (second similarity>second threshold?) (step ST181). If the second similarity exceeds the second threshold in step ST181 (step ST181 "YES"), the classification unit 180 determines whether the first similarity exceeds the first threshold (first similarity>first threshold?) (step ST182). If the first similarity exceeds the first threshold value in step ST182 ("YES" in step ST182), the classification section 180 classifies the set information into category 1 (step ST183). If the first similarity does not exceed the first threshold value in step ST182 ("NO" in step ST182), the classification section 180 classifies the set information into category 2 (step ST184).

[0055] If the second similarity does not exceed the second threshold in step ST181 (step ST181 "NO"), the classification unit 180 determines whether the first similarity exceeds the first threshold (first similarity>first threshold?) (step ST185). If the first similarity exceeds the first threshold value in step ST185 (step ST185 "YES"), the classification section 180 classifies the set information into category 3 (step ST186). If the first similarity does not exceed the first threshold value in step ST185 (step ST185 "NO"), the classification section 180 classifies the set information into category 4 (step ST187).

[0056] As described above, it is possible to search for images in an area and time period in which an image having facial features similar to those of a person shown in a specific person image was taken. Then, after obtaining assurance that the image is the same person through face recognition using universal features, the cameras and time ranges in which the same person may exist are narrowed down based on information such as the date and time, camera position, camera adjacency, and relative distance of the searched camera images, and then person identification is performed based on the similarity of the whole-body image features, thereby achieving the effect of speeding up and improving the accuracy of person search for a specific person.

[0057] Here, a modification of the person search device of the present disclosure will be described. In the present disclosure, the person search device 100 and the person search system 1 shown in FIG. 2 may be configured to distribute the load in order to reduce the load on the person search device 100. FIG. 7 is a diagram showing a first variation of the configuration of a person retrieval system 1″ including a person retrieval device 100′ according to the first embodiment of the present disclosure. FIG. 8 is a diagram showing a second variation of the configuration of a person retrieval system 1'''' including a person retrieval device 100'' according to the first embodiment of the present disclosure.

[0058] The person search system 1′ shown in FIG. 7 is obtained by removing the camera image acquisition unit 110, the whole-body image extraction unit 120, the facial image extraction unit 130, the whole-body image feature extraction unit 140, and the facial image feature extraction unit 150 from the person search device 100 shown in FIG. 2 and transforming it into an image analysis device 102. According to the person search device 100′ shown in FIG. 7, the processing load of the person search device 100′ can be distributed, and the input to the person search device 100′ becomes a whole-body image, a facial image, whole-body image features, and facial image features from a camera video stream, so that a reduction in the transmission bandwidth can be expected.

[0059] The person retrieval system 1'' shown in FIG. 8 is obtained by removing the camera image acquisition unit 110, the whole-body image extraction unit 120, the facial image extraction unit 130, the whole-body image feature extraction unit 140, and the facial image feature extraction unit 150 from the person retrieval device 100 shown in FIG. 2 and modifying it to a configuration in which the processing is performed within a camera 400''. This distributes the processing load and enables the person retrieval device 100'' to be realized with an inexpensive CPU. Also, the transmission band can be reduced.

[0060] In Figs. 7 and 8, the person retrieval devices 100', 100'' are shown equipped with the person feature storage unit 160, but it is also possible to provide the person feature storage unit 160 as an external configuration of the person retrieval device.

[0061] As described above, the person retrieval device according to the present disclosure is a person retrieval device that retrieves images including a search target person from among a plurality of images captured by a plurality of cameras, and when acquiring whole-body image features and face image features of the person in a specific person image in which the search target person is indicated, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the plurality of cameras, calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and The system is configured to include a similarity calculation unit that calculates a second similarity indicating the similarity between facial image features of the image and facial image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and whether the second similarity exceeds a second threshold; and a person search unit that uses a determination result of the classification unit to acquire shooting information linked to the facial image features for which the second similarity exceeds the second threshold from the database, calculates an area and time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, and generates specific person shooting information including camera identification information within the area and the time period. This makes it possible to provide a person search device that searches for images containing a specific person with high accuracy from among images captured by a plurality of cameras.

[0062] The person search device of the present disclosure is further configured such that the person search unit is further configured to output the specific person shooting information and an image captured by the camera indicated in the camera identification information of the specific person shooting information to the monitoring terminal that requested the search. This makes it possible to provide a person search device that presents search results including images.

[0063] The person search device according to the present disclosure is further configured to include a person feature storage unit that acquires whole-body image feature amounts and face image feature amounts of images captured by each camera and shooting information of the images, and associates the acquired whole-body image feature amounts, face image feature amounts, and shooting information and stores them in a database. This makes it possible to provide a person search device that adds and updates data in a database.

[0064] The person search device of the present disclosure further includes a whole-body image extraction unit that extracts a whole-body image from a received image, a face image extraction unit that extracts a face image from the received image, a whole-body image feature calculation unit that extracts whole-body image features from the whole-body image extracted by the whole-body image extraction unit, and a face image feature calculation unit that extracts face image features from the face image extracted by the face image extraction unit, and is configured so that the similarity calculation unit acquires the whole-body image features from the whole-body image feature calculation unit and acquires the face image features from the face image feature calculation unit. This makes it possible to provide a person search device that accepts a search request using a specific person image.

[0065] The person search device according to the present disclosure further includes a camera image acquisition unit that acquires images and shooting information captured by a plurality of cameras, and a person feature storage unit that acquires whole-body image features and facial image features of images captured by each camera and shooting information of the images, and links the acquired whole-body image features, facial image features, and shooting information to store them in a database, wherein the whole-body image extraction unit extracts whole-body images from the images acquired by the camera image acquisition unit, the facial image extraction unit extracts facial images from the images acquired by the camera image acquisition unit, the whole-body image feature calculation unit extracts whole-body image features from the whole-body images of the images acquired by the camera image acquisition unit and outputs them to the person feature storage unit, and the facial image feature calculation unit extracts facial image features from the facial images of the images acquired by the camera image acquisition unit and outputs them to the person feature storage unit. This has the effect of providing a person search device that acquires images from a camera and adds and updates data in a database.

[0066] The person search device according to the present disclosure is further configured such that, when generating specific person shooting information, if multiple pieces of camera identification information indicating the same camera are included, the person search unit thins out the camera identification information indicating the same camera at predetermined time intervals. This has the effect of providing a person search device that outputs search results that are less biased towards a specific camera or time period.

[0067] A person search system according to the present disclosure is a person search system having a person search device that searches for images including a search target person from among multiple images captured by multiple cameras in accordance with a search request from a monitoring terminal, the person search device having a monitoring terminal, multiple cameras, and the person search device, when acquiring full-body image features and facial image features of a person in a specific person image in which the search target person is indicated, refers to a database in which full-body image features, facial image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and compares the full-body image features of the specific person image with the full-body image features in the database The imaging device is characterized by comprising a similarity calculation unit that calculates a first similarity indicating the similarity, and calculates a second similarity indicating the similarity between the facial image features of the specific person image and the facial image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold, and determines whether the second similarity exceeds a second threshold; and a person search unit that uses a determination result of the classification unit to acquire shooting information linked to the facial image features for which the second similarity exceeds the second threshold, and uses the acquired shooting information to calculate an area and a time period in which the person shown in the specific person image may have been photographed, and generates specific person shooting information including camera identification information within the area and the time period. This makes it possible to provide a person search system that searches for images containing a specific person with high accuracy from among images captured by a plurality of cameras.

[0068] A person search program according to the present disclosure is a person search program that causes a computer to operate as a person search device that searches for an image including a search target person from among a plurality of images captured by a plurality of cameras, and when the computer acquires full-body image features and facial image features of a person in a specific person image in which the search target person is indicated, the computer refers to a database in which the full-body image features, facial image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the plurality of cameras, and calculates a first similarity indicating a similarity between the full-body image features of the specific person image and the full-body image features in the database. the specific person image includes a similarity calculation unit that calculates a first similarity indicating the similarity between the facial image features of the specific person image and the facial image features in the database, and a classification unit that determines whether the first similarity exceeds a first threshold and whether the second similarity exceeds a second threshold; and a person search unit that uses the determination result of the classification unit to obtain, from the database, shooting information linked to the facial image features for which the second similarity exceeds the second threshold, and uses the obtained shooting information to calculate an area and a time period in which the person shown in the specific person image may have been photographed, and generates specific person shooting information including camera identification information for the area and the time period. This makes it possible to provide a person search program that searches for images containing a specific person with high accuracy from among images captured by a plurality of cameras.

[0069] Embodiment 2 The second embodiment further improves the accuracy of person search by adjusting the differences in whole-body image features that arise due to differences in appearance caused by the installation position, angle of view, and external light difference of the surveillance camera, or differences in clothing. FIG. 9 is a diagram showing a configuration of a person search system 2 including a person search device 101 according to the second embodiment of the present disclosure. 9 differs in that person search device 100 shown in the first embodiment is changed to a person search device 101. Specifically, the difference is that a whole-body image feature amount correcting unit 145 is provided. 9 differs in that the database 200 shown in the first embodiment is changed to a database 202. Specifically, the database 202 differs in that it has a feature amount difference determination threshold (third threshold) 280. Therefore, in the following description, the whole-body image feature amount correcting unit 145 and the database 202 in the person retrieval system 2 will be described, and the contents already described will be omitted as appropriate.

[0070] The person retrieval device 101 in the person retrieval system 2 includes a camera image acquisition unit 110, a whole-body image extraction unit 120, a facial image extraction unit 130, a whole-body image feature extraction unit 140, a facial image feature extraction unit 150, a person feature storage unit 160, a similarity calculation unit 170, a classification unit 180, a person retrieval unit 190, a whole-body image feature correction unit 145, and a control unit (not shown).

[0071] When the whole-body image feature extraction unit 140 calculates the whole-body image feature, the whole-body image feature correction unit 145 modifies the correction value 240 for correcting the whole-body image feature by using the whole-body image feature linked to the face image feature whose second similarity exceeds the second threshold.

[0072] Specifically, the whole-body image feature amount correcting section 145 acquires the set information to which the classification information indicating the category 1 and the category 2 is added from the set information output by the classifying section 180. The whole-body image feature amount correction unit 145, for example, uses the result with the highest similarity of the whole-body image feature amounts of Class 1 and Class 2 as a standard, weights other whole-body image feature amounts or similarities to be approximately the same, and calculates a correction value for each camera 400 (400-1, 400-2, 400-3, . . . , 400-n). The whole-body image feature amount correction unit 145 statistically processes the results of multiple calculations, determines a correction value for each camera 400 (400-1, 400-2, 400-3, . . . , 400-n), and updates the correction value 240 stored in the database 202. The statistical processing may be set to perform a statistical processing for deriving a likely correction value from the results of a plurality of calculations, such as calculating an average value of a plurality of correction values.

[0073] The process when this function is provided will be described. FIG. 10 is a flowchart showing the correction value update process in the person retrieval device 101 according to the second embodiment of the present disclosure. After completing step ST210 in FIG. 5, person retrieval device 101 in the second embodiment proceeds to step ST310. The whole-body image feature amount correcting section 145 extracts whole-body image feature amounts of the images classified into the categories 1 and 2 from the database 202 (step ST310). The whole-body image feature amount correcting section 145 calculates a correction value using the whole-body image feature amount, and performs statistical processing on the results of a plurality of calculations (step ST320). The whole-body image feature amount correcting section 145 determines a correction value based on the result of the statistical processing (step ST330). The whole-body image feature amount correcting section 145 updates the correction value (step ST340).

[0074] This allows the accuracy of person identification using whole-body images to be improved by correcting the variations in whole-body image features due to changes in the angle of view and external light of each camera 400 (400-1, 400-2, 400-3, ..., 400-n) in cooperation with facial recognition.

[0075] Hereinafter, the description returns to the configuration of the person search system 2 including the person search device 101 according to the second embodiment. The database 202 includes a program 210, whole-body information 220, face information 230, correction value 240, a whole-body similarity determination threshold (first threshold) 250, a face similarity determination threshold (second threshold) 260, and shooting information 270, as well as a feature difference determination threshold (third threshold) 280. The feature difference determination threshold (third threshold) 280 is used to determine whether the difference in whole-body image features is due to a difference in appearance or clothing caused by a difference in the camera used to take the photo, when the facial image has a high similarity and is likely to be the same person as the specific person image. The whole-body image feature correction unit 145 further replaces the whole-body image feature of the specific person image with the whole-body image feature linked to the facial image feature whose second similarity exceeds the second threshold when the difference value between the whole-body image feature linked to the facial image feature whose second similarity exceeds the second threshold and the whole-body image feature of the specific person image exceeds the feature difference determination threshold (third threshold) 280.

[0076] Specifically, the whole-body image feature amount correcting section 145 acquires the set information to which the classification information indicating the category 2 is added from the set information output by the classifying section 180. The whole-body image feature amount correction unit 145 determines whether the difference value between the whole-body image feature amount of the set information of the class 2 and the whole-body image feature amount of the specific person image exceeds a whole-body feature amount difference determination threshold (third threshold). When the difference value exceeds the third threshold, the whole-body image feature correction unit 145 replaces the whole-body image feature of the specific person image used to calculate the difference value with the whole-body image feature linked to the face image feature in the database 202 whose second similarity exceeds the second threshold, and sets it as the second whole-body image feature of the specific person image.

[0077] As a result, the similarity calculation unit 170 calculates a similarity (third similarity) between the second whole-body image feature of the specific person image and the whole-body image feature in the database 202. The classification unit 180 classifies the set information by determining whether the third similarity exceeds the first threshold. The person search unit 190 selects from the database 202 set information having a whole-body image feature whose third similarity exceeds the first threshold, and outputs it as a search result.

[0078] The process when this function is provided will be described. FIG. 11 is a flowchart showing a process of extracting an image using whole-body image features in the person retrieval device 101 according to the second embodiment of the present disclosure. After completing step ST210 in FIG. 5, person retrieval device 101 in the second embodiment proceeds to step ST410. The whole-body image feature amount correcting section 145 extracts the whole-body image feature amounts of the images classified into the category 2 from the database 202 (step ST410). The whole-body image feature amount correcting unit 145 determines whether the difference value between the whole-body image feature amount of the person image to be searched (specific person image) and the whole-body image feature amount of the database 202 exceeds a third threshold value (difference value>third threshold value) (step ST420). The whole-body image feature amount correcting section 145 determines a second whole-body image feature amount (step ST430). The whole-body image feature amount correcting section 145 outputs the second whole-body image feature amount to the similarity calculating section 170. The similarity calculation unit 170 calculates a similarity (third similarity) between the second whole-body image feature of the specific person image and the whole-body image feature associated with the shooting information selected in step ST200. The similarity calculation unit 170 outputs the set information used in the calculation together with the third similarity to the classification unit 180. The classification unit 180 determines whether the third similarity exceeds the first threshold, and classifies the set information using the determination result in the same manner as in embodiment 1. The classification unit 180 performs classification using the third similarity instead of the first similarity described in embodiment 1. The person detection section selects, from the set information of category 2, set information whose third similarity exceeds the first threshold as search result 3 (step ST450). When the process of step ST450 ends, in the person retrieval device 101, the process proceeds to step ST220. In step ST220, the person search section 190 outputs the search results including the search result 3.

[0079] This allows the system to track large changes in whole-body image features due to differences in clothing for the same person, and by working in conjunction with facial recognition, the whole-body image features of a specific person are updated every hour, thereby improving the accuracy of person identification using whole-body images.

[0080] As described above, it is possible to ensure that the images are of the same person based on the similarity of facial image features, which are universal features, and it is also possible to correct variations in whole-body image features due to changes in the camera angle of view and external light, and to accommodate changes in features due to differences in clothing, etc., thereby improving the accuracy of person identification using whole-body images between cameras.

[0081] The person search device according to the present disclosure is further configured such that, when receiving an image acquired by the camera image acquisition unit, the whole-body image feature extraction unit calculates a corrected whole-body image feature using a correction amount for each camera indicated by the camera identification information, and includes a whole-body image feature correction unit that modifies the correction amount used by the whole-body image feature extraction unit using the whole-body image feature linked to the face image feature whose second similarity exceeds a second threshold. This has the effect of providing a person search device that further improves the accuracy of person identification using whole-body images taken by cameras.

[0082] The person search device according to the present disclosure is further configured to include a whole-body image feature correction unit that, when a difference value between a whole-body image feature linked to a facial image feature whose second similarity exceeds a second threshold and a whole-body image feature of a specific person image exceeds a third threshold, replaces the whole-body image feature of the specific person image with the whole-body image feature linked to the facial image feature whose second similarity exceeds the second threshold. This has the effect of providing a person search device that further improves the accuracy of person identification using whole-body images taken by cameras.

[0083] Embodiment 3 The third embodiment is an embodiment including a configuration for adjusting the first and second thresholds used in the first or second embodiment. FIG. 12 is a diagram showing a configuration of a person search system 2A including a person search device 101A according to the third embodiment of the present disclosure. The person search system 2A shown in Fig. 12 differs from the configurations described so far in that the person search device 101 in the person search system 2 shown in Fig. 9 is changed to a person search device 101A. Specifically, the person search device 101A differs in that it further includes a threshold adjustment unit 1110 and a relearning determination unit 1120. Therefore, in the following description, the description will be centered on the threshold adjustment unit 1110 and the relearning determination unit 1120 in the person search system 2A, and descriptions that overlap with the contents already explained will be omitted as appropriate.

[0084] The person retrieval device 101A in the person retrieval system 2A includes a camera image acquisition unit 110, a whole-body image extraction unit 120, a facial image extraction unit 130, a whole-body image feature extraction unit 140, a facial image feature extraction unit 150, a person feature storage unit 160, a similarity calculation unit 170, a classification unit 180, a person retrieval unit 190, a whole-body image feature correction unit 145, a threshold adjustment unit 1110, a relearning determination unit 1120, and a control unit (not shown).

[0085] The threshold adjustment unit 1110 realizes a threshold setting function for automatically adjusting the first threshold and the second threshold. FIG. 13 is a diagram showing a configuration of the threshold adjustment unit 1110 in this disclosure. The threshold adjusting unit 1110 shown in FIG. The first threshold adjuster 1111 adjusts and sets the first threshold. Specifically, the first threshold adjustment unit 1111 adjusts and sets the first threshold using the first similarity accumulated based on the results previously determined by the classification unit 180 and the similarity calculated based on the whole-body image features of each different person (whole-body feature similarity between different people).

[0086] More specifically, the first threshold adjustment unit 1111 first calculates a first similarity statistic, which is a statistical value of the first similarity. The first threshold adjustment unit 1111 accumulates the similarity (first similarity) of the whole-body image of the person classified into "(Classification 2)" in which only the similarity of the face image exceeds the threshold based on the determination result by the classification unit 180. The first threshold adjustment unit then calculates the first similarity statistic using the accumulated first similarity. The first similarity statistic is, for example, an average value of the first similarity or a weighted average value of the first similarity. The first similarity statistic may be all of the accumulated first similarity, or a part of the accumulated first similarity. The first threshold adjustment unit 1111 then calculates a second similarity statistic, which is a statistic of whole body feature similarities between different people. For example, the first threshold adjustment unit 1111 determines that people photographed by different cameras at the same time are different people, calculates whole body feature similarities based on the whole body image features of each of the different people, and accumulates the whole body feature similarities between the different people. The first threshold adjustment unit 1111 calculates a second similarity statistic using the accumulated whole body feature similarities between the different people. The second similarity statistic is, for example, an average value of the whole body feature similarities between the different people, or a weighted average value of the whole body feature similarities between the different people. The second similarity statistic may be all of the accumulated whole body feature similarities between the different people, or may be a part of the accumulated whole body feature similarities between the different people. The first threshold adjustment unit 1111 then adjusts and sets the first threshold using the first similarity statistics and the second similarity statistics. The first threshold adjustment unit 1111 adjusts and sets the first threshold so that, for example, the relationship is second similarity statistics < first threshold < first similarity statistics. This is based on the idea that the second similarity statistics are considered to be statistics of whole body feature similarity between different people, so the first threshold is made larger than the second similarity statistics. Also, the first similarity statistics are considered to be statistics of similarity between the same person, so the first threshold is made smaller than the first similarity statistics. With the above-described configuration, the first threshold value can be automatically adjusted and set.

[0087] The second threshold adjustment unit 1112 adjusts and sets the second threshold using the similarity calculated based on the facial image feature amounts of the different persons (the facial feature similarity between different persons). Specifically, the second threshold adjustment unit 1112 calculates a third similarity statistic, which is a statistical value of facial feature similarity between different people. For example, the second threshold adjustment unit 1112 determines that people photographed by different cameras at the same time are different people, calculates facial feature similarity based on the facial image feature amount of each of the different people, and accumulates the facial feature similarity between the different people. The second threshold adjustment unit 1112 calculates a third similarity statistic using the accumulated facial feature similarity between the different people. The third similarity statistic is, for example, an average value of the facial feature similarity between the different people, or a weighted average value of the facial feature similarity between the different people. In addition, the third similarity statistic may be all of the accumulated facial feature similarity between the different people, or may be a part of the accumulated facial feature similarity between the different people. The second threshold adjustment unit 1112 then adjusts and sets the second threshold using the third similarity statistic. The second threshold adjustment unit 1112 adjusts and sets the second threshold so that, for example, the relationship is third similarity statistic < second threshold. This is based on the idea that the third similarity statistic is considered to be a statistic of facial feature similarity between different persons, and therefore the second threshold is made larger than the third similarity statistic. With the above-described configuration, the second threshold value can be automatically adjusted and set.

[0088] The threshold adjustment process (first threshold adjustment process, second threshold adjustment process) by the threshold adjustment unit 1110 may be performed at any timing. For example, it may be performed when the person search system is installed, or may be performed during operation of the person search system. When it is performed during operation of the person search system, it may be performed at predetermined regular intervals, or may be performed in response to an appropriate command.

[0089] If the threshold adjustment unit 1110 cannot adjust and set the first threshold, the re-learning determination unit 1120 instructs the whole-body image feature extraction unit 140 to execute a learning process for extracting whole-body image features from the whole-body image. Specifically, when the re-learning judgment unit 1120 receives a notification from the threshold adjustment unit 1110 that the first threshold cannot be adjusted or set, the re-learning judgment unit 1120 judges that it is necessary to re-learn the information (information used to extract features from an image) that was previously obtained by a learning process (pre-learning process) in the whole-body image feature extraction unit 140, and instructs the whole-body image feature extraction unit 140 to execute the learning process. When receiving the notification from the threshold adjustment unit 1110, the relearning determination unit 1120 may acquire the whole-body images used in the calculation of each similarity and store them in, for example, the database 200. In this case, specifically, the relearning determination unit 1120 stores whole-body images that are in a relationship of the first similarity used in the first similarity statistics (a relationship classified as "(Classification 2)" in which only the similarity of the face images exceeds the threshold) so as to be distinguished as being of the same person. As a result, a true value indicating that they are the same person is assigned to a plurality of whole-body images that should be distinguished as being of the same person. In addition, the relearning determination unit 1120 stores whole-body images that are in a relationship of the whole-body feature similarity used in the second similarity statistics so as to be distinguished as being of different people. Upon receiving a command from the re-learning determination unit 1120, the whole-body image feature amount extraction unit 140 executes a learning process using the stored images. By executing the learning process, the whole-body image feature extraction unit 140 extracts mutually similar whole-body image feature amounts when extracting whole-body image feature amounts of persons included in different images who are likely to be the same person. Also, the whole-body image feature extraction unit 140 extracts mutually dissimilar whole-body image feature amounts when extracting whole-body image feature amounts of persons included in different images who are likely to be different persons. As a result, the whole-body image feature amount extraction section 140 can extract the whole-body image feature amount with higher accuracy.

[0090] Here, an example of the threshold adjustment process according to the present disclosure will be described. The first threshold adjustment process will be described first, followed by the second threshold adjustment process. FIG. 14 is a flowchart showing an example of the first threshold adjustment process in the present disclosure. When the first threshold adjustment process is started in threshold adjustment section 1110, first, first threshold adjustment section 1111 calculates a first similarity statistical value (step ST510). Specifically, the first threshold adjustment unit 1111 accumulates the similarity (first similarity) of the whole-body image of the person classified into "(Classification 2)" in which only the similarity of the face image exceeds the threshold based on the determination result by the classification unit 180. At this time, the first similarity is accumulated in, for example, the database 200. The first threshold adjustment unit 1111 then calculates a first similarity statistic value using the accumulated first similarity.

[0091] Next, the first threshold adjustment unit 1111 calculates a second similarity statistical value (step ST520). Specifically, the first threshold adjustment unit 1111 determines that people photographed by different cameras at the same time are different people, calculates whole-body feature similarities based on the whole-body image features of the different people, and accumulates the whole-body feature similarities between the different people. At this time, the whole-body feature similarities between the different people are accumulated in, for example, the database 200. The first threshold adjustment unit 1111 calculates a second similarity statistic value using the accumulated whole-body feature similarities between the different people.

[0092] Next, the first threshold adjustment unit 1111 adjusts and sets the first threshold using the first similarity statistical value and the second similarity statistical value (step ST530). Specifically, the first threshold adjustment unit 1111 adjusts and sets the first threshold so as to satisfy the relationship: second similarity statistical value<first threshold<first similarity statistical value.

[0093] Next, the first threshold adjusting unit 1111 determines whether adjustment is not possible (step ST540). Specifically, the first threshold adjustment unit 1111 determines that adjustment is not possible when the above relationship is not satisfied, for example, when the second similarity statistical value is greater than the first similarity statistical value.

[0094] Next, when the first threshold value adjusting unit 1111 determines that adjustment is not possible (step ST540 "YES"), it notifies the relearning determining unit 1120 (step ST550). When the re-learning determination unit 1120 receives a notification from the first threshold adjustment unit 1111, it instructs the whole-body image feature amount extraction unit 140 to execute a learning process. When the whole-body image feature amount extraction unit 140 receives a command to execute a learning process from the re-learning determination unit 1120, it executes the learning process.

[0095] After notifying the relearning determination section 1120, the first threshold value adjustment section 1111 repeats the process from step ST510.

[0096] Next, if the first threshold adjusting unit 1111 determines that adjustment is not impossible (step ST540 "NO"), it ends the process.

[0097] The second threshold adjustment process will be described. FIG. 15 is a flowchart showing an example of the second threshold adjustment process in the present disclosure. When the second threshold adjustment process is started in threshold adjustment section 1110, the second threshold adjustment section first calculates a third similarity statistical value (step ST610). Specifically, the second threshold adjustment unit 1112 determines that people photographed by different cameras at the same time are different people, calculates facial feature similarities based on the facial image features of the different people, and accumulates the facial feature similarities between the different people. At this time, the facial feature similarities between the different people are accumulated in, for example, the database 200. The second threshold adjustment unit 1112 calculates a third similarity statistic using the accumulated facial feature similarities between the different people.

[0098] The second threshold adjusting unit 1112 adjusts and sets the second threshold using the third similarity statistical value (step ST620). Specifically, the second threshold adjusting unit 1112 adjusts and sets the second threshold so that a relationship such as the third similarity statistical value<the second threshold is satisfied, for example.

[0099] After adjusting and setting the second threshold, the second threshold adjusting unit 1112 ends the process.

[0100] In the person search device of the present disclosure, the threshold adjustment unit is configured to set the first threshold using the first similarity accumulated based on the results previously determined by the classification unit, and a similarity calculated based on the whole-body image features of each different person. This brings about an effect of providing a person retrieval device that can automatically set the first threshold value.

[0101] In the person retrieval device according to the present disclosure, the threshold adjustment unit is configured to set the second threshold by using a similarity calculated based on facial image features of different persons. This brings about an effect of providing a person retrieval device that can automatically set the second threshold value.

[0102] The person search device of the present disclosure is configured to further include a re-learning determination unit that, if the threshold adjustment unit is unable to adjust the first threshold, instructs the whole-body image feature extraction unit to execute a learning process of information used to extract whole-body image features from a whole-body image. This has the effect of making it possible to adjust the whole-body image feature amount extracted by the whole-body image feature amount extraction section when there is a possibility that the whole-body image feature amount extracted by the whole-body image feature amount extraction section is inappropriate.

[0103] Here, the hardware configuration of the person retrieval device 100, 101, 101A of the present disclosure will be described. FIG. 16 is a diagram illustrating a first example of a hardware configuration of the person retrieval device 100, 101, or 101A according to the present disclosure. FIG. 17 is a diagram illustrating a second example of the hardware configuration of the person retrieval device 100, 101, or 101A according to the present disclosure. The person retrieval device 100, 101, 101A according to the first embodiment (including the modification) or the second embodiment is realized by hardware as shown in FIG. 16 or FIG. 17, respectively.

[0104] Each of the person search devices 100, 101, and 101A includes a processor 10001 and a memory 10002, as shown in FIG. The processor 10001 and the memory 10002 are installed in, for example, a computer. The memory 10002 stores programs for causing the computer to function as a camera image acquisition unit 110, a whole-body image extraction unit 120, a facial image extraction unit 130, a whole-body image feature extraction unit 140, a whole-body image feature correction unit 145, a facial image feature extraction unit 150, a person feature storage unit 160, a similarity calculation unit 170, a classification unit 180, a person search unit 190, a threshold adjustment unit 1110, a first threshold adjustment unit 1111, a second threshold adjustment unit 1112, a re-learning judgment unit 1120, and a control unit not shown. By having the processor 10001 read and execute the program stored in the memory 10002, the functions of the camera image acquisition unit 110, the whole-body image extraction unit 120, the facial image extraction unit 130, the whole-body image feature extraction unit 140, the facial image feature extraction unit 150, the whole-body image feature correction unit 145, the person feature storage unit 160, the similarity calculation unit 170, the classification unit 180, the person search unit 190, the threshold adjustment unit 1110, the first threshold adjustment unit 1111, the second threshold adjustment unit 1112, the re-learning judgment unit 1120, and a control unit not shown are realized. Furthermore, databases 200 and 202 are realized by the memory 10002 or other memories not shown. The processor 10001 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, or a digital signal processor (DSP). The memory 10002 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash memory, or a magnetic disk such as a hard disk or a flexible disk, or an optical disk such as a CD (Compact Disc) or a DVD (Digital Versatile Disc), or a magneto-optical disk.

[0105] Alternatively, the functions of the camera image acquisition unit 110, the whole-body image extraction unit 120, the facial image extraction unit 130, the whole-body image feature extraction unit 140, the whole-body image feature correction unit 145, the facial image feature extraction unit 150, the person feature storage unit 160, the similarity calculation unit 170, the classification unit 180, the person search unit 190, the threshold adjustment unit 1110, the first threshold adjustment unit 1111, the second threshold adjustment unit 1112, the re-learning judgment unit 1120, and a control unit (not shown) may be realized by a dedicated processing circuit 10003, as shown in FIG. 17. The processing circuit 10003 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration), etc. The functions of the camera image acquisition unit 110, the whole-body image extraction unit 120, the facial image extraction unit 130, the whole-body image feature extraction unit 140, the whole-body image feature correction unit 145, the facial image feature extraction unit 150, the person feature storage unit 160, the similarity calculation unit 170, the classification unit 180, the person search unit 190, the threshold adjustment unit 1110, the first threshold adjustment unit 1111, the second threshold adjustment unit 1112, the re-learning judgment unit 1120, and a control unit (not shown) may be realized by separate processing circuits 10003, or may be realized collectively by the processing circuit 10003.

[0106] Alternatively, some of the functions of the camera image acquisition unit 110, whole-body image extraction unit 120, facial image extraction unit 130, whole-body image feature extraction unit 140, whole-body image feature correction unit 145, facial image feature extraction unit 150, person feature storage unit 160, similarity calculation unit 170, classification unit 180, person search unit 190, threshold adjustment unit 1110, first threshold adjustment unit 1111, second threshold adjustment unit 1112, re-learning judgment unit 1120, and a control unit not shown in the figure may be realized by the processor 10001 and memory 10002, and the remaining functions may be realized by the processing circuit 10003.

[0107] It should be noted that, within the scope of the disclosure, the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted. [Explanation of symbols]

[0108] 1,1´,1´´,2,2A Person search system, 100,100´,100´´,101,101A Person search device, 102 Image analysis device, 110 Camera image acquisition unit, 120 Whole body image extraction unit, 130 Face image extraction unit, 140 Whole body image feature amount extraction unit, 145 Whole body image feature amount correction unit, 150 Face image feature amount extraction unit, 160 Person feature storage unit, 170 Similarity calculation unit, 180 Classification unit, 190 Person search unit, 200, 202 Database, 210 Program, 220 Whole body information, 230 Face information, 240 Correction value, 250 Similarity judgment threshold (first threshold), 260 Similarity judgment threshold (second threshold), 270 Shooting information, 280 Feature amount difference judgment threshold (third threshold), 300 Monitoring terminal, 400, 400-1, 400-2, 400-3, ··· 400-n, 400´´, 400´´-1, 400´´-2, 400´´-3, ··· 400´´-n Camera, 500 Image recording device, 600 Communication network, 700 Image, 701 Whole-body image, 702 Facial image, 1110 Threshold value determination unit, 1111 First threshold value determination unit, 1112 Second threshold value determination unit, 1120 Re-learning judgment unit, 10001 Processor, 10002 Memory, 10003 Processing circuit, 10004 Memory.

Claims

1. A person search device that searches for an image including a person to be searched from among a plurality of images captured by a plurality of cameras, comprising: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; Equipped with the whole-body image feature correction unit uses the judgment result of the classification unit to acquire from the database a whole-body image feature linked to a facial image feature for which the second similarity exceeds a second threshold, and when a difference value between the acquired whole-body image feature and the whole-body image feature of the specific person image exceeds a third threshold, replaces the whole-body image feature of the specific person image with the whole-body image feature linked to the facial image feature for which the second similarity exceeds the second threshold.

2. A person search device that searches for an image including a person to be searched from among a plurality of images captured by a plurality of cameras, comprising: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; The person search device includes:

3. A person search device that searches for an image including a person to be searched from among a plurality of images captured by a plurality of cameras, comprising: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to obtain, from the database, shooting information associated with a whole-body image feature whose first similarity exceeds a first threshold and a face image feature whose second similarity exceeds a second threshold, and generates specific person shooting information using the obtained shooting information; Equipped with the whole-body image feature correction unit uses the judgment result of the classification unit to acquire from the database a whole-body image feature linked to a facial image feature for which the second similarity exceeds a second threshold, and when a difference value between the acquired whole-body image feature and the whole-body image feature of the specific person image exceeds a third threshold, replaces the whole-body image feature of the specific person image with the whole-body image feature linked to the facial image feature for which the second similarity exceeds the second threshold.

4. The person search unit is Furthermore, the specific person photographing information and an image photographed by a camera identified by the camera identification information of the specific person photographing information are output to the monitoring terminal that has requested the search. The person search device according to any one of claims 1 to 3.

5. a person feature storage unit that acquires whole body image feature amounts and face image feature amounts of images captured by each camera and shooting information of the images, and associates the acquired whole body image feature amounts, face image feature amounts, and shooting information and stores them in the database; The person search device according to any one of claims 1 to 3.

6. a whole-body image extraction unit that extracts a whole-body image from the received image; a face image extraction unit that extracts a face image from the received image; a whole-body image feature extraction unit that extracts whole-body image feature values ​​from the whole-body image extracted by the whole-body image extraction unit; a facial image feature extraction unit that extracts facial image feature values ​​from the facial image extracted by the facial image extraction unit, The similarity calculation unit is acquiring a whole-body image feature amount from the whole-body image feature amount extraction unit, and acquiring a face image feature amount from the face image feature amount extraction unit; The person search device according to claim 2 .

7. a whole-body image extraction unit that extracts a whole-body image from the received image; a face image extraction unit that extracts a face image from the received image; a whole-body image feature extraction unit that extracts whole-body image feature values ​​from the whole-body image extracted by the whole-body image extraction unit; a facial image feature extraction unit that extracts facial image feature values ​​from the facial image extracted by the facial image extraction unit, The similarity calculation unit is acquiring a whole-body image feature amount from the whole-body image feature amount extraction unit, and acquiring a face image feature amount from the face image feature amount extraction unit; The person search device according to claim 3 .

8. a camera image acquisition unit that acquires images and shooting information captured by a plurality of cameras; a person feature storage unit that acquires whole-body image feature amounts and face image feature amounts of images captured by each camera and shooting information of the images, and stores the acquired whole-body image feature amounts, face image feature amounts, and shooting information in the database in association with each other; The whole-body image extraction unit extracts a whole-body image from the image acquired by the camera image acquisition unit, The facial image extraction unit extracts a facial image from the image acquired by the camera image acquisition unit, the whole-body image feature extraction unit extracts whole-body image feature values ​​from the whole-body image of the image acquired by the camera image acquisition unit, and outputs the whole-body image feature values ​​to the person feature storage unit; 8. The person search device according to claim 6, wherein the face image feature amount extraction unit extracts face image feature amounts from the face image in the image acquired by the camera image acquisition unit, and outputs the face image feature amounts to the person feature storage unit.

9. The person search unit is When generating the specific person shooting information, if a plurality of pieces of camera identification information indicating the same camera are included, the camera identification information indicating the same camera is thinned out at a predetermined time interval. The person search device according to any one of claims 1 to 3.

10. The whole-body image feature amount extraction unit, when receiving an image captured by the camera, calculates a whole-body image feature amount corrected using a correction amount for each camera indicated by camera identification information, 8. The person search device of claim 7, wherein the whole-body image feature correction unit corrects a correction amount used in the whole-body image feature extraction unit, using a facial image feature whose second similarity exceeds a second threshold and a plurality of whole-body image features linked to camera identification information indicating different cameras, so that each of the plurality of whole-body image features is comparable to a plurality of other whole-body image features.

11. The whole-body image feature amount extraction unit, when receiving an image captured by the camera, calculates a whole-body image feature amount corrected using a correction amount for each camera indicated by camera identification information, 7. The person retrieval device of claim 6, further comprising a whole-body image feature correction unit that corrects a correction amount used in the whole-body image feature extraction unit by using a facial image feature whose second similarity exceeds a second threshold and a plurality of whole-body image features linked to camera identification information indicating different cameras, so that each of the plurality of whole-body image features is approximately the same as a plurality of other whole-body image features.

12. 4. The person retrieval device according to claim 1, further comprising a threshold adjustment unit that sets the first threshold by using the first similarity accumulated based on a result determined in advance by the classification unit and a similarity calculated based on whole-body image features of each of different persons.

13. The threshold adjustment unit further setting the second threshold value using a similarity calculated based on the facial image feature amounts of each of different persons; The person search device according to claim 12.

14. a whole-body image extraction unit that extracts a whole-body image from the received image; a whole-body image feature extraction unit that extracts whole-body image feature values ​​from the whole-body image extracted by the whole-body image extraction unit; a re-learning determination unit that instructs the whole-body image feature extraction unit to execute a learning process for extracting whole-body image feature values ​​from a whole-body image when the first threshold cannot be adjusted by the threshold adjustment unit; The person search device according to claim 12, further comprising:

15. A person search system having a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras in response to a search request from a monitoring terminal, The monitoring terminal, the plurality of cameras, and the person search device, The person search device includes: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; Equipped with the whole-body image feature amount correction unit acquires, from the database, a whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds a second threshold value, using a determination result from the classification unit, and when a difference value between the acquired whole-body image feature amount and the whole-body image feature amount of the specific person image exceeds a third threshold value, replaces the whole-body image feature amount of the specific person image with the whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds the second threshold value; The person search system comprises:

16. A person search system having a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras in response to a search request from a monitoring terminal, The monitoring terminal, the plurality of cameras, and the person search device, The person search device includes: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; Equipped with The person search system comprises:

17. A person search system having a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras in response to a search request from a monitoring terminal, The monitoring terminal, the plurality of cameras, and the person search device, The person search device includes: a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to obtain, from the database, shooting information associated with a whole-body image feature whose first similarity exceeds a first threshold and a face image feature whose second similarity exceeds a second threshold, and generates specific person shooting information using the obtained shooting information; Equipped with the whole-body image feature amount correction unit acquires, from the database, a whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds a second threshold value, using a determination result from the classification unit, and when a difference value between the acquired whole-body image feature amount and the whole-body image feature amount of the specific person image exceeds a third threshold value, replaces the whole-body image feature amount of the specific person image with the whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds the second threshold value; The person search system comprises:

18. A person search method for searching for an image including a search target person from among a plurality of images captured by a plurality of cameras using a person search device, comprising: a similarity calculation step in which, when a similarity calculation unit in the person retrieval device acquires whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, to calculate a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculate a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification step in which a classification unit in the person retrieval device determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correcting step in which a whole-body image feature amount correcting unit in the person retrieval device corrects the whole-body image feature amount of the specific person image; a person search step in which a person search unit in the person search device uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; Equipped with the whole-body image feature amount correction unit acquires, from the database, a whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds a second threshold value, using a determination result from the classification unit, and when a difference value between the acquired whole-body image feature amount and the whole-body image feature amount of the specific person image exceeds a third threshold value, replaces the whole-body image feature amount of the specific person image with the whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds the second threshold value; The person search method comprises:

19. A person search method for searching for an image including a search target person from among a plurality of images captured by a plurality of cameras using a person search device, comprising: a similarity calculation step in which, when a similarity calculation unit in the person retrieval device acquires whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, to calculate a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculate a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification step in which a classification unit in the person retrieval device determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a person search step in which a person search unit in the person search device uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; The person search method includes:

20. A person search method for searching for an image including a search target person from among a plurality of images captured by a plurality of cameras using a person search device, comprising: a similarity calculation step in which, when a similarity calculation unit in the person retrieval device acquires whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, to calculate a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculate a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification step in which a classification unit in the person retrieval device determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correcting step in which a whole-body image feature amount correcting unit in the person retrieval device corrects the whole-body image feature amount of the specific person image; a person retrieval step in which a person retrieval unit in the person retrieval device retrieves, from the database, shooting information associated with a whole-body image feature whose first similarity exceeds a first threshold and a face image feature whose second similarity exceeds a second threshold, using a determination result from the classification unit, and generates specific person shooting information using the retrieved shooting information; The person search method includes:

21. A person search program that causes a computer to operate as a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras, the program comprising: The computer, a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; Equipped with the whole-body image feature amount correction unit acquires, from the database, a whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds a second threshold value, using a determination result from the classification unit, and when a difference value between the acquired whole-body image feature amount and the whole-body image feature amount of the specific person image exceeds a third threshold value, replaces the whole-body image feature amount of the specific person image with the whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds the second threshold value; A person search program that operates as a person search device.

22. A person search program that causes a computer to operate as a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras, the program comprising: The computer, a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a person search unit that uses a determination result of the classification unit to acquire, from the database, shooting information linked to the face image feature whose second similarity exceeds a second threshold, calculates an area and a time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, acquires, from the database, second shooting information linked to the whole-body image feature whose first similarity exceeds a first threshold and the face image feature whose second similarity does not exceed a second threshold, and narrows down the acquired second shooting information using the area and the time period as conditions to generate specific person shooting information; A person search program that causes the person search device to operate as a person search device having the above-mentioned.

23. A person search program that causes a computer to operate as a person search device that searches for an image including a person to be searched from among a plurality of images taken by a plurality of cameras, the program comprising: The computer, a similarity calculation unit that, when acquiring whole-body image features and face image features of a person in a specific person image indicating a person to be searched for, refers to a database in which whole-body image features, face image features, and shooting information including camera identification information, shooting position, and shooting date and time are linked and stored for each image output from the multiple cameras, and calculates a first similarity indicating a similarity between the whole-body image features of the specific person image and the whole-body image features in the database, and calculates a second similarity indicating a similarity between the face image features of the specific person image and the face image features in the database; a classification unit that determines whether the first similarity exceeds a first threshold and determines whether the second similarity exceeds a second threshold; a whole-body image feature amount correction unit that corrects a whole-body image feature amount of the specific person image; a person search unit that uses a determination result of the classification unit to obtain, from the database, shooting information associated with a whole-body image feature whose first similarity exceeds a first threshold and a face image feature whose second similarity exceeds a second threshold, and generates specific person shooting information using the obtained shooting information; Equipped with the whole-body image feature amount correction unit acquires, from the database, a whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds a second threshold value, using a determination result from the classification unit, and when a difference value between the acquired whole-body image feature amount and the whole-body image feature amount of the specific person image exceeds a third threshold value, replaces the whole-body image feature amount of the specific person image with the whole-body image feature amount linked to the face image feature amount for which the second similarity exceeds the second threshold value; A person search program that operates as a person search device.

Images