Person retrieval apparatus, person retrieval method, and person retrieval program

Abstract

To provide a person retrieval apparatus, method, and program capable of retrieving a target person without performing comprehensive comparison.SOLUTION: A person retrieval apparatus includes: a feature clustering unit 170 which assigns cluster numbers for uniquely identifying full-body features of persons as one cluster, regarding features of full bodies of a plurality of persons captured, generates a third cluster by combining a first cluster having a first cluster number with a second cluster having a second cluster number, assigns a third cluster number with the number of combinations reflected therein, to the third cluster, and generates a data group composed of a plurality of clusters for the captured persons; and a person retrieval unit 160 which refers, on receipt of an image of a target person, to the generated data group to retrieve the target person.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a person search technology for searching for an image containing a specific person from among a plurality of images. [Background technology]

[0002] Among techniques for identifying a person included in an image, there is a technique for identifying a person using features of the person in the image. For example, Patent Document 1 describes a technique for extracting a first feature related to the person's face and a second feature related to the person's body from an image including the person captured by an imaging unit, and identifying the person based on a third feature calculated by weighting the extracted first feature and second feature. More specifically, it describes a technique for an information integration unit to calculate a third feature by integrating the first feature extracted by the face area information extraction unit and the second feature calculated by the body area information extraction unit, and to calculate a third probability indicating the probability that the person included in the person area matches a person associated with face information or body information stored in a storage unit according to the third feature (paragraph 0067 of Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-23785 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the conventional method described in Patent Document 1, the probability that the search target person, who is the search target represented by the third feature, matches a person associated with face information or body information stored in a memory unit is calculated, which has the problem that it is necessary to comprehensively compare the search target person with people whose related information is stored in a memory unit.

[0005] The present disclosure has been made to solve such problems, and aims to provide a person search technology that can search for a person to be searched for without performing an exhaustive comparison. [Means for solving the problem]

[0006] One aspect of a person retrieval device according to an embodiment of the present disclosure includes a feature clustering unit that assigns cluster numbers to whole-body feature amounts of multiple captured people, which uniquely identify the whole-body feature amounts of each person as one cluster, combines a first cluster having a first cluster number with a second cluster having a second cluster number to generate a third cluster, assigns a third cluster number to the third cluster that reflects the number of times the third cluster is combined, and generates a data set consisting of multiple clusters for the captured people, and a person retrieval unit that, upon receiving an image of a person to be searched for, searches for the person to be searched for by referring to the generated data set. [Effects of the Invention]

[0007] According to the person search technology according to the embodiment of the present disclosure, it is possible to search for a person to be searched for without performing an exhaustive comparison. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a person search system including a person search device according to the present disclosure. [Figure 2] 1 is a diagram illustrating a configuration of a person search system including a person search device according to a first embodiment of the present disclosure. [Figure 3] 4 is a flowchart showing a data generation process in the person retrieval device according to the first embodiment of the present disclosure. [Figure 4] 4 is a flowchart showing a clustering process in the person retrieval device according to the first embodiment of the present disclosure. [Figure 5] 4 is a flowchart showing a person search process in the person search device according to the first embodiment of the present disclosure. [Figure 6]4 is a flowchart showing a feature amount comparison process in the person retrieval device according to the first embodiment of the present disclosure. [Figure 7] FIG. 10 is a diagram illustrating a configuration of a person retrieval system including a person retrieval device according to a second embodiment of the present disclosure. [Figure 8] 10 is a flowchart showing a database data reduction process in the person retrieval device according to the second embodiment of the present disclosure. [Figure 9A] FIG. 2 is a diagram illustrating an example of the hardware configuration of a person search device according to the present disclosure. [Figure 9B] FIG. 2 is a diagram illustrating an example of the hardware configuration of a person search device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Components with the same or similar reference numerals in the drawings have the same or similar configurations or functions, and redundant descriptions of such components will be omitted. Furthermore, in this disclosure, the term "or" means an inclusive logical OR unless otherwise specified.

[0010] Embodiment 1 <Configuration> (People search system) 1 and 2, a person retrieval device 100 and a person retrieval system 1 including the person retrieval device 100 according to the first embodiment of the present disclosure will be described. FIG. 1 is a diagram illustrating the person retrieval system 1 including the person retrieval device 100 according to the present disclosure. FIG. 2 is a diagram illustrating the configuration of the person retrieval system 1 including the person retrieval device 100 according to the first embodiment of the present disclosure.

[0011] The person retrieval system 1 is a system that searches for an image containing a person to be searched for from among a plurality of images taken by a plurality of cameras 400, such as surveillance cameras. The person retrieval system 1 shown in FIG. 1 includes a person retrieval device 100, a database device 200, a monitoring terminal 300, a camera 400, and an image recording device 500.

[0012] In the person retrieval system 1, the person retrieval device 100 is communicably connected to a database device 200, a monitoring terminal 300, a camera 400, and an image recording device 500. The monitoring terminal 300, the camera 400, and the image recording device 500 are communicably connected to one another.

[0013] 2, the person retrieval system 1 includes a person retrieval device 100 and a database device 200 connected by a communication line, and the person retrieval device 100, a monitoring terminal 300, a camera 400, and an image recording device 500 are mutually connected via a communication network 600 such as an IP network. The database device 200 may also be configured to be connected to other devices using the communication network 600.

[0014] (Person search device; overview) One of the key points of the present disclosure is that a feature amount clustering unit 170 or a data reduction unit 180 is added to an existing person search device (Japanese Patent Application Laid-Open No. 2019-23785). The detailed device configuration will be described below.

[0015] The person retrieval device 100 searches for an image containing a person to be searched for from among a plurality of images taken by a plurality of cameras 400. Furthermore, the person retrieval device 100 extracts full-body image features of the person included in the image from the image taken by each camera 400, associates them with shooting information indicating the shooting conditions of the image, and stores them in the database device 200.

[0016] The person retrieval device 100 receives a specific person image including the person to be searched and a search request from, for example, the monitoring terminal 300 via the communication network 600. After acquiring full-body image features of the person in the specific person image showing the person to be searched, the person retrieval device 100 refers to the database device 200 and calculates the similarity between the full-body image features of the specific person image and the full-body image features in the database device 200. The person retrieval device 100 also uses the determination result to calculate, from the database device 200, an area and a time period in which the person shown in the specific person image may have been photographed, and generates specific person photograph information including camera identification information for the area and the time period. The person retrieval device 100 outputs the specific person photograph information as a search result to the monitoring terminal 300 via the communication network 600. The search result may include an image identified in the specific person photograph information. Details of the person retrieval device 100 will be described later.

[0017] (database device) The database device 200 stores, in association with each other, whole-body image features and shooting information 240 including camera identification information, shooting position, and shooting date and time for each image output from a plurality of cameras 400. More specifically, the database device 200 has a program 210, whole-body information 220, a similarity determination threshold 230, and shooting information 240.

[0018] The program 210 is a program for causing a computer to operate as the person retrieval device 100 .

[0019] The whole-body information 220 includes whole-body image features for each image and a person's identification number. The whole-body image features are values ​​that represent the whole-body features of each person, such as clothing, physique, and camera angle of view.

[0020] The similarity determination threshold 230 is used when the person search unit 160, which will be described later, compares the whole-body image feature amount of the specific person image with the whole-body image feature amount of the database device 200.

[0021] The photography information 240 indicates the photography conditions for each camera, and includes at least the camera identification information, the photography position, and the photography date and time. The camera identification information is, for example, a camera number that is different for each camera.

[0022] In the database device 200, when a command to store information is received, new information is added or the information is updated. Furthermore, the database device 200 presents the stored data in response to a request for presentation from the person retrieval device 100. Note that the database device 200 may be configured to be included inside the person retrieval device 100.

[0023] (Monitoring terminal) The monitoring terminal 300 requests the person retrieval device 100 to search for images of the same person as the person shown in the specific person image indicating the person to be searched for. The monitoring terminal 300 is also configured to be able to acquire images captured by the camera 400 via the communication network 600. The monitoring terminal 300 is also configured to be able to acquire images recorded in the image recording device 500 via the communication network 600. The monitoring terminal 300 is also configured to be able to acquire images from an external source other than the camera 400 and the image recording device 500. The specific person image is an image designated from among the images acquired by the monitoring terminal 300.

[0024] After requesting an image search, the monitoring terminal 300 receives a search result from the person retrieval device 100. The search result is, for example, specific person photographic information generated by the person retrieval device 100. In this case, the monitoring terminal 300 acquires an image from the camera 400 or the image recording device 500 using the specific person photographic information. However, if the search result includes an image, the monitoring terminal 300 does not need to perform processing using the specific person photographic information.

[0025] (Person search device; details) The following describes in detail the person retrieval device 100. The person retrieval device 100 includes a camera image receiving unit 110, a whole-body image extraction unit 120, a whole-body image feature extraction unit 130, a person feature storage unit 140, a similarity calculation unit 150, a person retrieval unit 160, a feature clustering unit 170, and a control unit (not shown).

[0026] (Camera image receiving section) The camera image receiving unit 110 acquires images (camera images) and shooting information taken by the multiple cameras 400. Specifically, the camera image receiving unit 110 receives camera images distributed from the multiple cameras 400 at a constant frame rate, and outputs the received camera images to the whole-body image extracting unit 120.

[0027] (Whole-body image extraction section) The whole-body image extraction unit 120 receives an image and extracts a whole-body image of a person from the received image. For example, when the whole-body image extraction unit 120 receives an image, it extracts a whole-body region of the person based on an image of the person that has been learned in advance by machine learning such as deep learning, and outputs an image of the whole-body region (whole-body image) to the whole-body image feature extraction unit 130.

[0028] (Whole-body image feature extraction unit) The whole-body image feature extraction unit 130 extracts whole-body image feature amounts from the whole-body image extracted by the whole-body image extraction unit 120. Specifically, the whole-body image feature extraction unit 130 extracts whole-body image feature amounts from the whole-body image of the camera image acquired by the camera image receiving unit 110, and outputs the whole-body image feature amounts to the person feature storage unit 140. In this case, when the whole-body image feature extraction unit 130 receives the whole-body image, it extracts whole-body image feature amounts (X1) that digitize features such as clothing, physique, and camera angle of view based on an image of a person that has been learned in advance by machine learning such as deep learning, and outputs the whole-body image feature amounts of the camera image to the person feature storage unit 140.

[0029] Specifically, the whole-body image feature extraction unit 130 extracts whole-body image feature values ​​from the whole-body image of the specific person image and outputs the extracted feature values ​​to the similarity calculation unit 150. In this case, when the whole-body image of the specific person image is received, the whole-body image feature extraction unit 130 extracts whole-body image feature values ​​(X1) that quantify features such as clothing, physique, and camera angle of view based on an image of the person that has been learned in advance by machine learning such as deep learning, and outputs the whole-body image feature values ​​of the specific person image to the similarity calculation unit 150.

[0030] (Personal Feature Storage Unit) The person feature storage unit 140 acquires the whole-body image feature of the image captured by each camera and the image capture information of the image, links the acquired whole-body image feature and the image capture information, and stores them in the database device 200. Specifically, the person feature storage unit 140 also stores the whole-body image feature extracted from the same image, the camera number, the capture position, the capture date and time, a thumbnail image cut out from the whole-body image, and the like.

[0031] (Similarity calculation part) When the similarity calculation unit 150 acquires the whole-body image feature of the person in the specific person image indicating the person to be searched, it refers to the database device 200 and calculates the similarity between the whole-body image feature of the specific person image and the whole-body image feature in the database device 200. The similarity indicates the degree to which the compared images are similar, and takes a value between 0 and 1, for example, with the closer to 1 the value is, the higher the similarity.

[0032] (People Search Department) The person search unit 160 acquires from the database device 200 whole-body image features whose similarity exceeds the similarity determination threshold 230 and shooting information linked to the whole-body image features, calculates the area and time period in which the person shown in the specific person image may have been photographed using the acquired shooting information, and generates specific person shooting information including camera identification information within the area and the time period. Specifically, the person search unit 160 receives the specific person image together with a search request from the monitoring terminal 300 and outputs the specific person image to the whole-body image extraction unit 120. After outputting the specific person image to the whole-body image extraction unit 120, the person search unit 160 notifies the monitoring terminal 300 of the search results.

[0033] More specifically, the person search unit 160 selects a certain number of images in descending order of similarity, for example. The certain number is not particularly limited as long as it is the number necessary for the search results. The person search unit 160 also acquires information such as camera positions, camera adjacency, and relative distances, and calculates the area and time period in which the person to be searched for, shown in the specific person image, may have been photographed. The person search unit 160 outputs the specific person photograph information as the search results.

[0034] Here, in order to avoid biasing search results toward a specific camera or time period, person search unit 160 may perform thinning such that multiple images from the same camera are not selected within a certain time period. In this case, when generating specific person shooting information, if multiple pieces of camera identification information indicating the same camera are included, person search unit 160 thins out the camera identification information indicating the same camera at predetermined time intervals.

[0035] The person search unit 160 may further output, as a search result, the specific person shooting information and the image captured by the camera identified by the camera identification information of the specific person shooting information to the monitoring terminal 300 that requested the search.

[0036] (Feature clustering part) The feature clustering unit 170 is a functional unit that aggregates multiple whole-body image features into one whole-body image feature. More specifically, in one aspect, the feature clustering unit 170 has a function of assigning a cluster number to the whole-body features of multiple captured people, which uniquely identifies the whole-body feature of each person as one cluster, combining a first cluster having a first cluster number with a second cluster having a second cluster number to generate a third cluster, assigning a third cluster number to the third cluster that reflects the number of times the clusters have been combined, and generating a data set consisting of multiple clusters for the captured people.

[0037] To perform this function, the feature clustering unit 170 designates each of the multiple whole-body image features as a cluster, and combines any one cluster with the closest cluster among the remaining clusters to generate a new cluster. The distance between clusters can be, for example, the average distance between the features included in each cluster. The feature clustering unit 170 assigns a unique cluster number to the combined cluster that can distinguish the clusters from each other and indicates the number of times the clusters have been combined.

[0038] As an example, assume that there are nine clusters: cluster 1, cluster 2, cluster 3, ..., cluster 9. If cluster 2 is closest to cluster 1, the feature clustering unit 170 combines cluster 1 and cluster 2, assigns a new cluster number of "12," and generates cluster 12.

[0039] If the distance between cluster 5 and cluster 6 is the shortest, the feature clustering unit 170 combines clusters 5 and 6, assigns a new cluster number of "56", and generates cluster 56. If the distance between cluster 56 and cluster 4 is the shortest, the feature clustering unit 170 combines cluster 4 and cluster 56, assigns a new cluster number of "456", and generates cluster 456.

[0040] If the distance of cluster 9 to cluster 8 is the shortest, the feature amount clustering unit 170 combines clusters 8 and 9, assigns a new cluster number of "89", and generates cluster 89. If the distance of cluster 89 to cluster 7 is the shortest, the feature amount clustering unit 170 combines clusters 7 and 89, assigns a new cluster number of "789", and generates cluster 789.

[0041] The clusters 12, 3, 456, and 789 thus generated may be combined into a single cluster representing the vertex.

[0042] In this way, the feature clustering unit 170 aggregates nine clusters 1 to 9 into four clusters, cluster 12, cluster 3, cluster 456, and cluster 789, and integrates them into a single cluster representing the vertex. The numbers assigned to the generated clusters are unique numbers that allow the clusters to be identified. The numbers assigned to the generated clusters also indicate the number of times they are combined. For example, cluster 12 indicates that it has been combined once. Cluster 3 indicates that it has not been combined at all. Cluster 456 or cluster 789 indicates that it has been combined twice.

[0043] In this way, the feature amount clustering unit 170 assigns to the combined clusters a unique cluster number that can distinguish between classes and indicate the number of times of combining.

[0044] By repeating this combining until the number of clusters becomes one, the feature amount clustering unit 170 aggregates a plurality of whole-body image feature amounts (a plurality of clusters) into one whole-body image feature amount (one cluster).

[0045] Next, the hardware configuration of the person retrieval device 100 will be described with reference to Figures 9A and 9B. Each function of the person retrieval device 100 is realized by a processing circuitry. The processing circuitry may be a dedicated processing circuit 100a as shown in Figure 9A, or a processor 100b that executes a program stored in a memory 100c as shown in Figure 9B.

[0046] When the processing circuitry is a dedicated processing circuit 100a, the dedicated processing circuit 100a may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (application specific integrated circuit), an FPGA (field-programmable gate array), or a combination thereof. The functions of the person retrieval device 100 may be realized by a plurality of separate processing circuits, or the functions of the person retrieval device 100 may be realized together by a single processing circuit.

[0047] When the processing circuitry is the processor 100b, the functions of the person retrieval device 100 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 100c. The processor 100b realizes the functions of the person retrieval device 100 by reading and executing the programs stored in the memory 100c. The person retrieval device 100 is realized by the processor 100b, and the database device 200 is realized by the memory 100c. Here, examples of the memory 100c include non-volatile or volatile semiconductor memories such as RAM (random access memory), ROM (read-only memory), flash memory, EPROM (erasable programmable read-only memory), and EEPROM (electrically erasable programmable read-only memory), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs.

[0048] Note that some of the functions of the person retrieval device 100 may be realized by dedicated hardware, and other functions may be realized by software or firmware. In this way, the processing circuit can realize the functions of the person retrieval device 100 by hardware, software, firmware, or a combination of these.

[0049] <Operation> (Data generation process) The following describes the operation of the person retrieval device 100. First, with reference to Fig. 3, a process of generating data to be stored in the database device 200 referenced by the person retrieval device 100 and storing the generated data in the database device 200 will be described. Fig. 3 is a flowchart showing the data generation process performed by the person retrieval device 100 according to the first embodiment of the present disclosure.

[0050] (Step S1) The person retrieval device 100 starts the data generation process when, for example, a control unit (not shown) receives a command to start the process. In step S1, the camera image receiving unit 110 acquires images. Specifically, the camera image receiving unit 110 acquires images and shooting information captured by multiple cameras.

[0051] (Step S2) In step S2, the whole-body image extraction unit 120 receives the camera image from the camera image reception unit 110, and extracts a whole-body image of the person from the image acquired by the camera image reception unit 110.

[0052] (Step S3) In step S3, the whole-body image feature extraction unit 130 extracts whole-body image feature amounts from the whole-body image extracted by the whole-body image extraction unit 120. Specifically, the whole-body image feature extraction unit 130 extracts whole-body image feature amounts from the whole-body image of the camera image acquired by the camera image receiving unit 110, and outputs the extracted whole-body image feature amounts to the person feature storage unit 140.

[0053] (Step S4) In step S4, the person feature storage unit 140 stores the feature amounts of the whole-body image in the database device 200. Specifically, the person feature storage unit 140 acquires the whole-body image feature amounts of the images captured by each camera and the shooting information of the images, and stores in the database device 200 set information linking the acquired whole-body image feature amounts and the shooting information.

[0054] After storing the set information in the database device 200, the person retrieval device 100 waits until the next processing starts.

[0055] (Feature clustering processing) Next, the feature amount clustering process will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the feature amount clustering process performed by the person retrieval device 100 according to the first embodiment of the present disclosure.

[0056] (Step S5) The person retrieval device 100 starts the feature amount clustering process when, for example, a control unit (not shown) receives a command to start the process. In step S5, the feature amount clustering unit 170 designates each of all the feature amounts in the database device 200 as a cluster.

[0057] (Step S6) In step S6, the feature clustering unit 170 selects one of the multiple clusters and combines the selected cluster with the cluster closest to it. The feature clustering unit 170 assigns a unique cluster number to the combined cluster. This allows each cluster in the intermediate processing results to be referenced even after all clusters have been combined. The distance between clusters may be determined freely using the average distance between the features contained in each cluster, for example.

[0058] (Step S7) In step S7, the feature amount clustering unit 170 determines whether the number of clusters has become 1. If the number of clusters has become 1 ("YES" in step S7), the feature amount clustering process ends. If the number of clusters has not become 1 ("NO" in step S7), the process returns to step S6.

[0059] (Person search processing) Next, the person search process will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the person search process performed by the person search device 100 according to the first embodiment of the present disclosure.

[0060] (Step S8) The person retrieval device 100 starts the feature amount clustering process, for example, when a control unit (not shown) receives a command to start the process. In step S8, the person retrieval unit 160 receives a search request from the monitoring terminal 300 and acquires a specific person image including the person to be searched for. The person retrieval unit 160 outputs the specific person image to the whole-body image extraction unit 120.

[0061] (Step S9) In step S9, The whole-body image extracting unit 120 extracts a whole-body image of a person from the specific person image acquired from the monitoring terminal 300.

[0062] (Step S10) In step S10, the whole-body image feature extraction unit 130 extracts whole-body image feature values ​​from the extracted whole-body image. Specifically, upon receiving the whole-body image of the specific person image, the whole-body image feature extraction unit 130 extracts whole-body image feature values ​​(X1) and outputs the whole-body image feature values ​​of the specific person image to the similarity calculation unit 150.

[0063] (Step S11) In step S11, the similarity calculation unit 150 calculates the similarity between the whole-body image feature amount of the person to be searched and the whole-body image feature amount in the database device 200.

[0064] (Step S12) In step S12, the similarity calculation unit 150 compares the feature amounts. This process will be described later with reference to FIG.

[0065] (Step S13) In step S13, person search unit 160 notifies monitoring terminal 300 of the search results.

[0066] (Feature comparison processing) Next, the feature amount comparison process will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the feature amount comparison process of Fig. 5.

[0067] (Step S12-1) In step S12-1, the similarity calculation unit 150 compares features in a breadth-first order, starting from the apex of the cluster. That is, starting from the apex of the cluster, the feature comparison of the target person is performed on the clusters closest to the apex of the cluster. In the example where a data set is generated based on the above-mentioned clusters 1 to 9, the feature comparison is performed with the four clusters closest to the apex of the cluster: cluster 12, cluster 3, cluster 456, or cluster 789. After this comparison is completed, the feature of the target person is compared with cluster 1, cluster 2, cluster 4, cluster 56, cluster 7, or cluster 89, which is second closest to the apex of the cluster. After this comparison is completed, the feature of the target person is compared with cluster 5, cluster 6, cluster 8, or cluster 9, which is third closest to the apex of the cluster.

[0068] (Step S12-2; Step S12-3) If the termination condition is met in step S12-2, the process proceeds to step S12-3. In step S12-3, the similarity calculation unit 150 outputs similar features from among the compared features. One or more features may be output. The termination condition may be, for example, that a specified processing time has been reached or that a next search request has been received, and the search results may be returned without comparing with all features in the database.

[0069] As described above, by performing a search process on a data set clustered by the feature clustering unit, it is possible to compare features starting with those with the highest priority. That is, it is possible to compare a search target person with clusters having a small number of joins. When searching for a search target person, the search target person may have unique features that are significantly different from those of other people. Therefore, according to the person search device 100 configured as in the first embodiment, it is possible to complete the search without comprehensively comparing the information of the search target person with the information of people stored in the storage unit. Therefore, it is possible to configure an adaptive system according to various conditions, such as the processing performance of the processor and the timing of the search request.

[0070] Embodiment 2 <Configuration> 7 and 8, a second embodiment will be described in which the amount of data in a database is reduced by further providing a data reduction unit that organizes data in the database according to priority in addition to the first embodiment. FIG. 7 is a diagram showing a system configuration including a data reduction unit 180 according to the second embodiment. As shown in FIG. 7, a person retrieval system 1A including a person retrieval device 100A according to the second embodiment has a configuration in which a data reduction unit 180 is added to the person retrieval device 100 according to the first embodiment. The data reduction unit 180 selects unnecessary data from the database and reduces the selected data.

[0071] <Operation> The operation of the person retrieval device 100A will be described below with reference to Fig. 8. Fig. 8 is a flowchart showing the operation when reducing the database in the second embodiment.

[0072] (Step S16) The data reduction unit 180 accesses the database device 200 and performs a depth-first search starting from the vertex of the feature cluster. That is, starting from the vertex of the cluster, the search proceeds in the depth direction of the collection of clusters (tree) until it reaches a dead end, at which point it returns to the previous cluster and repeats the search. In the example where a data set is generated based on the above-mentioned clusters 1 to 9, the search proceeds in the order of cluster 12 → cluster 1 → cluster 2 → cluster 3 → cluster 456 → cluster 4 → cluster 56 → cluster 5 → cluster 6 → cluster 789 → cluster 7 → cluster 89 → cluster 8 → cluster 9, for example.

[0073] (Step S17; Step S18) If the searched feature cluster satisfies the deletion condition, the data reduction unit 180 deletes the cluster that satisfies the deletion condition. Deletion is repeated until the termination condition is met, and when the termination condition is met, the deletion process of FIG. 8 ends. The reduction condition is, for example, that the clusters are at a certain depth or more. This makes it possible to delete clusters with many similar features, and to delete data other than valid data that is likely to be searched. It is also possible to delete data older than a specified shooting time, or data acquired with a specified camera number. It is also possible to use the number of clusters or the number of data in the database as the termination condition.

[0074] As described above, the person retrieval device is configured to have a data reduction unit that can delete data in the database, which provides the effect of reducing the amount of data in the database.

[0075] It is possible to combine the embodiments, and to modify or omit each embodiment as appropriate. [Industrial Applicability]

[0076] The person search device of the present disclosure can be used as a device constituting a person search system for searching for people such as suspicious individuals. [Explanation of symbols]

[0077] 1 (1A) person search system, 100 (100A) person search device, 110 camera image receiving unit, 120 whole-body image extraction unit, 130 whole-body image feature extraction unit, 140 person feature storage unit, 150 similarity calculation unit, 160 person search unit, 170 feature clustering unit, 180 data reduction unit, 200 database device, 210 program, 220 whole-body information, 230 similarity determination threshold, 240 shooting information, 300 monitoring terminal, 400 camera, 500 image recording device, 600 communication network.

Claims

1. assigning a cluster number to the whole-body feature amounts of the captured images of the plurality of people, which uniquely identifies the whole-body feature amount of each person as one cluster; generating a third cluster by combining a first cluster having a first cluster number with a second cluster having a second cluster number, and assigning a third cluster number to the third cluster that reflects the number of combinations; generating a data set consisting of a plurality of clusters for the plurality of imaged people; a feature clustering unit; a person search unit that, upon receiving an image of a person to be searched for, searches for the person to be searched for by referring to the generated data set; A person search device comprising:

2. the person search unit searches for the search target person from a cluster having a cluster number with a smaller number of connections; 2. The person search device according to claim 1.

3. a data reduction unit that deletes data belonging to clusters with cluster numbers having larger numbers of joins from the generated data set; The person retrieval device according to claim 1 or 2, further comprising:

4. A person search method performed by a person search device including a feature amount clustering unit and a person search unit, The feature amount clustering unit assigning a cluster number to the whole-body feature amounts of the captured images of the plurality of people, which uniquely identifies the whole-body feature amount of each person as one cluster; generating a third cluster by combining a first cluster having a first cluster number with a second cluster having a second cluster number, and assigning a third cluster number to the third cluster that reflects the number of combinations; generating a data set consisting of a plurality of clusters for the plurality of imaged people; Steps and When the person search unit receives an image of the person to be searched for, the person search unit searches for the person to be searched for by referring to the generated data set; A person search method comprising:

5. assigning a cluster number to the whole-body feature amounts of the captured images of the plurality of people, which uniquely identifies the whole-body feature amount of each person as one cluster; generating a third cluster by combining a first cluster having a first cluster number with a second cluster having a second cluster number, and assigning a third cluster number to the third cluster that reflects the number of combinations; generating a data set consisting of a plurality of clusters for the plurality of imaged people; Function and a function of receiving an image of a person to be searched for, and then searching for the person to be searched for by referring to the generated data set; A person search program that causes a computer to execute the following.

Images