Person tracking system, person tracking method, and program

Abstract

Provided is a person tracking system, wherein a calculation processing means performs processing for detecting a first person from an image, and calculates a first feature vector representing a feature of the detected first person. A data transmission processing means acquires a hash value corresponding to a key generated on the basis of information related to the first feature vector, and also causes feature data that includes the first feature vector and the key to be stored in a storage region of a storage destination external device determined on the basis of the acquired hash value. A determination processing means determines whether the first person represented by the first feature vector included in the feature data, which has been read from the external device, and a second person registered in advance are the same person. A tracking processing means tracks the first person as a person to be tracked if a determination result that the first person and the second person are the same person has been obtained.

Description

[Technical Field]

[0001] The present disclosure relates to the art of people tracking. [Background technology]

[0002] 2. Description of the Related Art Techniques relating to person tracking that track a specific person using images captured by a camera are known in the art.

[0003] Specifically, for example, Patent Document 1 discloses a technology in which a person tracking device calculates the movement path of a tracked person appearing in an image captured by a camera using information on the image capture area and the image capture time associated with the image.

[0004] Furthermore, for example, Patent Document 2 discloses a technology for identifying and tracking a moving object extracted from an image in a mobile object monitoring system that monitors the moving object across multiple cameras, by determining the correlation between the feature values ​​of the overall shape of the moving object calculated from the images taken by each of the multiple cameras and the shape of the moving object extracted from the image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-236255 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-142527 Summary of the Invention [Problem to be solved by the invention]

[0006] Here, in conventional configurations related to person tracking, the system configuration or system components such as the number of cameras, installation locations, shooting conditions, computer resources for data processing, and networks connecting multiple computers are centrally configured under fixed conditions.

[0007] Therefore, in conventional configurations for person tracking, when changes are needed to a system in operation, such as increasing the number of cameras, adding a new service and new functions to realize it, or sharing cameras with other services, it becomes necessary to review the system configuration, data flow, and the functions and performance of each element.

[0008] That is, the conventional configuration for person tracking has a problem in that it is not possible to change the configuration during operation.

[0009] One object of the present disclosure is to provide a person tracking system that can change its configuration even during operation. [Means for solving the problem]

[0010] In one aspect of the present disclosure, a person tracking system in which a plurality of cameras, at least one data processing device, and a plurality of data storage devices are connected to a common network includes: a calculation processing means provided in each of the plurality of cameras, for performing processing to detect a first person from an image and for calculating a first feature vector representing features of the detected first person; provided on each of the plurality of cameras, The first feature vector is transmitted to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices. data transmission processing means; a determination processing means provided in the data processing device for determining whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; a tracking processing means provided in the data processing device, which tracks the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person; and The determination processing means calculates a first distance by applying a predetermined calculation method to a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector, and determines that the first person and the second person are the same person if the first distance is less than or equal to a threshold, and determines that the first person and the second person are not the same person if the first distance is greater than the threshold.

[0011] In another aspect of the present disclosure, a person tracking method executed by a computer in a person tracking system in which a plurality of cameras, at least one data processing device, and a plurality of data storage devices are connected to a common network includes: each of the plurality of cameras performs processing to detect a first person from an image and calculates a first feature vector representing features of the detected first person; Each of the plurality of cameras transmitting the first feature vector to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices; the data processing device determines whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; when a determination result that the first person and the second person are the same person is obtained, the data processing device tracks the first person as a tracking target; The data processing device calculates a first distance by applying a predetermined calculation method to a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector, and if the first distance is less than or equal to a threshold, determines that the first person and the second person are the same person, and if the first distance is greater than the threshold, determines that the first person and the second person are not the same person.

[0012] According to yet another aspect of the present disclosure, there is provided a person tracking system including a plurality of cameras, at least one data processing device, and a plurality of data storage devices connected to a common network, the program being executed in the person tracking system, the program comprising: each of the plurality of cameras performs processing to detect a first person from an image and calculates a first feature vector representing features of the detected first person; Each of the plurality of cameras transmitting the first feature vector to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices; the data processing device determines whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; the data processing device causes a computer to execute a process of tracking the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person; The data processing device causes a computer to execute a process in which a first distance is calculated by applying a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector to a predetermined calculation method, and if the first distance is equal to or less than a threshold, determines that the first person and the second person are the same person, and if the first distance is greater than the threshold, determines that the first person and the second person are not the same person. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to provide a person tracking system that can change its configuration even during operation. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a person tracking system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the data storage device according to the first embodiment. [Figure 3] FIG. 1 is a block diagram showing the hardware configuration of a smart camera according to a first embodiment. [Figure 4] FIG. 1 is a block diagram showing the hardware configuration of a data processing device according to a first embodiment. [Figure 5] FIG. 1 is a diagram showing the functional configuration of a data processing device according to a first embodiment. [Figure 6] FIG. 1 is a diagram showing the functional configuration of a smart camera according to a first embodiment. [Figure 7] FIG. 2 is a diagram showing the functional configuration of a calculation processing unit according to the first embodiment. [Figure 8] FIG. 1 is a diagram showing the functional configuration of a data processing device according to a first embodiment. [Figure 9] A scatter plot generated by plotting feature vectors as data points. [Figure 10A] FIG. 10 is a diagram showing an example of the scatter plot of FIG. 9 divided into a plurality of sections. [Figure 10B] FIG. 10 is a diagram showing an example of the scatter plot of FIG. 9 divided into a plurality of sections. [Figure 11A] A histogram obtained by counting the number of data points contained in each of the multiple sections in a scatter plot divided as in Figure 10A. [Figure 11B] A histogram obtained by counting the number of data points contained in each of the multiple sections in a scatter plot divided as in Figure 10B. [Figure 12A] FIG. 11B is a diagram showing an example of a case where a section where a peak in the number of data points occurs is extracted from the histogram of FIG. 11A. [Figure 12B] FIG. 11C is a diagram showing an example of a case where a section where a peak in the number of data points occurs is extracted from the histogram of FIG. 11B. [Figure 13A] FIG. 12C is a diagram showing an example in which the data points of FIGS. 12A and 12B are superimposed. [Figure 13B] FIG. 13B is a diagram showing an example in which a representative data point is extracted from each data point in FIG. 13A. [Figure 14] 4 is a flowchart for explaining processing performed in the person tracking system according to the first embodiment. [Figure 15] FIG. 10 is a block diagram showing the functional configuration of a person tracking system according to a second embodiment. [Figure 16] 10 is a flowchart for explaining processing performed in a person tracking system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.

[0016] First Embodiment [System Configuration] FIG. 1 is a diagram showing a schematic configuration of a person tracking system according to the first embodiment.

[0017] 1, the person tracking system 1 is configured to include a plurality of data storage devices 100, a plurality of smart cameras 110, and a plurality of data processing devices 120. The plurality of data storage devices 100, the plurality of smart cameras 110, and the plurality of data processing devices 120 are each connected to a network 140. The plurality of smart cameras 110 are each placed, for example, at positions where they can capture images of the same location in a store from different angles, or at positions where they can capture images of a continuous space such as a road in sequential order with the areas shifted slightly from one another.

[0018] [Hardware configuration] FIG. 2 is a block diagram showing the hardware configuration of the data storage device according to the first embodiment.

[0019] As shown in FIG. 2, the data storage device 100 includes an interface (IF) 101, a processor 102, a memory 103, a recording medium 104, and a database 105.

[0020] The IF 101 inputs and outputs data to and from devices connected to the network 140 .

[0021] The processor 102 is a computer such as a CPU (Central Processing Unit), and controls the entire data storage device 100 by executing a program prepared in advance.

[0022] The memory 103 is configured by a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 103 is also used as a working memory while the processor 102 is executing various processes.

[0023] The recording medium 104 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the data storage device 100. The recording medium 104 stores, for example, data input via the IF 101 and processing results obtained by processing by the processor 102. The recording medium 104 also has a database 105.

[0024] The database 105 manages the data stored in the recording medium 104 .

[0025] FIG. 3 is a block diagram showing the hardware configuration of the smart camera according to the first embodiment.

[0026] As shown in FIG. 3, the smart camera 110 includes an interface (IF) 111, a processor 112, a memory 113, a recording medium 114, and a camera unit .

[0027] The IF 111 inputs and outputs data to and from devices connected to the network 140 .

[0028] The processor 112 is a computer such as a CPU, and controls the entire smart camera 110 by executing a program prepared in advance.

[0029] The memory 113 is configured by a ROM, a RAM, etc. The memory 113 is also used as a working memory while the processor 112 is executing various processes.

[0030] The recording medium 114 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the smart camera 110. Images captured by the camera unit 116 are recorded on the recording medium 114. In this embodiment, various programs executed by the processor 112 are recorded on a recording medium separate from the recording medium 114. When the smart camera 110 executes various processes, the programs recorded on the separate recording medium are loaded into the memory 113 and executed by the processor 112.

[0031] The camera unit 116 captures images of a predetermined area such as a store or a road.

[0032] FIG. 4 is a block diagram showing the hardware configuration of the data processing device according to the first embodiment.

[0033] As shown in FIG. 4, the data processing device 120 includes an interface (IF) 121, a processor 122, a memory 123, and a recording medium .

[0034] The IF 121 inputs and outputs data to and from devices connected to the network 140 .

[0035] The processor 122 is a computer such as a CPU, and controls the entire data processing device 120 by executing a program prepared in advance.

[0036] The memory 123 is configured by a ROM, a RAM, etc. The memory 123 is also used as a working memory while the processor 122 is executing various processes.

[0037] The recording medium 124 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the data processing device 120. The recording medium 124 records various programs to be executed by the processor 122. When the data processing device 120 executes various processes, the programs recorded on the recording medium 124 are loaded into the memory 123 and executed by the processor 122.

[0038] [Function Configuration] FIG. 5 is a diagram showing the functional configuration of the data storage device according to the first embodiment.

[0039] As shown in FIG. 5, the data storage device 100 includes a data sharing processing unit 201 and a storage unit 202.

[0040] The data sharing processor 201 performs processing related to the control of reading and writing of data. Specifically, the data sharing processor 201 performs, for example, processing related to parallelizing multiple data sets, processing to specify a location to store data, and processing to determine the data unit when reading and writing data. The data sharing processor 201 also functions as an interface for communicating with each device connected to the network 140.

[0041] The storage unit 202 stores data received by the data sharing processing unit 201 and the like.

[0042] Fig. 6 is a diagram showing the functional configuration of the smart camera according to the first embodiment, and Fig. 7 is a diagram showing the functional configuration of the calculation processing unit according to the first embodiment.

[0043] As shown in FIG. 6, the smart camera 110 includes an image capturing unit 211, a calculation processing unit 212, and a data sharing processing unit 213.

[0044] The photographing unit 211 photographs a predetermined area such as a store or a road to obtain an image.

[0045] The calculation processing unit 212 performs processing for detecting a person from the image obtained by the photographing unit 211, and also performs processing for calculating a feature vector representing the features of the detected person. Furthermore, the calculation processing unit 212 has an image processing unit 311, a person detection unit 312, and a feature vector calculation unit 313, as shown in FIG.

[0046] The image processing unit 311 performs image processing such as size conversion on the image obtained by the image capturing unit 211 .

[0047] The person detection unit 312 performs processing to detect a person from within an image that has been subjected to image processing by the image processing unit 311. Furthermore, when the person detection unit 312 detects a person from within an image that has been subjected to image processing by the image processing unit 311, the person detection unit 312 sets a rectangular area within the image to indicate the position of the detected person. Furthermore, when the person detection unit 312 detects multiple people within an image that has been subjected to image processing by the image processing unit 311, the person detection unit 312 sets rectangular areas within the image that correspond to each of the multiple people.

[0048] The feature vector calculation unit 313 calculates a feature vector representing the features of the person set by the person detection unit 312, based on the pixel values ​​of each pixel in the rectangular area set by the person detection unit 312. Note that when multiple people are detected by the person detection unit 312, that is, when multiple rectangular areas are set by the person detection unit 312, the feature vector calculation unit 313 calculates a feature vector corresponding to each of the multiple rectangular areas.

[0049] The data sharing processor 213 performs processing related to the control of data reading and writing. Specifically, the data sharing processor 213 performs, for example, processing related to parallelization of multiple data sets, processing to specify a location to store data, and processing to determine the data unit when reading and writing data. The data sharing processor 213 also functions as an interface for communicating with each device connected to the network 140. The data sharing processor 213 also selects one data storage device 100X from among the multiple data storage devices 100 connected to the network 140 as a destination of the feature vector acquired by the calculation processor 212. The data sharing processor 213 also transmits feature data including the feature vector acquired by the calculation processor 212 to the data storage device 100X. The feature data transmitted to the data storage device 100X is associated with location information indicating the location of the smart camera 110 that is the source of the feature data.

[0050] Here, a specific example of the processing related to transmission of feature data performed by the data sharing processing unit 213 will be described.

[0051] First, the data sharing processing unit 213 performs processing to generate a key KYX corresponding to the feature vector SVX acquired by the calculation processing unit 212. The key KYX may be generated using unique information such as the time when the feature vector SVX is calculated.

[0052] Next, the data sharing processor 213 applies the key KYX to a predetermined hash function to obtain a hash value HSA, and sets a destination for the feature vector SVX corresponding to the hash value HSA. According to such processing by the data sharing processor 213, the destination for the feature vector SVX is set to the data storage device 100X, and a memory block address ADX in the data storage device 100X (memory unit 202) where the feature vector SVX is to be stored is also set.

[0053] Finally, the data sharing processing unit 213 generates feature data TDX including the key KY and the feature vector SVX, and transmits the generated feature data TDX to the data storage device 100X. According to such processing by the data sharing processing unit 213, the feature vector SV is stored at the memory block address ADX of the data storage device 100X (memory unit 202).

[0054] In other words, the data sharing processing unit 213 functions as a data transmission processing means, acquires a hash value HSA corresponding to a key KYX generated based on information related to the feature vector SVX, and stores feature data TDX including the feature vector SVX and the key KYX in a storage area (memory block address ADX of the data storage device 100X (memory unit 202)) of the external device as a storage destination determined based on the acquired hash value HSA.

[0055] The processing according to the specific example described above is performed individually in each smart camera 110. Therefore, according to the processing according to the specific example described above, it is possible to identify that the feature vector SVX acquired by the smart camera 110 is stored in the data storage device 100X, even without performing control such as synchronizing the devices connected to the network 140.

[0056] FIG. 8 is a diagram showing the functional configuration of the data processing device according to the first embodiment.

[0057] As shown in FIG. 8, the data processing device 120 includes a data sharing processing unit 221, a conversion processing unit 222, a determination processing unit 223, and a tracking processing unit 224.

[0058] The data sharing processor 221 performs processing related to the control of data reading and writing. Specifically, the data sharing processor 221 performs, for example, processing related to parallelization of multiple data sets, processing to specify a location for storing data, and processing to determine data units for reading and writing data. The data sharing processor 221 also functions as an interface for communicating with each device connected to the network 140. The data sharing processor 221 also performs processing to read data used in processing by each unit of the data processing device 120 from the data storage device 100. The data sharing processor 221 also generates transmission data corresponding to processing results obtained by the processing by each unit of the data processing device 120, and transmits the generated transmission data to the data storage device 100. Specifically, the data sharing processor 221 performs processing similar to that of the data sharing processor 213 on a feature vector SVD (described later), thereby setting a destination for the feature vector SVD, generating feature data including the feature vector SVD, and transmitting the generated feature data to the destination.

[0059] The conversion processing unit 222 acquires the feature data TDX stored in the data storage device 100X through the data sharing processing unit 221. The conversion processing unit 222 also acquires the feature data TDY stored in the data storage device 100 through the data sharing processing unit 221.

[0060] The feature data TDY includes a feature vector SVY that represents the features of the person PY that has been registered in advance. The feature vector SVY is calculated in advance as a representative vector that best represents the features of the person PY.

[0061] The conversion processing unit 222 performs a calculation using locality sensitive hashing (hereinafter also referred to as LSH) on a feature vector SVX included in the feature data TDX, thereby converting the feature vector SVX into a hash value HSB. The conversion processing unit 222 also performs a calculation using LSH on a feature vector SVY included in the feature data TDY, thereby converting the feature vector SVY into a hash value HSC. Note that the calculation using LSH is an example of a stochastic calculation. Therefore, in this embodiment, a method other than LSH may be used as long as the feature vector SVX, the feature vector SVY, and a feature vector SVD described below are converted into hash values ​​using the same stochastic calculation.

[0062] The determination processing unit 223 calculates the distance DBC between the hash values ​​HSB and HSC. The distance DBC may be calculated using a predetermined calculation method such as Euclidean distance, Hamiltonian distance, or cosine distance.

[0063] Thereafter, the determination processing unit 223 determines whether the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVY are the same person by comparing the distance DBC with a threshold value TH. The threshold value TH may be set as a value corresponding to the calculation method of the distance DBC.

[0064] If the distance DBC is equal to or smaller than the threshold value TH, the determination processing unit 223 obtains a determination result that the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVY are the same person.

[0065] On the other hand, if the distance DBC is greater than the threshold value TH, the determination processing unit 223 determines that the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVY are not the same person. Note that such a determination result can also be obtained, for example, when the feature vector SVX representing the features of the person PX is not stored in any of the multiple data storage devices 100, or when the image obtained by photographing the person PX is unclear.

[0066] When the judgment processing unit 223 obtains a judgment result that the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVY are not the same person, it acquires N (N≧2) pieces of feature data TDZ1, TDZ2, ..., TDZN that are different from the feature data TDX and TDY from the data storage device 100 via the data sharing processing unit 221.

[0067] The feature data TDZi (1≦i≦N) includes a feature vector SVZi that represents the feature of the person PY using a vector different from the feature vector SVY.

[0068] The determination processing unit 223 plots the feature vector SVX and the N feature vectors SVZ1, SVZ2, ..., SVZN included in the N pieces of feature data TDZ1, TDZ2, ..., TDZN as data points in a predetermined coordinate system, thereby generating a scatter diagram SPD as shown in Fig. 9, for example. Fig. 9 is a scatter diagram generated by plotting the feature vectors as data points.

[0069] Thereafter, the determination processing unit 223 divides the scatter diagram SPD into a plurality of rectangular regions (hereinafter also referred to as sections) using a plurality of division methods. The determination processing unit 223 also counts the number of data points included in each of the plurality of rectangular regions, thereby generating a plurality of histograms according to the number of division methods used to divide the scatter diagram SPD.

[0070] Specifically, the determination processing unit 223 divides the scatter plot SPD into multiple sections as shown in FIG. 10A and counts the number of data points contained in each of the multiple sections to obtain a histogram HGA as shown in FIG. 11A. According to FIG. 10A, among the sections obtained by dividing the scatter plot SPD, sections with a predetermined area or more are assigned labels that allow the position of the sections to be identified as two-dimensional coordinates. FIG. 10A is a diagram illustrating an example of the scatter plot of FIG. 9 divided into multiple sections. FIG. 11A is a histogram obtained by counting the number of data points contained in each of the multiple sections in the scatter plot divided as shown in FIG. 10A.

[0071] Furthermore, the determination processing unit 223, for example, divides the scatter plot SPD into multiple sections as shown in FIG. 10B and counts the number of data points contained in each of the multiple sections to obtain a histogram HGB as shown in FIG. 11B. According to FIG. 10B, among the sections obtained by dividing the scatter plot SPD, sections with a predetermined area or more are assigned labels that allow their positions to be identified as two-dimensional coordinates. FIG. 10B is a diagram showing an example of the scatter plot of FIG. 9 divided into multiple sections. FIG. 11B is a histogram obtained by counting the number of data points contained in each of the multiple sections in the scatter plot divided as shown in FIG. 10B.

[0072] The determination processing unit 223 extracts, from among the sections obtained by dividing the scatter diagram SPD, sections in which a peak in the number of data points occurs in each of the multiple histograms.

[0073] Specifically, the determination processing unit 223 extracts sections in which peaks in the number of data points occur from among the sections obtained by dividing the scatter plot SPD as shown in Fig. 10A, based on the histogram HGA of Fig. 11A. Then, according to this processing, the determination processing unit 223 extracts sections (3,5) and (7,2) from among the sections of the scatter plot SPD labeled as shown in Fig. 10A as sections in which peaks in the number of data points occur (see Fig. 12A). Fig. 12A is a diagram showing an example of the case where sections in which peaks in the number of data points occur are extracted from the histogram of Fig. 11A.

[0074] Furthermore, the determination processing unit 223 extracts sections in which peaks in the number of data points occur from among the sections obtained by dividing the scatter plot SPD as shown in Fig. 10B, based on the histogram HGB of Fig. 11B. Then, according to this processing, the determination processing unit 223 extracts sections (3, 7) and (7, 2) from among the sections of the scatter plot SPD labeled as shown in Fig. 10B as sections in which peaks in the number of data points occur (see Fig. 12B). Fig. 12B is a diagram showing an example of the case where sections in which peaks in the number of data points occur are extracted from the histogram of Fig. 11B.

[0075] The determination processing unit 223 overlaps data points in sections where peaks in the number of data points occur in each of the multiple histograms, and extracts the data point corresponding to the median value of the overlapped data points as the representative data point DDP.

[0076] Specifically, the determination processing unit 223 generates a plot diagram PRD, such as that shown in FIG. 13A, by superimposing each data point in section (3,5) of FIG. 12A with each data point in section (3,7) of FIG. 12B, and by superimposing each data point in section (7,2) of FIG. 12A with each data point in section (7,2) of FIG. 12B. The determination processing unit 223 then extracts, for example, two representative data points DDP1 and DDP2, as shown in FIG. 13B, from the data points included in the plot diagram PRD. The determination processing unit 223 also obtains two feature vectors SVD1 and SVD2 corresponding to the two representative data points DDP1 and DDP2. FIG. 13A illustrates an example of the superimposition of the data points of FIGS. 12A and 12B. FIG. 13B illustrates an example of the extraction of representative data points from the data points of FIG. 13A.

[0077] In this embodiment, the multiple division methods used to divide the scatter plot SPD into multiple partitions are randomly determined as division methods that satisfy predetermined conditions defined by, for example, the size of each partition after division. Therefore, in this embodiment, the processes shown in FIGS. 9B to 13B can be treated as stochastic processes for acquiring feature vectors SVD1 and SVD2 using N feature vectors SVZ1, SVZ2, ..., SVZN. That is, the determination processing unit 223 of this embodiment performs stochastic processes for acquiring feature vectors SVD1 and SVD2 using multiple feature vectors SVZ1, SVZ2, ..., SVZN that represent the features of person PY using vectors different from feature vector SVY.

[0078] The conversion processing unit 222 identifies one feature vector SVD corresponding to one representative data point DDP that is closest to the data point corresponding to the feature vector SVX, from the two feature vectors SVD1 and SVD2 obtained by the determination processing unit 223. Furthermore, the conversion processing unit 222 performs a calculation using LSH on the feature vector SVD, thereby converting the feature vector SVD into a hash value HSD.

[0079] The determination processing unit 223 calculates the distance DBD between the hash values ​​HSB and HSD. The distance DBD may be calculated using the same calculation method as the distance DBC.

[0080] Thereafter, the determination processing unit 223 compares the distance DBD with a threshold value TH to determine whether the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVD are the same person.

[0081] If the distance DBD is equal to or less than the threshold value TH, the determination processing unit 223 determines that the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVD are the same person. Furthermore, when such a determination result is obtained, the determination processing unit 223 updates the feature vector SVY stored in the data storage device 100 as the representative vector that best represents the features of the person PY to the feature vector SVD.

[0082] On the other hand, if the distance DBD is greater than the threshold value TH, the determination processing unit 223 obtains a determination result that the person PX represented by the feature vector SVX and the person PY represented by the feature vector SVD are not the same person. Note that, if such a determination result is obtained, the feature vector stored in the data storage device 100 as the representative vector that best represents the features of the person PY is maintained as the feature vector SVY. Furthermore, if the determination processing unit 223 obtains the above-mentioned determination result, it stores the feature vector SVX in a location separate from the feature vector SVY, and registers the person PX represented by the feature vector SVX in the data storage device 100 as a new tracking target.

[0083] According to the processing of the determination processing unit 223 as described above, it is determined whether or not the person PX and the person PY are the same person by using the hash value obtained by converting the feature vector. Therefore, according to the processing of the determination processing unit 223 as described above, it is possible to reduce the processing load related to person tracking performed in the data processing device 120.

[0084] Furthermore, according to the processing of the determination processing unit 223 as described above, it is possible to determine whether or not the person PX and the person PY are the same person by using the feature vector SVD obtained by the processing shown in Figures 9 to 13B. Therefore, according to the processing of the determination processing unit 223 as described above, even if there is variation in the group of feature vectors representing the characteristics of the person PX, it is possible to determine with high accuracy whether or not the person PX and the person PY are the same person.

[0085] When the determination processing unit 223 obtains a determination result that the person PX and the person PY are the same person, the tracking processing unit 224 extracts P (P≧2) pieces of feature data TDX1, TDX2, ..., TDXP, each including a feature vector representing the feature of the person PX, from the data storage device 100, from among the feature data stored in the data storage device 100. Furthermore, the tracking processing unit 224 tracks the person PX as a tracking target based on the movement route of the person PX identified based on the position information linked to each of the P pieces of feature data TDX1, TDX2, ..., TDXP.

[0086] In addition, if there is a missing part in the movement path of person PX identified from P pieces of feature data TDX1, TDX2, ..., TDXP, the tracking processing unit 224 complements the missing part by performing processing using a Kalman filter or the like.

[0087] [Processing flow] Next, the flow of processing performed in the person tracking system will be described. Fig. 14 is a flowchart for explaining processing performed in the person tracking system according to the first embodiment.

[0088] First, the photographing unit 211 of the smart camera 110 photographs a predetermined area such as a store or a road and acquires an image (step S11).

[0089] Next, the calculation processing unit 212 of the smart camera 110 performs processing to detect a person from the image acquired in step S11 (step S12), and also performs processing to calculate a feature vector SVX representing the features of the detected person (step S13). Then, the feature vector SVX calculated in step S13 is transmitted from the data sharing processing unit 213 of the smart camera 110, and then stored in the data storage device 100 via the network 140.

[0090] Thereafter, the determination processing unit 223 of the data processing device 120 calculates the distance DBC between the hash value HSB obtained by converting (in the conversion processing unit 222) the feature vector SVX calculated in step S13, and the hash value HSC obtained by converting (in the conversion processing unit 222) the feature vector SVY stored in the data storage device 100 as a representative vector that best represents the features of a pre-registered person. Then, the determination processing unit 223 compares the distance DBC calculated as described above with a predetermined threshold value TH to determine whether the person represented by the feature vector SVX calculated in step S13 and the person represented by the feature vector (representative vector) SVY stored in the data storage device 100 are the same person (step S14).

[0091] If the determination processing unit 223 determines that the person represented by the feature vector SVX calculated in step S13 and the person represented by the feature vector (representative vector) SVY stored in the data storage device 100 are the same person (step S14: YES), the process proceeds to the process of step S18, which will be described later. If the determination processing unit 223 determines that the person represented by the feature vector SVX calculated in step S13 and the person represented by the feature vector (representative vector) SVY stored in the data storage device 100 are not the same person (step S14: NO), the determination processing unit 223 acquires multiple feature vectors from the data storage device 100 (step S15). Note that the multiple feature vectors acquired in step S15 do not include the feature vector SVX calculated in step S13 and the feature vector (representative vector) SVY acquired to be used in the process of step S14. Furthermore, each of the plurality of feature vectors acquired in step S15 represents the same person as the person represented by the feature vector (representative vector) SVY.

[0092] The determination processing unit 223 performs the processing illustrated in Fig. 9 to Fig. 13B using the multiple feature vectors acquired in step S15. Furthermore, the determination processing unit 223 calculates a distance DBD between a hash value HSB obtained by converting (in the conversion processing unit 222) the feature vector SVX calculated in step S13, and a hash value HSD obtained by converting (by the conversion processing unit 222) the feature vector SVD obtained by the processing illustrated in Fig. 9 to Fig. 13B. Then, the determination processing unit 223 compares the distance DBD calculated as described above with a predetermined threshold TH to determine whether the person represented by the feature vector SVX calculated in step S13 and the person represented by the feature vector SVD obtained through the processing illustrated in Fig. 9 to Fig. 13B are the same person (step S16).

[0093] 9 to 13B are the same person (step S16: YES), the determination processing unit 223 updates the representative vector stored in the data storage device 100 from the feature vector SVY to the feature vector SVD (step S17), and then proceeds to step S18, which will be described later. On the other hand, if the determination processing unit 223 determines that the person represented by the feature vector SVX calculated in step S13 and the person represented by the feature vector SVD obtained through the processes illustrated in Figures 9 to 13B are not the same person (step S16: NO), the determination processing unit 223 registers the person represented by the feature vector SVX in the data storage device 100 as a new tracking target (step S19), and then ends the series of processes.

[0094] Based on the determination result obtained in step S14 or step S16, the tracking processing unit 224 extracts a plurality of feature vectors representing the characteristics of the person identified in the determination result from the data storage device 100. Furthermore, the tracking processing unit 224 tracks the person as a tracking target based on the movement route of the person identified based on the position information of the smart cameras linked to each of the plurality of feature vectors extracted from the data storage device 100 (step S18).

[0095] As described above, the person tracking system 1 according to this embodiment allows a plurality of feature vectors acquired by each of the plurality of smart cameras 110 to be shared among a plurality of data storage devices 100, and allows the data processing device 120 to perform processing for tracking a tracking target person using the plurality of feature vectors read from any of the plurality of data storage devices 100. Furthermore, the person tracking system 1 according to this embodiment allows processing by each of the plurality of smart cameras 110 and each of the plurality of data processing devices 120 to be performed without recording or synchronizing the state of each part of the system. Therefore, the person tracking system 1 according to this embodiment allows configuration changes, such as increasing or decreasing the number of installed smart cameras 110 and the number of installed data processing devices 120, to be performed even during operation. Furthermore, the person tracking system 1 according to this embodiment allows each of the plurality of smart cameras 110 to perform processing related to calculation of a feature vector and processing for determining the data storage device 100 in which the feature vector will be stored. Therefore, the person tracking system 1 according to this embodiment can reduce, for example, the load associated with processing performed by the data processing device 120 and the load associated with communication performed over the network 140.

[0096] Second Embodiment FIG. 15 is a block diagram showing the functional configuration of a person tracking system according to the second embodiment.

[0097] The person tracking system 1A according to this embodiment includes a calculation processing means 401, a data transmission processing means 402, a determination processing means 403, and a tracking processing means 404. Each of the components constituting the person tracking system 1A has the same hardware configuration as the data processing device 120.

[0098] FIG. 16 is a flowchart illustrating the processing performed in the person tracking system according to the second embodiment.

[0099] The calculation processing means 401 performs processing for detecting a person from within an image, and calculates a first feature vector representing the features of the detected first person (step S41).

[0100] The data transmission processing means 402 acquires a hash value corresponding to the key generated based on information related to the first feature vector, and stores the feature vector and feature data including the key in a storage area of ​​the external device 500 determined based on the acquired hash value (step S42).

[0101] The determination processing means 403 determines whether the first person represented by the first feature vector included in the feature data read from the external device 500 and the pre-registered second person are the same person (step S43). Note that if the determination result indicates that the first person represented by the first feature vector included in the feature data read from the external device 500 and the pre-registered second person are not the same person (step S43: NO), the process of step S44, which will be described later, is not performed, and the series of processes in FIG. 16 ends.

[0102] When the tracking processing means 404 determines that the first person represented by the first feature vector included in the feature data read from the external device 500 and the pre-registered second person are the same person (step S43: YES), it tracks the first person as the tracking target (step S44).

[0103] According to the person tracking system 1A of this embodiment, the configuration can be changed even during operation.

[0104] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0105] (Appendix 1) a calculation processing means for performing processing to detect a first person from an image and calculating a first feature vector representing features of the detected first person; a data transmission processing means for acquiring a hash value corresponding to a key generated based on information related to the first feature vector, and storing feature data including the first feature vector and the key in a storage area of ​​an external device that is determined as a storage destination based on the acquired hash value; a determination processing means for determining whether a first person represented by the first feature vector included in the feature data read from the external device and a pre-registered second person are the same person; a tracking processing means for tracking the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person; A person tracking system having

[0106] (Appendix 2) The person tracking system of Appendix 1, wherein the determination processing means determines whether the first person and the second person are the same person based on the distance between a first hash value calculated by performing a predetermined probabilistic calculation on the first feature vector and a second hash value calculated by performing the predetermined probabilistic calculation on a second feature vector corresponding to a representative vector that best represents the features of the second person.

[0107] (Appendix 3) The person tracking system of Appendix 2, wherein when a determination result is obtained that the first person and the second person are not the same person, the determination processing means performs probabilistic processing to obtain a third feature vector using a plurality of feature vectors that represent features of the second person using vectors different from the second feature vector, and re-determines whether the first person and the second person are the same person based on the distance between the first hash value and a third hash value calculated by applying the predetermined probabilistic calculation to the third feature vector.

[0108] (Appendix 4) The person tracking system of claim 3, wherein the determination processing means updates the representative vector from the second feature vector to the third feature vector when the re-determination results in a determination that the first person and the second person are the same person.

[0109] (Appendix 5) 5. The person tracking system of claim 2, wherein the predetermined probabilistic calculation is a calculation using locality-sensitive hashing.

[0110] (Appendix 6) performing processing for detecting a first person from the image and calculating a first feature vector representing features of the detected first person; acquiring a hash value corresponding to a key generated based on information related to the first feature vector, and storing feature data including the first feature vector and the key in a storage area of ​​an external device that is determined as a storage destination based on the acquired hash value; determining whether a first person represented by the first feature vector included in the feature data read from the external device and a pre-registered second person are the same person; A person tracking method for tracking the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person.

[0111] (Appendix 7) performing processing for detecting a first person from the image and calculating a first feature vector representing features of the detected first person; acquiring a hash value corresponding to a key generated based on information related to the first feature vector, and storing feature data including the first feature vector and the key in a storage area of ​​an external device that is determined as a storage destination based on the acquired hash value; determining whether a first person represented by the first feature vector included in the feature data read from the external device and a pre-registered second person are the same person; A recording medium having a program recorded thereon that causes a computer to execute processing to track the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person.

[0112] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]

[0113] 1. People Tracking System 100 Data storage device 110 Smart Camera 120 Data Processing Device 202 Storage section 211 Photography Department 212 Computational Processing Unit 213 Data sharing processing section 223 Judgment processing unit 224 Tracking Processing Unit

Claims

1. A person tracking system in which a plurality of cameras, at least one data processing device, and a plurality of data storage devices are connected to a common network, a computing means provided in each of the plurality of cameras, for performing processing to detect a first person from an image and for calculating a first feature vector representing a feature of the detected first person; a data transmission processing means provided in each of the plurality of cameras, the data transmission processing means transmitting the first feature vector to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices; a determination processing means provided in the data processing device for determining whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; a tracking processing means provided in the data processing device, which tracks the first person as a tracking target when a determination result is obtained that the first person and the second person are the same person; and The judgment processing means calculates a first distance by applying a predetermined calculation method to a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector, and if the first distance is less than or equal to a threshold, determines that the first person and the second person are the same person, and if the first distance is greater than the threshold, determines that the first person and the second person are not the same person.

2. 2. The person tracking system of claim 1, wherein, when a determination result is obtained that the first person and the second person are not the same person, the determination processing means acquires, as a third feature vector, a feature vector that is closest to the first feature vector from among a plurality of feature vectors that are different from both the first feature vector and the second feature vector and that are read from any of the plurality of data storage devices, calculates a second distance by applying the first hash value and a third hash value calculated by performing a calculation using locality-sensitive hashing on the third feature vector to the predetermined calculation method, and determines whether the first person represented by the first feature vector and the second person represented by the third feature vector are the same person based on the second distance and the threshold.

3. 3. The person tracking system of claim 2, wherein the determination processing means updates the second feature vector stored in the second data storage device to the third feature vector when it determines that the first person represented by the first feature vector and the second person represented by the third feature vector are the same person.

4. 3. The person tracking system of claim 2, wherein the determination processing means, when obtaining a determination result that the first person represented by the first feature vector and the second person represented by the third feature vector are not the same person, registers the first person in the first data storage device as a new tracking target.

5. 1. A computer-implemented person tracking method for a person tracking system in which a plurality of cameras, at least one data processing device, and a plurality of data storage devices are connected to a common network, comprising: each of the plurality of cameras performs processing for detecting a first person from an image and calculates a first feature vector representing features of the detected first person; each of the plurality of cameras transmits the first feature vector to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices; the data processing device determines whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; when a determination result that the first person and the second person are the same person is obtained, the data processing device tracks the first person as a tracking target; The data processing device calculates a first distance by applying a predetermined calculation method to a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector, and if the first distance is less than or equal to a threshold, determines that the first person and the second person are the same person, and if the first distance is greater than the threshold, determines that the first person and the second person are not the same person.

6. A program executed in a person tracking system in which a plurality of cameras, at least one data processing device, and a plurality of data storage devices are connected to a common network, the program comprising: each of the plurality of cameras performs processing for detecting a first person from an image and calculates a first feature vector representing features of the detected first person; each of the plurality of cameras transmits the first feature vector to a first data storage device, which is a destination set using unique information related to the first feature vector, among the plurality of data storage devices; the data processing device determines whether the first person and the second person are the same person based on a distance calculated according to the first feature vector read from the first data storage device and a second feature vector read from a second data storage device as a vector representing a feature of a second person pre-registered in the second data storage device among the plurality of data storage devices; the data processing device causes a computer to execute a process of tracking the first person as a tracking target when a determination result that the first person and the second person are the same person is obtained; The data processing device applies a predetermined calculation method to a first hash value calculated by performing a calculation using locality-sensitive hashing on the first feature vector and a second hash value calculated by performing a calculation using locality-sensitive hashing on the second feature vector to calculate a first distance, and determines that the first person and the second person are the same person if the first distance is equal to or less than a threshold, and determines that the first person and the second person are not the same person if the first distance is greater than the threshold.

Images