Person identification method, person identification system, and person identification program

By leveraging short-range wireless communication and multiple cameras, the method effectively identifies individuals in camera images with reduced antenna usage, optimizing infrastructure costs and improving tracking accuracy.

JP7582236B2Active Publication Date: 2024-11-13TOYOTA JIDOSHA KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022040152
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-03-15
Publication Date
2024-11-13
Estimated Expiration
2042-03-15

AI Technical Summary

Technical Problem

The challenge is to identify individuals in camera images efficiently without increasing the number of antennas beyond the number of cameras, as doing so would increase costs while improving location accuracy and identification precision.

Method used

A method and system that utilizes a combination of short-range wireless communication with user terminals and long-range communication with antennas, along with multiple cameras, to generate tracking support and confirmation information, enabling identification of individuals across different camera ranges.

Benefits of technology

This approach allows for accurate identification of individuals in camera images even when the number of antennas is less than the number of cameras, reducing infrastructure costs and enhancing tracking efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582236000001
    Figure 0007582236000001
  • Figure 0007582236000002
    Figure 0007582236000002
  • Figure 0007582236000003
    Figure 0007582236000003
Patent Text Reader

Abstract

To provide a person identification method, a person identification system, and a person identification program, capable of identifying a person included in a camera image even when the total number of antennas for transmitting and receiving user information is smaller than the total number of installed cameras.SOLUTION: In a person identification system, a management server 5 includes: an information acquisition unit 53 which acquires various kinds of information from the outside and stores them in a memory 52; a first identification unit 54 which executes processing for recognizing a person included in an image included in camera information CAM1; a first display control unit 55 which generates image data DIMG based on an image of a shooting range processed in first identification processing of the first identification unit and outputs the image data to an external device; a second identification unit 56 which executes processing for recognizing a person included in an image included in camera information CAM2; and a second display control unit 57 which generates image data DIMG based on an image of a shooting range processed in second identification processing of the second identification unit 56 and outputs the image data to the external device.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a method, system, and program for combining camera images and communications to identify people in the camera images. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2016-122300 discloses a system that identifies a person included in a camera image by combining a camera image and communication. This conventional system includes a beacon device that is installed near an imaging device (camera) and transmits a detection signal to a user's mobile terminal, and an image processing device that performs a person identification process. When the user's mobile terminal receives a detection signal from the beacon device, the user's mobile terminal transmits identification information of the beacon device and user information to the image processing device. This user information includes identification information given to the user, characteristic information such as a face image of the user, or a combination of these pieces of information.

[0003] In the identification process performed by the image processing device, it is predicted that a user is present within the angle of view of a camera installed near the beacon device based on the identification information of the beacon device and the time information when the user information was acquired. In the identification process, a person is extracted from the camera image by a recognition process using the camera image acquired by the camera and the time information when the user information was acquired. Then, the characteristic information of the extracted person is compared with the characteristic information included in the user information, thereby identifying the user included in the camera image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-122300 A [Patent Document 2] JP 2012-008802 A [Patent Document 3] JP 2020-194295 A Summary of the Invention [Problem to be solved by the invention]

[0005] Consider a case where the beacon device in the conventional system is replaced with an antenna capable of transmitting and receiving data. In this case, the antenna acquires user information by short-distance wireless communication with a mobile terminal, for example, and transmits the information to an external device (for example, an image processing device) by long-distance communication. A system including such an antenna makes it possible to identify a person included in a camera image by combining a camera image and communication, as in the conventional system.

[0006] Here, an increase in the number of cameras that make up the system is desirable from the standpoint of monitoring people in camera images and crime prevention using camera images. Also, an increase in the number of antennas installed is expected to increase the accuracy of location information of mobile terminals (i.e., users) and improve the accuracy of identifying people in camera images. On the other hand, an increase in the number of antennas installed raises concerns about increased costs. Therefore, there is a need to develop technology that can identify people in camera images even in a situation where the number of antennas is fewer than the total number of cameras installed.

[0007] One object of the present disclosure is to provide a technology capable of identifying a person included in a camera image even when the number of antennas for transmitting and receiving user information is fewer than the total number of installed cameras. [Means for solving the problem]

[0008] The first aspect is a method for identifying a person included in a camera image, which has the following features. The method comprises: acquiring user information assigned to the user terminal by short-range wireless communication using an antenna, and acquiring the user information by long-range communication using the antenna; acquiring an image of a first range including at least a portion of a range of short-range wireless communication by the antenna using a first camera; acquiring an image of a second range not including the range of the short-range wireless communication using a second camera; identifying people included in the first range of images; identifying people included in the second range of images; Includes. The step of identifying a person included in the first range of images includes: generating recognition information for a person included in the first range of images; determining whether a user associated with the user terminal is included in the first range of images based on recognition information of a person included in the first range of images, identification information of the antenna, and the user information acquired by the antenna; generating tracking assistance information for a user associated with the user terminal when the user is determined to be included in the first range of images; Includes. The step of identifying a person included in the second range of images includes: generating recognition information of a person included in the image of the second range; determining whether or not a user that is the same as the user determined to be included in the image of the first range is included in the image of the second range based on recognition information of a person included in the image of the second range and the tracking assistance information; Includes.

[0009] The second aspect has the same features as the first aspect, as follows. The tracking assistance information includes user information of a user determined to be included in the image of the first range, information on the position of a first camera that acquired the image of the first range, and information on the time when the image of the first range was acquired by the first camera. The step of identifying a person included in the second range of images further includes: The method includes a step of narrowing down the tracking support information based on information on the time when the image of the first range was acquired, information on the position of the first camera that acquired the image of the first range, information on the time when the image of the second range was acquired, and information on the position of the second camera, which are included in the tracking support information. The step of narrowing down the tracking support information includes a step of extracting tracking support information generated based on an image of the first range acquired by a first camera located within a predetermined distance from the position of the second camera during a period going back a predetermined time from the time when the image of the second range was acquired.

[0010] The third aspect has the following further features in the first or second aspect. The second camera includes at least two cameras. The step of identifying a person included in the second range of images further includes: a step of generating tracking confirmation information for a user when the same user as the user determined to be included in the image of the first range is determined to be included in the image of the second range, the tracking confirmation information including user information of the user determined to be included in the image of the second range, information on the position of a second camera that captured the image of the second range, and information on the time when the image of the second range was captured by the second camera; narrowing down the tracking confirmation information based on information on the time when the image of the second range was acquired, which is included in the tracking confirmation information, information on the position of the second camera, which is included in the tracking confirmation information, information on the time when the image of the second range that is the subject of person identification was acquired, and information on the position of the second camera that acquired the image of the second range that is the subject of person identification; Includes. The step of narrowing down the tracking assistance information includes a step of extracting tracking confirmation information generated based on an image of the second range acquired by another second camera located within a predetermined distance from the position of the second camera that acquired the image of the second range that is the subject of person identification, during a period going back a predetermined time from the time when the image of the second range that is the subject of person identification was acquired.

[0011] A fourth aspect is any one of the first to third aspects, further comprising the following feature. The first camera includes at least two cameras. The method further comprises: When it is determined that the same user is included in at least two of the images of the first range acquired by the at least two first cameras, the method includes a step of calculating the movement direction of the user based on information on each position of the first cameras that acquired the at least two images and information on each time when the at least two images were acquired. The step of identifying a person included in the first range of images further includes: When it is determined that a user associated with the user terminal is included in the first range of images, determining whether information on a moving direction of the user has been calculated; adding, to the tracking assistance information, identification information of a second camera located ahead in the moving direction as viewed from a first camera that has acquired an image of the first range, based on the information of the moving direction when it is determined that the information of the moving direction has been calculated; Includes. The step of identifying a person included in the second range of images further includes: determining whether or not identification information of a second camera located ahead in the moving direction is included in the tracking assistance information; When it is determined that identification information of a second camera located ahead in the moving direction is included in the tracking support information, narrowing down the persons included in the image of the second range based on the identification information of the second camera; Includes.

[0012] The fifth aspect is a system for identifying people in camera images, which has the following features: The system includes an antenna, first and second cameras, and a management server. The antenna acquires user information assigned to the user terminal by short-range wireless communication, and transmits the user information to the outside by long-range communication. The first camera captures an image of a first range including at least a part of a range of short-range wireless communication by the antenna. The second camera captures an image of a second range that does not include the range of the short-range wireless communication. The management server performs a first identification process to identify a person included in an image of the first range, and a second identification process to identify a person included in an image of the second range. The management server, in the first identification process, generating recognition information for a person included in the first range of images; determining whether a user associated with the user terminal is included in the first range of images based on recognition information of a person included in the first range of images, identification information of the antenna, and the user information acquired by the antenna; When it is determined that a user associated with the user terminal is included in the first range of images, tracking assistance information for the user is generated. The management server, in the second identification process, generating recognition information for a person included in the image of the second range; Based on the recognition information of the person included in the image of the second range and the tracking assistance information, it is determined whether the same user as the user determined to be included in the image of the first range is included in the image of the second range.

[0013] The sixth aspect has the following further features in addition to the fifth aspect. The tracking assistance information includes user information of a user determined to be included in the image of the first range, information on the position of a first camera that acquired the image of the first range, and information on the time when the image of the first range was acquired by the first camera. The management server, in the second identification process, The tracking support information is further narrowed down based on information on the time when the image of the first range was acquired, which is included in the tracking support information, information on the position of the first camera that acquired the image of the first range, which is included in the tracking support information, information on the time when the image of the second range was acquired, and information on the position of the second camera. The management server, in the narrowing down process of the tracking support information, Tracking support information is extracted that is generated based on an image of the first range acquired by a first camera located within a predetermined distance from the position of the second camera during a period going back a predetermined time from the time when the image of the second range was acquired.

[0014] The seventh aspect of the present invention is the fifth or sixth aspect further characterized by the following: The second camera includes at least two cameras. The management server further includes, in the second identification process, When it is determined that the same user as the user determined to be included in the image of the first range is included in the image of the second range, tracking confirmation information of the user is generated. The tracking confirmation information includes user information of the user determined to be included in the image of the second range, information on the position of the second camera that captured the image of the second range, and information on the time when the image of the second range was captured by the second camera. The management server further includes, in the second identification process, The tracking confirmation information is further narrowed down based on information on the time when the image of the second range was acquired, which is included in the tracking confirmation information, information on the position of the second camera, which is included in the tracking confirmation information, information on the time when the image of the second range that is the subject of the second identification process was acquired, and information on the position of the second camera that acquired the image of the second range that is the subject of the second identification process. The management server, in the process of narrowing down the tracking confirmation information, Tracking confirmation information is extracted that is generated based on an image of the second range acquired by another second camera located within a predetermined distance from the position of the second camera that acquired the image of the second range that is the subject of the second identification process, during a period going back a predetermined time from the time when the image of the second range that is the subject of the second identification process was acquired.

[0015] An eighth aspect is any one of the fifth to seventh aspects, further comprising the following features. The first camera includes at least two cameras. The management server further comprises: If it is determined that the same user is included in at least two of the images of the first range acquired by the at least two first cameras, the movement direction of the user is calculated based on information on each position of the first cameras that acquired the at least two images and information on each time when the at least two images were acquired. The management server further includes, in the first identification process, When it is determined that a user associated with the user terminal is included in the image of the first range, it is determined whether information on a moving direction of the user is calculated; If it is determined that the information on the movement direction has been calculated, identification information of a second camera located ahead in the movement direction from the first camera that captured the image of the first range is added to the tracking support information based on the information on the movement direction. The management server further includes, in the second identification process, determining whether or not identification information of a second camera located ahead in the moving direction is included in the tracking assistance information; If it is determined that the identification information of a second camera located ahead in the direction of movement is included in the tracking support information, the people included in the images in the second range are narrowed down based on the identification information of the second camera.

[0016] A ninth aspect is a program for causing a computer to identify a person included in a camera image, the program having the following features. The program is A process of acquiring user information assigned to a user terminal by short-range wireless communication using an antenna, and acquiring the user information by long-range communication using the antenna; acquiring an image of a first range including at least a part of a range of short-range wireless communication by the antenna using a first camera; A process of acquiring an image of a second range not including the range of the short-range wireless communication using a second camera; A process of identifying a person included in the first range of images; A process of identifying a person included in the image of the second range; to be executed by the computer. The process of identifying a person included in the image of the first range includes: A process of generating recognition information of a person included in the image of the first range; A process of determining whether or not a user associated with the user terminal is included in the first range of images based on recognition information of a person included in the first range of images, identification information of the antenna, and the user information acquired by the antenna; generating tracking assistance information for a user associated with the user terminal when the user is determined to be included in the first range of images; Includes. The process of identifying a person included in the image of the second range includes: A process of generating recognition information of a person included in the image of the second range; A process of determining whether or not a user that is the same as a user determined to be included in the image of the first range is included in the image of the second range based on recognition information of a person included in the image of the second range and the tracking assistance information; Includes. Effect of the Invention

[0017] According to the first, fifth or eighth aspect, in the process of identifying a person included in an image of a first range (first identification process), when it is determined that a user associated with a user terminal is included in an image of a first range, tracking support information of the user is generated. In the process of identifying a person included in an image of a second range (second identification process), this tracking support information is used to determine whether or not a user identical to the user determined to be included in an image of the first range is included in an image of a second range. Therefore, even if the total number of installed antennas is less than the total number of installed cameras, it is possible to identify a user included in the second range. This contributes to efficient tracking of users in an environment with limited infrastructure resources.

[0018] According to the second or sixth aspect, the tracking support information is narrowed down. Therefore, it is possible to reduce the processing load of the computer that executes the second identification process. This contributes to more efficient tracking of the user.

[0019] According to the third or seventh aspect, in the second identification process, when it is determined that the same user as the user determined to be included in the first range of images is included in the second range of images, tracking confirmation information of the user is generated. This tracking confirmation information is used in a second identification process separate from the second identification process. Therefore, it is possible to continue tracking the user by repeating the second identification process and the separate second identification process.

[0020] According to the fourth or eighth aspect, based on the moving direction of the user, the identification information of the second camera located ahead in the moving direction is added to the tracking support information. Therefore, it is possible to reduce the processing load of the computer that executes the second identification process. This contributes to more efficient tracking of the user. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram illustrating an overview of person identification performed in a system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a diagram for explaining a situation assumed in an embodiment. [Diagram 3] 1 is a block diagram showing an example of the overall configuration of a system according to an embodiment; [Figure 4] 4 is a block diagram showing an example of a functional configuration of the management server shown in FIG. 3. [Diagram 5] 11A to 11C are diagrams illustrating an example of narrowing down tracking support information used in the second identification process. [Figure 6] 11A and 11B are diagrams illustrating an example of a method for identifying a camera having identification information to be added to tracking support information. [Figure 7] 13 is a flowchart showing an example of the flow of a first identification process by a management server (processor). [Figure 8] 13 is a flowchart showing an example of the flow of a second identification process by a management server (processor). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] A person identification method, a person identification system, and a person identification program according to an embodiment of the present disclosure will be described below with reference to the drawings. The person identification method according to the embodiment is realized by computer processing performed in the person identification system according to the embodiment. The person identification program according to the embodiment is a program for causing a computer to execute the person identification method according to the embodiment. In each drawing, the same or corresponding parts are given the same reference numerals, and their description will be simplified or omitted.

[0023] 1. Overview of the embodiment 1-1. Person Identification FIG. 1 is a diagram for explaining an overview of person identification performed in a system (hereinafter also simply referred to as the "system") according to an embodiment of the present disclosure. The system according to the embodiment includes at least two antennas. In FIG. 1, antennas 11, 12, and 1X are depicted as examples of the at least two antennas. Hereinafter, unless there is a need to distinguish between them, antennas 11, 12, and 1X will be collectively referred to as "antenna 1." Antenna 1 communicates with a terminal (hereinafter also referred to as a "user terminal") carried by a user U of a service (e.g., a person monitoring service, a person tracking service) provided by the system according to the embodiment.

[0024] FIG. 1 illustrates user terminals 21 and 22 as examples of user terminals. The user terminal 21 is a terminal carried by a user U1, and the user terminal 22 is a terminal carried by a user U2. Hereinafter, when there is no particular distinction between these, the user terminals 21 and 22 are collectively referred to as "user terminal 2." Communication with the user terminal 2 via the antenna 1 is performed, for example, by short-range wireless communication. Here, a communication method with a communication distance of several meters to several tens of meters is used for the short-range wireless communication. Specific examples of communication methods include Bluetooth (registered trademark) and Zigbee (registered trademark).

[0025] In short-distance wireless communication with the user terminal 2, the antenna 1 acquires information about the user U (hereinafter also referred to as "user information"). An example of the user information is identification information given to the user U. The identification information is set for each user terminal 2 by the system according to the embodiment, for example, when the service is used for the first time. The setting of the identification information of the user terminal 2 may be reset periodically (for example, once a day). The user information also includes information on the distance from the antenna 1 to the user terminal 2 and the orientation of the user terminal 2 relative to the antenna 1 (i.e., information on the position of the user terminal 2).

[0026] The user information may include feature information such as a facial image of the user U. The feature information is set, for example, by the user U. The feature information may be set based on the result of person recognition by a management server (described later) performed while the user U is using a service. In this case, for example, the feature information set based on the result of person recognition is added to the user information held by the user terminal 2.

[0027] The antenna 1 also performs long-distance communication with a base station (not shown) on the network. Here, a communication method with a communication distance of several kilometers to several tens of kilometers is used for the long-distance communication. Specific examples of the communication method include various types of LPWA (Low Power Wide Area) and LTE (Long Term Evolution). The connection destination of the antenna 1 via the base station includes a management server (described later). A relay module such as Wi-Fi (registered trademark) may be present between the antenna 1 and the base station. This relay module may also be present between the base station and the management server.

[0028] In long-distance communication with a base station, the antenna 1 transmits identification information of the antenna 1. Examples of the identification information of the antenna 1 include information on the location where the antenna 1 is installed, and information on the range of short-distance wireless communication by the antenna 1 (CR11, CR12, and CR1X. Hereinafter, when the ranges CR11, CR12, and CR1X are not particularly distinguished from each other, they are collectively referred to as the "communication range CR1"). When user information is acquired from the user terminal 2, the antenna 1 transmits this user information and information on the time when this user information was acquired to the base station.

[0029] The identification information of the antenna 1, the user information acquired by this antenna 1, and the information on the time when this user information was acquired are examples of information included in "antenna information ANT" (described later).

[0030] The system according to the embodiment also includes at least two cameras. In FIG. 1, cameras 31, 32, and 3Y are illustrated as an example of at least two cameras. Camera 31 captures a range SR31 including at least a part of a communication range CR11 by antenna 11. Camera 32 captures a range SR32 including at least a part of a communication range CR12 by antenna 12. Camera 3Y captures a range SR3Y including at least a part of a communication range CR1X by antenna 1X. Hereinafter, when cameras 31, 32, and 3Y are not particularly distinguished from each other, they are collectively referred to as "camera 3." Furthermore, when ranges SR31, SR32, and SR3Y are not particularly distinguished from each other, they are collectively referred to as "capture range SR3."

[0031] The camera 3 is attached to an indoor or outdoor structure to observe its surroundings. The shooting direction of the camera 3 may be fixed at one point, or the shooting direction may move horizontally or vertically. The camera 3 also performs long-distance communication with a base station on the network. This long-distance communication uses the same communication method as that used by the antenna 1. In long-distance communication with the base station, the camera 3 transmits identification information of the camera 3 and image information acquired by the camera 3. An example of the identification information of the camera 3 is information on the location where the camera 3 is installed. The image information acquired by the camera 3 is typically a video, but may be a still image. The image information acquired by the camera 3 also includes information on the time when this image information was acquired.

[0032] The identification information of the camera 3 and the image information acquired by this camera 3 are examples of information included in "camera information CAM1" (described later).

[0033] In the system according to the embodiment, an "identification process" is performed in a management server (described later) to identify a person included in an image in the shooting range SR3. In the identification process, for example, a process of recognizing a person included in an image in the shooting range SR3 is performed. When a person included in an image in the shooting range SR3 is recognized, recognition information of this person is generated. An example of the recognition information is information on the position of the recognized person. The recognition information may include characteristic information such as a facial image of the recognized person. In the identification process, it is also determined whether or not a user U is included in the image in the shooting range SR3 based on the recognition information of the person included in the image in the shooting range SR3, the identification information of the antenna 1, and the user information acquired by this antenna 1.

[0034] 1-2. Characteristics of the identification process Here, consider increasing the number of installed cameras 3 and antennas 1. Increasing the number of installed cameras 3 is desirable from the standpoint of monitoring people included in the images of the shooting range SR3 and crime prevention using image information acquired by the cameras 3. Also, if the number of installed antennas 1 increases, it is expected that the accuracy of the position information of the user terminal 2 will increase, and the accuracy of identifying people included in the images of the shooting range SR3 will improve. On the other hand, increasing the number of installed antennas 1 is a concern in terms of increased costs.

[0035] Therefore, in the embodiment, a situation is considered in which the total number of installed antennas 1 is less than that of cameras. Here, the shooting range SR3 of the camera 3 includes at least a part of the communication range CR1 of the antenna 1. Therefore, in a situation in which the total number of installed antennas 1 is less than that of cameras, there will be cameras that shoot a range that does not include the communication range CR1.

[0036] FIG. 2 is a diagram for explaining a situation assumed in the embodiment. In FIG. 2, in addition to the antenna 1 and camera 3 shown in FIG. 1, cameras 41, 42, and 4Z are depicted. The cameras 41, 42, and 4Z are examples of cameras that capture ranges SR41, 42, and 4Z, respectively, that do not include the communication range CR1. The basic configuration of the cameras 41, 42, and 4Z is the same as that of the camera 3. Hereinafter, when the cameras 41, 42, and 4Z are not particularly distinguished from each other, they are collectively referred to as "camera 4." Furthermore, when the ranges SR41, SR42, and SR4Z are not particularly distinguished from each other, they are collectively referred to as "capture range SR4."

[0037] In long-distance communication with the base station, the camera 4 transmits identification information of the camera 4 and image information acquired by the camera 4. An example of the identification information of the camera 4 is information on the location where the camera 4 is installed. The image information acquired by the camera 4 is typically a video, but may be a still image. The image information acquired by the camera 4 also includes information on the time when this image information was acquired.

[0038] The identification information of the camera 4 and the image information acquired by this camera 4 are examples of information included in "camera information CAM2" (described later).

[0039] In the embodiment, in the identification process, in addition to identifying the person included in the image in the shooting range SR3, the person included in the image in the shooting range SR4 is also identified. Hereinafter, for convenience of explanation, the process for identifying the former is also referred to as the "first identification process" and the latter is also referred to as the "second identification process." The outline of the first identification process is as described above, but the first identification process further generates "tracking support information" for the user U. The generation of the tracking support information is performed when it is determined that the user U is included in the image in the shooting range SR3.

[0040] The tracking support information includes, for example, user information of the user U determined to be included in the image of the shooting range SR3 (specifically, identification information and position information of the user terminal 2), identification information of the camera 3 that captured the image of the shooting range SR3, and information on the time when this image was captured. When the tracking support information is generated, this tracking support information is used in the second identification process.

[0041] In the second identification process, a process of recognizing a person included in the image of the shooting range SR4 is performed. When a person included in the image of the shooting range SR4 is recognized, recognition information of this person is generated. An example of the recognition information is information on the position of the recognized person. The recognition information may include feature information such as a facial image of the recognized person. In the second identification process, it is also determined whether or not the user U is included in the image of the shooting range SR4 based on the recognition information of the person included in the image of the shooting range SR4 and the tracking support information.

[0042] In this way, according to the system of the embodiment, the tracking assistance information generated in the first identification process is used to identify the person included in the image of the shooting range SR4 in the second identification process. Therefore, even if the total number of installed antennas 1 is less than the total number of installed cameras, it is possible to identify the user U included in the shooting range SR4. This contributes to efficient tracking of the user U in an environment with limited infrastructure resources.

[0043] In the system according to the embodiment, "tracking confirmation information" of user U may be generated in the second identification process. "Tracking assistance information" is generated when it is determined that user U is included in the image in shooting range SR3. In contrast, "tracking confirmation information" is generated when it is determined that user U is included in the image in shooting range SR4. The tracking confirmation information includes, for example, user information of user U who is determined to be included in the image in shooting range SR4, identification information of camera 4 that captured the image in shooting range SR4, and information on the time when this image was captured. When tracking confirmation information is generated, this tracking confirmation information is used in "another second identification process".

[0044] A different second identification process means a second identification process targeting an image in a shooting range SR4 different from the second identification process in which the tracking confirmation information was generated, or a second identification process targeting an image in a shooting range SR that is common to the second identification process in which the tracking confirmation information was generated, and which is performed at a different time from the second identification process in which the tracking confirmation information was generated.

[0045] The system according to the embodiment will be described in more detail below.

[0046] 2. Example of a person identification system configuration 2-1. Overall configuration example Fig. 3 is a block diagram showing an example of the overall configuration of a system according to an embodiment. In the example shown in Fig. 3, the system according to the embodiment includes an antenna 1, cameras 3 and 4, a management server 5, and a display 6.

[0047] The antenna 1 is a communication device installed in an indoor or outdoor structure. The total number of antennas 1 installed is at least two. The antenna 1 performs short-distance wireless communication with the user terminal 2. For this short-distance wireless communication, a communication method with a communication distance of several meters to several tens of meters is used. In the short-distance wireless communication, the antenna 1 acquires user information USR from the user terminal 2. The user information USR includes, for example, identification information of the user terminal 2, the distance from the antenna 1 to the user terminal 2, and information on the orientation of the user terminal 2 relative to the antenna 1. The user information USR may include characteristic information of the user U, such as a facial image.

[0048] The antenna 1 also performs long-distance communication with the management server 5 via the network 7. For long-distance communication, a communication method with a communication distance of several kilometers to several tens of kilometers is used. In this long-distance communication, the antenna 1 transmits antenna information ANT to the management server 5. This antenna information ANT includes, for example, identification information of the antenna 1. When the antenna 1 acquires user information USR, this user information USR is added to the antenna information ANT.

[0049] Cameras 3 and 4 are observation devices attached to indoor or outdoor structures. The total number of cameras 3 installed is at least two, and the total number of cameras 4 installed is also at least two. The total number of cameras 3 and 4 installed combined is greater than the total number of antennas 1 installed. Camera 3 captures an area that includes at least a portion of the communication range CR1 of antenna 1 (i.e., shooting range SR3). On the other hand, camera 4 captures an area that does not include communication range CR1 (i.e., shooting range SR4).

[0050] Cameras 3 and 4 each perform long-distance communication with management server 5 via network 7. In this long-distance communication, camera 3 transmits camera information CAM1 to management server 5. The camera information CAM1 includes, for example, identification information of camera 3 and image information acquired by this camera 3. Meanwhile, camera 4 transmits camera information CAM2 to management server 5. The camera information CAM2 includes, for example, identification information of camera 4 and image information acquired by this camera 4.

[0051] Here, the correspondence between the embodiments and the viewpoints of the present disclosure will be described. Camera 3 corresponds to the "first camera" in the viewpoint of the present disclosure, and camera 4 corresponds to the "second camera" in the viewpoint of the present disclosure. Moreover, shooting range SR3 corresponds to the "first range" in the viewpoint of the present disclosure, and shooting range SR4 corresponds to the "second range" in the viewpoint of the present disclosure.

[0052] The management server 5 is a computer that manages the services provided by the system according to the embodiment. The management server 5 performs information processing based on various information (antenna information ANT, camera information CAM1 and CAM2) received from the antenna 1 and the cameras 3 and 4. As a configuration for performing information processing, the management server 5 includes at least one processor 51 and at least one memory 52. ​​The processor 51 includes a CPU (Central Processing Unit). The memory 52 is a volatile memory such as a DDR memory, and expands various programs used by the processor 51 and temporarily stores various information. The various information received by the management server 5 from the antenna 1 and the cameras 3 and 4 is stored in the memory 52.

[0053] The processor 51 performs various processes related to the first and second identification processes by executing a program for person identification stored in the memory 52. ​​The processor 51 also performs display control for outputting the results of the first and second identification processes to the display 6.

[0054] The display 6 is a display device on which image data DIMG generated by display control is output. The display 6 is installed in the premises of a business providing the service according to the embodiment. The display 6 is a display device on a terminal of a guardian of the user U who watches over the user U. In the former case, the display 6 is connected to the management server 5 by wire or wirelessly. In the latter case, the display 6 is connected to the management server 5 via a network 7. The total number of displays 6 is not particularly limited, but is, for example, at least two in the former case and at least one in the latter case.

[0055] 2-2. Example of management server function configuration FIG. 4 is a block diagram showing an example of the functional configuration of the management server 5 shown in FIG. 3. In the example shown in FIG. 4, the management server 5 includes an information acquisition unit 53, a first identification unit 54, a first display control unit 55, a second identification unit 56, and a second display control unit 57. The functions of the blocks shown in FIG. 3 are realized by the processor 51 executing a predetermined program stored in the memory 52. ​​Although FIG. 4 shows one each of the first identification unit 54 and the first display control unit 55, these functions may be set according to the total number of cameras 3 installed. The same can be said for the functions of the second identification unit 56 and the second display control unit 57.

[0056] 2-2-1. Information acquisition section The information acquisition unit 53 acquires various pieces of information from outside and stores them in the memory 52. ​​The various pieces of information include antenna information ANT and camera information CAM1 and CAM2. The information acquisition unit 53 also reads out the various pieces of information stored in the memory 52. ​​The information acquisition unit 53 further transmits the antenna information ANT to the first identification unit 54 and the second identification unit 56, transmits the camera information CAM1 to the first identification unit 54 and the first display control unit 55, and transmits the camera information CAM2 to the second identification unit 56 and the second display control unit 57.

[0057] 2-2-2. First Identification Section The first identification unit 54 performs a first identification process. In the first identification process, a process of recognizing a person included in an image included in the camera information CAM1 (i.e., an image of the shooting range SR3) is performed. When a person included in the image of the shooting range SR3 is recognized, recognition information of this person is generated. An example of the recognition information is information on the position of the recognized person. The recognition information may include characteristic information of the recognized person, such as a facial image.

[0058] In the first identification process, antenna information ANT of the antenna 1 corresponding to the camera 3 is also acquired. The antenna 1 corresponding to the camera 3 is an antenna 1 whose communication range DR1 is partly included in the shooting range SR3. The antenna 1 corresponding to the camera 3 is identified, for example, based on information on the location where the camera 3 is installed, information on the location where the antenna 1 is installed, and information on the communication range CR1 of this antenna 1. The information on the location where the camera 3 is installed is acquired from the identification information of the camera 3. The information on the location where the antenna 1 is installed and information on the communication range CR1 of this antenna 1 are acquired from the identification information of the antenna 1.

[0059] When the antenna information ANT of the antenna 1 corresponding to the camera 3 is acquired, the user information included in this antenna information ANT is collated with the recognition information of the person included in the image in the shooting range SR3. In this collation, for example, the user information USR acquired by the antenna 1 (specifically, the information on the position of the user terminal 2) is compared with the information on the position of the recognized person. At the same time, the information on the time when the antenna 1 acquired the user information is compared with the information on the time when the image included in the camera information CAM1 was acquired. Then, if the user information and the recognition information match in terms of position and time, the recognized person is identified as the user U. Note that, in consideration of the design error of the antenna 1 and the camera 3, a margin may be set for the position and time.

[0060] When the user U is identified in the image of the shooting range SR3, in the first identification process, identification information IDN is assigned to the recognized person. The identification information IDN is set by the system according to the embodiment, and is composed of information different from the identification information of the user terminal 2. Note that the assignment of the identification information IDN is not essential in the present disclosure, and other information that can associate the recognized person with the identification information of the user terminal 2 may be assigned to the recognized person. The identification information IDN is transmitted to the first display control unit 55.

[0061] When the user U is identified in the image of the shooting range SR3, the first identification process also generates tracking support information TRA. The tracking support information TRA includes, for example, user information USR of the user U identified in the image of the shooting range SR3 (specifically, identification information and position information of the user terminal 2), identification information of the camera 3 that captured the image of the shooting range SR3, and information on the time when this image was captured. The tracking support information TRA is transmitted to the second identification unit 56.

[0062] 2-2-3. First display control section The first display control unit 55 performs the first display control. In the first display control, image data DIMG is generated based on the image in the shooting range SR3 processed in the first identification process by the first identification unit 54, and is output to an external device (for example, the display 6). If identification information IDN is given to the user U identified in the first identification process, in the first display control, the identification information IDN is superimposed near the user U included in the image in the shooting range SR3.

[0063] 2-2-4. Second Identification Section The second identification unit 56 performs a second identification process. In the second identification process, a process of recognizing a person included in an image included in the camera information CAM2 (i.e., an image of the shooting range SR4) is performed. When a person included in the image of the shooting range SR4 is recognized, recognition information of this person is generated. An example of the recognition information is information on the position of the recognized person. The recognition information may include characteristic information of the recognized person, such as a facial image.

[0064] In the second identification process, tracking support information TRA of the camera 3 corresponding to the camera 4 is also acquired. The camera 3 corresponding to the camera 4 is exemplified by the cameras 3 installed around the camera 4. The camera 3 corresponding to the camera 4 is identified based on, for example, information on the location where the camera 4 is installed and that of the camera 3. Note that information on the locations where the cameras 3 and 4 are installed is acquired from the identification information of the cameras 3 and 4.

[0065] When the tracking support information TRA of camera 3 corresponding to camera 4 is acquired, the position information of camera 3 included in this tracking support information TRA is compared with the position information of camera 4. At the same time, information on the time when camera 3 acquired an image of shooting range SR3 is compared with information on the time when camera 4 acquired an image of shooting range SR4. Note that, taking into consideration the design error of cameras 3 and 4, a margin may be set for the position and time.

[0066] For example, if there is a camera 3 that is within a predetermined distance from the camera 4 and has captured an image of the shooting range SR3 where the time difference between the time the image of the shooting range SR4 was captured and the time the image of the shooting range SR3 was captured is within a predetermined time, the person recognized in the image of the shooting range SR4 is identified as the user U included in the tracking support information TRA generated based on the image of the shooting range SR3. In order to improve the accuracy of this identification, a comparison may be further made between the feature information of the person recognized in the image of the shooting range SR4 and the feature information of the user U included in the tracking support information TRA.

[0067] When the user U is identified in the image in the shooting range SR4, in the second identification process, identification information IDN is assigned to the recognized person. The identification information IDN may be the identification information IDN received from the first identification unit 54 as is, or may be identification information IDN that is uniquely set in the image in the shooting range SR4. For efficient tracking of the user U, it is desirable to reuse the identification information IDN received from the first identification unit 54. The identification information IDN is transmitted to the second display control unit 57.

[0068] If the user U is identified in the image of the shooting range SR4, the second identification process may also generate tracking confirmation information TRC. The tracking confirmation information TRC includes, for example, user information USR of the user U identified in the image of the shooting range SR4 (specifically, identification information and position information of the user terminal 2), identification information of the camera 4 that captured the image of the shooting range SR4, and information on the time when the image was captured.

[0069] When the tracking confirmation information TRC is generated, in the second identification process, the position information of the camera 4 (hereinafter also referred to as the "comparison target camera") included in the tracking confirmation information TRC is compared with the position information of the camera 4 (hereinafter also referred to as the "processing target camera") that captured the image of the shooting range SR4 that is the target of the second identification process. At the same time, the information of the time when the comparison target camera captured the image of the shooting range SR4 is compared with the information of the time when the processing target camera captured the image of the shooting range SR4.

[0070] Then, for example, if there is a comparison target camera that is within a predetermined distance from the processing target camera and the time difference between the times when the images of the shooting range SR4 were acquired is within a predetermined time, the person recognized in the image of the shooting range SR4 acquired by the processing target camera is identified as the user U included in the tracking confirmation information TRC generated based on that of the comparison target camera. In order to improve the accuracy of this identification, a comparison may be further made between the characteristic information of the person recognized in the image of the shooting range SR4 and the characteristic information of the user U included in the tracking confirmation information TRC.

[0071] 2-2-5. Second display control section The second display control unit 57 performs the second display control. In the second display control, image data DIMG is generated based on the image in the shooting range SR4 processed in the second identification process by the second identification unit 56, and is output to an external device (for example, the display 6). If identification information IDN is given to the user U identified in the second identification process, in the second display control, the identification information IDN is superimposed near the user U included in the image in the shooting range SR4.

[0072] 2-2-6. Another example of the second identification part Here, the second identification unit 56 may narrow down the tracing support information TRA or tracing confirmation information TRC used in the second identification process. Hereinafter, an example of narrowing down will be described using the tracing support information TRA as a representative example.

[0073] FIG. 5 is a diagram for explaining an example of narrowing down of tracking support information TRA used in the second identification process. In this narrowing down example, it is assumed that the image of the shooting range SR41 is acquired at time t1, focusing on the camera 41. In this narrowing down example, a past time (t4-Tp) that is a predetermined time Tp before the time t1 is set. Then, a circle CA1 of a predetermined time Tp×V is set with the position where the camera 41 is installed as the center (V is the moving speed of a general pedestrian). Then, the camera 3 (cameras 31 to 35 in the example shown in FIG. 5) located inside this circle CA1 is identified.

[0074] In the example shown in Fig. 5, tracking support information TRA generated based on an image of the shooting range SR3 of the camera 3 located inside the circle CA1 is extracted. In the example shown in Fig. 5, the tracking support information TRA generated based on the image of the shooting range SR3 acquired from time (t1-Tp) to time t1 is also extracted.

[0075] 2-2-7. Another example of the first and second identification parts As already explained, the total number of cameras 3 installed is two. Therefore, a case is assumed in which the same user U is identified in each image of the shooting range SR3 captured by the two cameras 3. In such a case, the movement direction (direction of the line of movement) of the user can be predicted based on information on the positions of the two cameras 3 and information on the times when the images of the two types of shooting range SR3 were captured by these cameras 3. This prediction of the movement direction is calculated separately by the processor 51.

[0076] In this example, consider the case where the moving direction is calculated. When the user U is identified in the image of the shooting range SR3, the first identification unit 54 acquires information on the moving direction of the user U. Based on the information on the moving direction, the first identification unit 54 also identifies the camera 4 located at the end of the moving direction when viewed from the camera 3 that acquired the image of the shooting range SR3. Then, the first identification unit 54 adds the identification information of this camera 4 (for example, information on the position where the camera 4 is installed) to the tracking support information TRA.

[0077] Fig. 6 is a diagram for explaining an example of a method for identifying a camera 4 having identification information to be added to the tracking support information TRA. In the example shown in Fig. 6, attention is focused on the cameras 34 and 38. In the example shown in Fig. 6, it is also assumed that an image of the shooting range SR38 is acquired at time t2, and an image of the shooting range SR34 is acquired at time t3, which is later than time t2.

[0078] When the user U is identified as a result of the first identification process based on the image of the shooting range SR34 acquired at time t3, the first identification unit 54 judges whether or not the moving direction Du of the user U has been calculated. Then, when it is judged that the moving direction Du has been calculated, the first identification unit 54 identifies the camera 4 (the cameras 41 and 45 in the example shown in FIG. 6) existing ahead of the moving direction Du. When identifying the camera 4, narrowing down may be performed based on the moving speed V of a general pedestrian, or narrowing down may be performed based on the moving speed Vu of the user U. The moving speed Vu can be calculated based on, for example, the position information of the cameras 34 and 38 and the time difference between times t2 and t3.

[0079] As already described, the tracking support information TRA includes the user information USR of the user U identified in the image of the shooting range SR3. Therefore, by adding the identification information of the camera 4 existing ahead in the moving direction Du to the tracking support information TRA, the user U becomes easier to identify in the second identification process based on the image of the shooting range SR4 acquired by the camera 4.

[0080] When the second identification unit 56 acquires the tracking support information TRA of the camera 3 corresponding to the camera 4, it judges whether or not the tracking support information TRA includes the identification information of the processing target camera. Then, when it is judged that the tracking support information TRA includes the identification information of the processing target camera, it narrows down the persons recognized in the image of the shooting range SR4 based on the identification information.

[0081] 2-3. Example of processing by the management server Fig. 7 is a flowchart showing an example of the flow of the first identification process by the management server 5 (processor 51). The processing routine shown in Fig. 7 is repeatedly executed for each camera 3 at a predetermined cycle, for example.

[0082] 7, information is first acquired (step S11). In the processing of step S11, for example, antenna information ANT and camera information CAM1 are acquired. As already explained, the antenna information ANT includes, for example, identification information of antenna 1 (location information and information on communication range CR1), user information USR acquired by this antenna 1 (identification information of user terminal 2), and information on the time when this user information USR was acquired. The camera information CAM1 includes, for example, identification information (location information) of camera 3, and image information acquired by this camera 3 (an image in shooting range SR3 and the time when this image information was acquired).

[0083] Following the processing of step S11, a person included in the image of the shooting range SR3 is recognized (step S12). In the processing of step S12, the position of the person included in the image of the shooting range SR3 is recognized based on known image processing. When the position of the person included in the image of the shooting range SR3 is recognized, recognition information of this person is generated. In the processing of step S12, in addition to recognizing the position of the person included in the image of the shooting range SR3, the characteristics of this person may also be recognized. In this case, the characteristic information is added to the recognition information.

[0084] Following the process of step S12, it is determined whether or not a person included in the image in the shooting range SR3 has been recognized (step S13). The process of step S13 is determined, for example, based on whether or not person recognition information has been generated in step S12. If the determination result of step S13 is negative, the first identification process is terminated. If not, the process proceeds to step S14.

[0085] In the process of step S14, it is determined whether or not the antenna information ANT of the antenna 1 corresponding to the camera 3 includes the user information USR. As already described, the antenna 1 corresponding to the camera 3 is the antenna 1 whose communication range DR1 is partly included in the shooting range SR3. If the determination result of step S14 is negative, the first identification process is ended. If not, the process proceeds to step S15.

[0086] In the process of step S15, the recognition information generated in the process of step S12 is compared with the user information USR. Specifically, the position of the person included in the recognition information is compared with the position of the user U included in the user information USR. At the same time, the time when the image of the shooting range SR3 targeted in the process of step S12 was acquired is compared with the time when the user information USR was acquired. Then, if the recognition information and the user information USR match in terms of position and time, the person included in the recognition information is identified as the user U. Note that, in consideration of design errors of the antenna 1 and the camera 3, a margin may be set for the position and time.

[0087] In the process of step S15, if there is no user information USR that matches the recognition information, the person included in the recognition information is not identified, and the first identification process is terminated. In this case, information that the person included in the recognition information is not the user U may be given to the person.

[0088] When a person is identified in the process of step S15, identification information IDN is assigned to the person (step S16). As already explained, the identification information IDN is arbitrary information set by the system according to the embodiment, and is composed of information different from the identification information of the user terminal 2. Furthermore, in the process of step S16, the user information USR and the identification information IDN are associated with each other and recorded in a predetermined memory. An example of the predetermined memory is a memory higher than the memory 52, such as a cache memory.

[0089] Following the process of step S16, tracking support information TRA is generated (step S17). As already described, the tracking support information TRA includes, for example, user information USR of the user U identified in the image of the shooting range SR3 (specifically, identification information and position information of the user terminal 2), identification information of the camera 3 that captured the image of the shooting range SR3, and information on the time when this image was captured. In addition, in the process of step S17, the generated tracking support information TRA is recorded in a predetermined memory.

[0090] In the process of step S17, the camera 4 existing ahead in the moving direction Du may be specified based on the information of the moving direction Du described in Fig. 6. Furthermore, when the camera 4 is specified, the recognition information of this camera 4 may be added to the tracking support information TRA.

[0091] Fig. 8 is a flowchart showing an example of the flow of the second identification process by the management server 5 (processor 51). The processing routine shown in Fig. 8 is repeatedly executed at a predetermined cycle for each camera 4, for example.

[0092] In the routine shown in Fig. 8, steps S21 to S23 are first performed. The content of step S21 is the same as that of step S11 in Fig. 7, the content of step S22 is the same as that of step S12 in Fig. 7, and the content of step S23 is the same as that of step S13 in Fig. 7. If the determination result of step S23 is negative, the second identification process is terminated. If not, the process proceeds to step S24.

[0093] In the process of step S24, it is determined whether or not tracking support information TRA of a camera 3 corresponding to the camera 4 exists. As already described, the camera 3 corresponding to the camera 4 is, for example, a camera 3 installed in the vicinity of the camera 4. If the determination result of step S24 is negative, the second identification process is ended. If not, the process proceeds to step S25.

[0094] In the process of step S24, before determining whether or not tracking support information TRA exists, the tracking support information TRA described in Fig. 5 may be narrowed down. In this case, the narrowing down is performed based on, for example, information on the position where the camera 4 is installed, information on the time when the image of the shooting range SR4 that is the target of the process of step S22 was acquired, information on the position where the camera 3 corresponding to the camera 4 is installed, and the tracking support information TRA that is generated based on the image of the shooting range SR3 acquired by the camera 3.

[0095] In the process of step S25, the tracking support information TRA is compared with the identification information of the camera 4 and the information on the time when the image of the shooting range SR4 targeted in the process of step S22 was acquired. Specifically, the position of the camera 3 that acquired the image of the shooting range SR3 included in the tracking support information TRA is compared with the position of the camera 4. At the same time, the time when the image of the shooting range SR3 was acquired is compared with the time when the image of the shooting range SR4 was acquired.

[0096] Then, for example, if there is a camera 3 that is within a predetermined distance from camera 4 and has captured an image of shooting range SR3 where the time difference between the time the image of shooting range SR4 is captured and the time the image of shooting range SR3 is captured is within a predetermined time, the person recognized in the processing of step S22 is identified as user U included in the tracking support information TRA.

[0097] In the processing of steps S24 and S25, the person included in the image of the shooting range SR4 may be identified based on the tracking confirmation information TRC instead of the tracking support information TRA. In this case, in the explanation of steps S24 and S25 described above, "camera 3" should be read as "comparison target camera" and "camera 4" should be read as "processing target camera".

[0098] In the process of step S25, the identification information of the camera 4 included in the tracking support information TRA described with reference to FIG. 6 may be used to narrow down the persons recognized in the image of the shooting range SR4 targeted in step S22.

[0099] If the person is identified in the process of step S25, identification information IDN is assigned to the person (step S26). The content of the process of step S26 is the same as the process of step S16 in FIG.

[0100] Following the process of step S26, the tracking confirmation information TRC is generated (step S27). As already described, the tracking support information TRA includes, for example, user information of the user U determined to be included in the image of the shooting range SR4, identification information of the camera 4 that captured the image of the shooting range SR4, and information on the time when the image was captured. In addition, in the process of step S27, the generated tracking confirmation information TRC is recorded in a predetermined memory.

[0101] 3.Effects According to the embodiment described above, when the user U is identified in the process (first identification process) for identifying the person included in the image of the shooting range SR3, tracking support information TRA is generated. Then, this tracking support information TRA is used in the process (second identification process) for identifying the person included in the image of the shooting range SR4. Therefore, even if the total number of installed antennas 1 is less than the total number of installed cameras, it is possible to identify the user U included in the shooting range SR4. This contributes to efficient tracking of the user U in an environment with limited infrastructure resources.

[0102] Also, according to the embodiment, when the user U is identified in a process (second identification process) for identifying a person included in an image of the shooting range SR4, the tracking confirmation information TRC may be generated, and this tracking confirmation information TRC may be used in another second identification process. In this case, once the user U is identified in the first identification process, it is possible to continue tracking the user U by repeating the second identification process and another second identification process. This contributes to improving the convenience of the service.

[0103] According to the embodiment, the tracking support information TRA may be narrowed down as described in FIG. 5 or the identification information of the camera 4 may be added to the tracking support information TRA as described in FIG. 6. In this case, it is possible to reduce the processing load of the computer that executes the second identification process. This contributes to more efficient tracking of the user U. [Explanation of symbols]

[0104] 1,11,12,1X Antenna 2,21,22 User terminal 3,31,32,33,34,35,36,37,38,39,3Y Camera 4,41,42,43,44,45,4Z Camera 5 Management Server 51 Processors 52 Memory U,U1,U2 users Du moving direction CR1, CR11, CR12, CR1X Communication range SR3, SR31, SR32, SR3Y shooting range SR4, SR41, SR42, SR4Z shooting range ANT Antenna Information CAM1,CAM2 camera information IDN Identification Information TRA Tracking Assistance Information TRC Tracking Confirmation Information USR User Information

Claims

1. A person identification method for identifying a person included in a camera image, comprising: acquiring user information assigned to each user terminal by short-range wireless communication using at least two antennas, and transmitting the user information to an outside via long-range communication using the at least two antennas; acquiring, for each of the at least two first cameras, an image of a first range including at least a portion of a range of short-range wireless communication by any of the at least two antennas; acquiring, by means of at least one second camera, an image of a second range not including a range of short-range wireless communication by the at least two antennas; identifying people included in the first range of images for each camera; identifying people included in the second range of images for each camera; Including, a total number of the at least two antennas is less than a total number of the at least two first cameras and the at least one second camera; The step of identifying people included in the first range of images for each camera includes: generating recognition information for a person included in the first range of images; determining whether or not a user associated with the user terminal is included in the first range of images based on recognition information of a person included in the first range of images, identification information of an antenna whose short-range wireless communication range is included in the first range of images, and the user information acquired by the antenna; generating tracking assistance information for a user associated with the user terminal when the user is determined to be included in the first range of images; Including, The step of identifying a person included in the image of the second range for each camera includes: generating recognition information of a person included in the second range of images; determining whether or not a user that is the same as a user determined to be included in the image of the first range is included in the image of the second range based on recognition information of a person included in the image of the second range and the tracking assistance information; Includes A person identification method comprising:

2. The person identification method according to claim 1, the tracking assistance information includes user information of a user determined to be included in the image of the first range, information on a position of a first camera that captured the image of the first range, and information on a time when the image of the first range was captured by the first camera; The step of identifying the person included in the image of the second range for each camera further includes: narrowing down the tracking support information based on information on a time when the image of the first range was acquired, which is included in the tracking support information, information on a position of a first camera that acquired the image of the first range, information on a time when the image of the second range was acquired, and information on a position of a second camera that acquired the image of the second range, which is included in the tracking support information; The step of narrowing down the tracking support information includes a step of extracting tracking support information generated based on an image of the first range acquired by a first camera located within a predetermined distance from a position of a second camera that acquired the image of the second range during a period going back a predetermined time from the time when the image of the second range was acquired. A person identification method comprising:

3. The person identification method according to claim 1 or 2, the at least one second camera includes at least two cameras; The step of identifying the person included in the image of the second range for each camera further includes: a step of generating tracking confirmation information for a user when the same user as the user determined to be included in the image of the first range is determined to be included in the image of the second range, the tracking confirmation information including user information of the user determined to be included in the image of the second range, information on the position of a second camera that captured the image of the second range, and information on the time when the image of the second range was captured by the second camera; narrowing down the tracking confirmation information based on information on the time when the image of the second range was acquired, which is included in the tracking confirmation information, information on the position of the second camera that acquired the image of the second range, which is included in the tracking confirmation information, information on the time when the image of the second range that is the subject of person identification was acquired, and information on the position of the second camera that acquired the image of the second range that is the subject of person identification; Including, The step of narrowing down the tracking assistance information includes a step of extracting tracking confirmation information generated based on an image of the second range that is a target of person identification and that is acquired by another second camera located within a predetermined distance from a position of the second camera that acquired the image of the second range that is a target of person identification during a period going back a predetermined time from the time when the image of the second range that is a target of person identification was acquired. A person identification method comprising:

4. The person identification method according to any one of claims 1 to 3, further comprising: and when it is determined that the same user is included in at least two of the images of the first range acquired by the at least two first cameras, calculating a moving direction of the user based on information on each position of the first cameras that acquired the at least two images and information on each time when the at least two images were acquired; The step of identifying people included in the first range of images further comprises: When it is determined that a user associated with the user terminal is included in the first range of images, determining whether information on a moving direction of the user has been calculated; adding, to the tracking assistance information, identification information of a second camera located ahead in the moving direction as viewed from a first camera that has acquired an image of the first range, based on the information of the moving direction when it is determined that the information of the moving direction has been calculated; Including, The step of identifying the person included in the image of the second range for each camera further includes: determining whether or not identification information of a second camera located ahead in the moving direction is included in the tracking assistance information; When it is determined that identification information of a second camera located ahead in the moving direction is included in the tracking support information, narrowing down the persons included in the image of the second range based on the identification information of the second camera; Includes A person identification method comprising:

5. A person identification system for identifying a person included in a camera image, comprising: At least two antennas each acquiring user information assigned to a user terminal by short-range wireless communication and transmitting the user information to an outside via long-range communication; at least two first cameras each capturing an image of a first range including at least a part of a range of short-range wireless communication by any of the at least two antennas; at least one second camera that captures a second range that does not include a range of short-range wireless communication by the at least two antennas; a management server that performs a first identification process for identifying a person included in an image of the first range for each camera, and a second identification process for identifying a person included in an image of the second range for each camera; Equipped with a total number of the at least two antennas is less than a total number of the at least two first cameras and the at least one second camera; In the first identification process, the management server generating recognition information for a person included in the first range of images; determining whether or not a user associated with the user terminal is included in the image of the first range based on recognition information of a person included in the image of the first range, identification information of an antenna whose short-range wireless communication range is included in the image of the first range, and the user information acquired by the antenna; generating tracking assistance information for a user associated with the user terminal when the user is determined to be included in the first range of images; In the second identification process, the management server generating recognition information for a person included in the image of the second range; Based on the person recognition information of the person included in the image of the second range and the tracking assistance information, it is determined whether or not the same user as the user determined to be included in the image of the first range is included in the image of the second range. A person identification system comprising:

6. The person identification system according to claim 5, the tracking assistance information includes user information of a user determined to be included in the image of the first range, information on a position of a first camera that captured the image of the first range, and information on a time when the image of the first range was captured by the first camera; In the second identification process, the management server Further performing a process of narrowing down the tracking support information based on information on a time when the image of the first range was acquired, information on a position of a first camera that acquired the image of the first range, information on a time when the image of the second range was acquired, and information on a position of a second camera that acquired the image of the second range, which are included in the tracking support information; The management server, in the narrowing down process of the tracking support information, Extracting tracking support information generated based on an image of the first range acquired by a first camera located within a predetermined distance from the position of a second camera that acquired the image of the second range during a period going back a predetermined time from the time when the image of the second range was acquired. A person identification system comprising:

7. A person identification system according to claim 5 or 6, the at least one second camera includes at least two cameras; The management server further includes, in the second identification process, When it is determined that the same user as the user determined to be included in the first range of images is included in the second range of images, generating tracking confirmation information for the user; the tracking confirmation information includes user information of a user determined to be included in the image of the second range, information on a position of a second camera that captured the image of the second range, and information on a time when the image of the second range was captured by the second camera; The management server further includes, in the second identification process, further performing a process of narrowing down the tracking confirmation information based on information on a time when the image of the second range was acquired, which is included in the tracking confirmation information, information on a position of the second camera that acquired the image of the second range, which is included in the tracking confirmation information, information on a time when the image of the second range that is the target of the second identification process was acquired, and information on the position of the second camera that acquired the image of the second range that is the target of the second identification process; The management server, in the process of narrowing down the tracking confirmation information, Extracting tracking confirmation information generated based on an image of the second range acquired by another second camera located within a predetermined distance from the position of the second camera that acquired the image of the second range that is the target of the second identification process during a period going back a predetermined time from the time when the image of the second range that is the target of the second identification process was acquired. A person identification system comprising:

8. The person identification system according to any one of claims 5 to 7, The management server further comprises: when it is determined that the same user is included in at least two of the images of the first range acquired by the at least two first cameras, a movement direction of the user is calculated based on information on each position of the first cameras that acquired the at least two images and information on each time when the at least two images were acquired; The management server further includes, in the first identification process, When it is determined that a user associated with the user terminal is included in the image of the first range, it is determined whether information on a moving direction of the user is calculated; when it is determined that the information on the moving direction has been calculated, adding, to the tracking assistance information, identification information of a second camera located ahead in the moving direction as viewed from a first camera that has acquired an image of the first range, based on the information on the moving direction; The management server further includes, in the second identification process, determining whether or not identification information of a second camera located ahead in the moving direction is included in the tracking assistance information; When it is determined that the identification information of a second camera located ahead in the moving direction is included in the tracking support information, the persons included in the image of the second range are narrowed down based on the identification information of the second camera. A person identification system comprising:

9. A person identification program that causes a computer to identify a person included in a camera image, The person identification program, A process of acquiring user information assigned to each user terminal by short-range wireless communication using at least two antennas, and transmitting the user information to an outside via long-range communication using the at least two antennas; A process of acquiring, using at least two first cameras, an image of a first range including at least a part of a range of short-range wireless communication by any of the at least two antennas for each camera; acquiring an image of a second range that does not include a range of short-range wireless communication by the at least two antennas, using at least one second camera; A process of identifying a person included in an image of the first range for each camera; A process of identifying a person included in an image of the second range for each camera; on the computer, a total number of the at least two antennas is less than a total number of the at least two first cameras and the at least one second camera; The process of identifying a person included in an image of the first range for each camera includes: A process of generating recognition information of a person included in the image of the first range; a process of determining whether or not a user associated with the user terminal is included in the image of the first range, based on recognition information of a person included in the image of the first range, identification information of an antenna whose range of short-range wireless communication is included in the image of the first range, and the user information acquired by the antenna; generating tracking assistance information for a user associated with the user terminal when the user is determined to be included in the first range of images; Including, The process of identifying a person included in the image of the second range for each camera includes: A process of generating recognition information of a person included in the image of the second range; A process of determining whether or not a user that is the same as a user determined to be included in the image of the first range is included in the image of the second range based on recognition information of a person included in the image of the second range and the tracking assistance information; Includes A person identification program comprising:

Citation Information

Patent Citations

  • Evacuation route finding device and evacuation route finding method

    JP2009193549A

  • Monitoring system and person specification method

    JP2012008802A

  • Image processing apparatus, image processing method, and program

    JP2016122300A

  • Imaging apparatus, imaging system, and street light

    JP2019192969A

  • Information processor, and information processing program

    JP2020194295A