Multi-camera tracking system and multi-camera tracking program

Abstract

A multi-camera tracking system and a multi-camera tracking program are provided. [Solution] A multi-camera tracking system assigns the same tracking identification ID to the same person who appears in each of the images taken by the multiple cameras based on the images taken by the multiple cameras, and tracks the person to which the tracking identification ID has been assigned across the shooting ranges of the multiple cameras. The multi-camera tracking system further comprises a person determination means for determining, from among the people who appear in the images taken by the multiple cameras, people who belong to the majority when viewed from a predetermined viewpoint, and a person exclusion means for excluding people who are determined to belong to the majority by the person determination means from people who are targets of multi-camera tracking, and the ID assignment means assigns tracking identification IDs only to people who appear in each of the images taken by the multiple cameras and who have not been excluded by the person exclusion means.

Description

[Technical Field]

[0001] The present invention relates to a multi-camera tracking system and a multi-camera tracking program. [Background technology]

[0002] Conventionally, a multi-camera tracking technique has been known in which multiple cameras are installed in a certain area and an object (such as a person) within the range of the cameras is tracked using images captured by these cameras (see, for example, Non-Patent Document 1). More precisely, this multi-camera tracking is a process in which the same tracking identification ID is assigned to the same object captured in each of the images captured by the multiple cameras, and the object assigned the same tracking identification ID is tracked across the range of the multiple cameras. By using the above multi-camera object tracking technique, it is possible to analyze the behavior of the tracked object or confirm the safety of the tracked object. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Nicolai Wojke, Alex Bewley, Dietrich Paulus, "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", [online], March 21, 2017, University of Koblenz-Landau, Queensland University of Technology, [Retrieved January 6, 2023], Internet<URL:https: / / userpages.uni-koblenz.de / ~agas / Documents / Wojke2017SOA.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, in multi-camera tracking, it is necessary to assign the same tracking identification ID to the same object that appears in each of the images captured by the multiple cameras. For this reason, for example, if the object to be tracked is a person and many people appear in the images captured by the cameras, the amount of calculation required for the multi-camera tracking process would be extremely large if all of the people were to be the subject of multi-camera tracking. Therefore, if the people to be tracked can be limited to those who exhibit unusual behavior, such as suspicious or abnormal behavior (for example, when it is desired to track people who exhibit unusual behavior that does not belong to the majority, for purposes such as crime prevention), it would be a waste of computer resources to subject all of the people that appear in each of the images captured by the multiple cameras to multi-camera tracking.

[0005] Furthermore, with conventional multi-camera tracking, even if a person to be tracked is temporarily hidden by another person or object and disappears from view, tracking can be continued as long as the hiding time is short (for example, for a few dozen frames). However, tracking cannot be continued if the person to be tracked enters a space outside the camera's range of capture, such as a restroom or fitting room, and does not emerge for a while. However, in spaces such as the above-mentioned restroom or fitting room, which are outside the camera's range of capture but have a fixed single entrance and exit, and from which anyone who enters will always emerge, the tracking identification ID of a person who exits this space should be included among the tracking identification IDs of people who enter this space. By utilizing this, it should be possible to continue tracking a person who enters such a space. In the following description, a space outside the camera's range of capture and from which anyone who enters will always emerge, such as the above-mentioned restroom or fitting room, is referred to as a "specific space."

[0006] The present invention aims to solve the above-mentioned problems by providing a multi-camera tracking system and a multi-camera tracking program that can reduce the amount of calculation required for multi-camera tracking processing and prevent waste of computer resources required for multi-camera tracking processing when the person to be tracked can be limited to a person who behaves in a unique manner that does not belong to the majority. It is also an object of the present invention to provide a multi-camera tracking system and a multi-camera tracking program that can continue tracking (multi-camera tracking) of a person to be tracked even if the person enters a specific space such as a restroom or a changing room. [Means for solving the problem]

[0007] In order to solve the above problem, a multi-camera tracking system according to a first aspect of the present invention comprises an ID assignment means for assigning the same tracking ID to the same person who appears in each of the images taken by the multiple cameras based on the images taken by the multiple cameras, and the multi-camera tracking system performs multi-camera tracking, which is a process of tracking the person to whom the tracking ID has been assigned by the ID assignment means across the shooting ranges of the multiple cameras, and further comprises a person determination means for determining, from the people who appear in the images taken by the multiple cameras, a person who belongs to the majority who moves in the same way as many other people, and a person exclusion means for excluding, from people who are targets of the multi-camera tracking, people who are determined to belong to the majority by the person determination means, and the ID assignment means continues the process of assigning the tracking ID only to people who have not been excluded by the person exclusion means, from the people who appear in each of the images taken by the multiple cameras, and the person determination means When the user's area of ​​interest is near a blind spot within the shooting range, or a place where a person does not stagnate outside the flow of people, Among the people reflected in the images captured by the plurality of cameras, The aforementioned People outside the user's area of ​​interest is determined to be a person belonging to the majority, and when the user's area of ​​interest is a place within the shooting range where people tend to pass by or where people tend to gather, among the people reflected in the images captured by the multiple cameras, A person included in the user's area of ​​interest is determined to be a person belonging to the majority group.

[0009] In this multi-camera tracking system, the person determination means determines, based on the direction of travel of a person captured in images captured by the plurality of cameras, The aforementioned It may also be possible to determine who belongs to the majority group.

[0010] In this multi-camera tracking system, the person determination means determines, based on the density of people in the surrounding area of ​​a person captured in the images captured by the plurality of cameras, The aforementioned It may also be possible to determine who belongs to the majority group.

[0015] A multi-camera tracking program according to a second aspect of the present invention is a multi-camera tracking program for performing multi-camera tracking, which is a process of assigning the same tracking ID to the same person who appears in each of the images taken by the multiple cameras based on the images taken by the multiple cameras, and tracking the person to which the tracking ID has been assigned, across the shooting ranges of the multiple cameras, and causes a computer to function as an ID assignment means for assigning the same tracking ID to the same person who appears in each of the images taken by the multiple cameras based on the images taken by the multiple cameras, a person determination means for determining, from the people who appear in the images taken by the multiple cameras, a person who belongs to the majority who moves in the same way as many other people, and a person exclusion means for excluding people who are determined to belong to the majority by the person determination means from people who are targets of the multi-camera tracking, and the ID assignment means continues the process of assigning the tracking ID only to people who have not been excluded by the person exclusion means from the people who appear in each of the images taken by the multiple cameras When the user's area of ​​interest is near a blind spot within the shooting range, or a place where a person does not stagnate outside the flow of people, Among the people reflected in the images captured by the plurality of cameras, The aforementioned People outside the user's area of ​​interest is determined to be a person belonging to the majority, and when the user's area of ​​interest is a place within the shooting range where people tend to pass by or where people tend to gather, among the people reflected in the images captured by the multiple cameras, A person included in the user's area of ​​interest is determined to be a person belonging to the majority group. [Effects of the Invention]

[0017] A multi-camera tracking system according to a first aspect of the present invention, and 2 According to the multi-camera tracking program of the present invention, from a person who is a target of multi-camera tracking, Moves in the same way as many other people Individuals who are determined to belong to the majority are excluded, and the process of assigning tracking IDs continues only to those who are not excluded. This reduces the amount of calculation required for multi-camera tracking processing when the individuals to be tracked can be limited to those who do not belong to the majority and who behave in a unique manner, thereby preventing waste of computer resources required for multi-camera tracking processing. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram of a multi-camera tracking system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing a configuration of an edge device. [Figure 3] FIG. 2 is a block diagram showing the configuration of a cloud server. [Figure 4] FIG. 1 is a functional block diagram of a multi-camera tracking system. [Figure 5] 10 is a flowchart illustrating an example of a processing procedure in a cloud server according to the first embodiment. [Figure 6] FIG. 1 is an explanatory diagram of processing in a multi-camera tracking system. [Figure 7] FIG. 1 is an explanatory diagram of processing in a multi-camera tracking system. [Figure 8] FIG. 1 is an explanatory diagram of processing in a multi-camera tracking system. [Figure 9] FIG. 1 is an explanatory diagram of processing in a multi-camera tracking system. [Figure 10] FIG. 10 is a functional block diagram of a multi-camera tracking system according to a second embodiment. [Figure 11] 10 is a flowchart illustrating an example of a processing procedure in a cloud server according to the second embodiment. [Figure 12]FIG. 10 is an explanatory diagram of processing in the multi-camera tracking system of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] The present disclosure will be specifically described with reference to the drawings showing embodiments thereof.

[0021] [First embodiment] FIG. 1 is a schematic diagram of a multi-camera tracking system 100 according to a first embodiment. The multi-camera tracking system 100 is installed in a facility or outdoor area to be monitored, and monitors the movement of a target object in the facility or outdoor area. The multi-camera tracking system 100 uses multiple cameras 2 installed in, for example, a store or a train station, analyzes images captured by the cameras 2, and, if a suspicious person is detected among the target person traffic, saves the scene in which the person is captured. Generally, multi-camera tracking refers to a process of assigning the same tracking ID to the same person captured in each of the images captured by the multiple cameras, and tracking the person assigned the tracking ID across the capture ranges of the multiple cameras, based on the images captured by the multiple cameras.

[0022] If multiple cameras 2 are installed in locations with many people to be detected, such as large stores or stations with many passengers, and all people captured by different cameras are identified and the movements of the people captured across the multiple cameras 2 are tracked across the multiple cameras, the processing load on the multi-camera tracking system 100 would be excessive. Therefore, the multi-camera tracking system 100 of the first embodiment excludes from the multi-camera tracking those people who belong to the majority of people captured in images captured by the multiple cameras 2 and who move in a similar manner, and tracks people who are determined not to belong to the majority across different (multiple) cameras 2. This prevents multi-camera tracking of many people whose movements are determined to be normal within the capture range of the cameras 2, reducing the processing load on the multi-camera tracking system 100 while focusing on people whose movements are different from normal, thereby achieving accurate tracking and scene storage according to the purpose.

[0023] The multi-camera tracking system 100 includes a plurality of cameras 2, an edge device 1 to which the cameras 2 are connected, and a cloud server 3 that can be communicatively connected to the edge device 1. The multi-camera tracking system 100 may also include a client 4 that can be communicatively connected to the cloud server 3 and that displays analysis results.

[0024] Each of the multiple cameras 2 uses an image element that is compatible with visible light and / or near-infrared light, and outputs captured images. The cameras 2 output captured images in time series at a rate of several fps to several tens of fps. The multiple cameras 2 are each installed so as to look down from above, such as on the ceiling or upper part of a wall of the installation space, and are installed so as to cover the installation space with different capture ranges. The capture ranges may partially overlap each other. The cameras 2 may be of a type that is attached to the ceiling and has a field of view that covers the entire space 360 ​​degrees. If the installation space is outdoors, the cameras 2 may be installed on a wall or pillar so as to look down.

[0025] The camera 2 and the edge device 1 can be connected for communication via a local network LN, which may be wireless or wired. The local network LN is installed in a facility including an installation space. The local network LN may be a wired LAN or a wireless network such as WiFi or Bluetooth (registered trademark). The camera 2 sequentially transmits captured images to the edge device 1 via the local network LN.

[0026] The edge device 1 can be connected to the cloud server 3 via a network N. The network N is a wired and / or wireless communication network that may include a public communication network, a dedicated line, or a carrier network. The client 4 can be connected to the cloud server 3 and the edge device 1 via the network N.

[0027] In the multi-camera tracking system 100 of the first embodiment, an edge device 1 extracts feature values ​​from each image captured by multiple cameras 2 installed in an installation space, and detects objects such as a target person from the captured image based on the extracted feature values. The edge device 1 transmits object detection results for the captured images acquired from each camera 2 to a cloud server 3, correlating them with the identification data of the camera 2. The cloud server 3 integrates the detection results transmitted from the edge devices 1 and performs the multi-camera tracking process described below. When an object detected in the images captured by each camera 2 moves across the capture ranges of different cameras 2, the cloud server 3 identifies the object as the same object based on the feature values ​​of each object and tracks it. Furthermore, when there are many objects to be detected, the multi-camera tracking system 100 of the first embodiment appropriately selects an object of interest (a person whose movement is different from normal movement).

[0028] The configurations of the edge device 1 and cloud server 3 that realize such a multi-camera tracking system 100, as well as the details of the processing executed by each, will be described below.

[0029] 2 is a block diagram showing the configuration of the edge device 1. The edge device 1 is a box-shaped device that can be installed together with a camera 2. One edge device 1 may be installed for one camera 2, or one edge device 1 may be installed for multiple cameras 2. The edge device 1 includes a processing unit 10, a storage unit 11, a first communication unit 12, and a second communication unit 13.

[0030] The processing unit 10 includes one or more processors such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), or a neutral processing unit (NPU). The processing unit 10 includes a memory that is a temporary storage medium such as a static random access memory (SRAM) or a dynamic random access memory (DRAM). The processing unit 10 includes a timer and can acquire time information at each point in time from data from the timer. The processing unit 10 may be configured as a single piece of hardware (SoC: System On a Chip) that integrates a processor, a memory, a storage unit 11, a first communication unit 12, and a second communication unit 13. The specifications of the processing unit 10 may be the same or different between edge devices 1.

[0031] The processing unit 10 reads the edge-side program P1 stored in the storage unit 11 into the memory and executes it, thereby causing the processor to execute various processes described below and function as the edge device 1 of the present disclosure.

[0032] The storage unit 11 is a relatively large-capacity non-transitory storage medium such as a hard disk, a flash memory, etc. A part of the storage unit 11 may be removable.

[0033] The storage unit 11 stores a program (program product) required for the processing unit 10 to execute processing, the results of processing by the processing unit 10, and setting information for reference. The setting information includes identification data of the edge device 1, identification data of the connected camera 2, etc. The setting information includes a determination criterion for determining the majority in an image, which is set by processing described later. This determination criterion may be information on the specific movement of objects captured in the shooting range of the camera 2 to which the edge device 1 is connected, or information on the density of objects for determining that they are the majority. The determination criterion may be set in advance or may be determined by processing by the cloud server 3 described later.

[0034] The program product stored in the storage unit 11 includes an OS (Operation System) program, an edge-side program P1, and a detection learning model M1. The detection learning model M1 is selected in advance depending on the object to be detected. The edge-side program P1 is also selected to execute a process corresponding to the selected detection learning model M1. The edge-side program P1 and the detection learning model M1 may be deployed from the cloud server 3 or another download server depending on the object settings. The edge-side program P1 and the detection learning model M1 may be read by the processing unit 10 from the edge-side program P9 and the detection learning model M9 stored in a computer-readable non-transitory storage medium 9 and stored in the storage unit 11, or may be pre-stored at the time of shipment.

[0035] The first communication unit 12 is a communication device that realizes communication with the camera 2 via the local network LN. The first communication unit 12 is, for example, a network card for a LAN. The first communication unit 12 may also include an interface such as a USB (Universal Serial Bus) that is connected to the camera 2. The first communication unit 12 can be replaced by an interface that is connected to the camera 2 via a coaxial cable or another serial bus. The first communication unit 12 may be a communication device that supports a wireless network such as Wi-Fi or Bluetooth (registered trademark). The first communication unit 12 may include multiple communication devices that are compatible with various types of cameras 2. The processing unit 10 acquires image information from the camera 2 via the local network LN using the first communication unit 12. The first communication unit 12 may be the same device as the second communication unit 13, which will be described later.

[0036] The second communication unit 13 is a communication device that realizes communication via an external network N. The second communication unit 13 is connected to the network N via a router or the like. The second communication unit 13 may be a network card for a LAN, or may be a communication device that realizes carrier communication via a carrier network. The second communication unit 13 may be a communication device that supports a wireless network such as WiFi or Bluetooth (registered trademark). The second communication unit 13 may support secure communication with the cloud server 3 using SSL (Secure Socket Layer) or the like. The second communication unit 13 may be an interface that realizes a communication connection with the cloud server 3 via a dedicated line.

[0037] 3 is a block diagram showing the configuration of the cloud server 3. The cloud server 3 may be configured as a single server computer, or may be configured to distribute processing among multiple server computers. The cloud server 3 includes a processing unit 30, a storage unit 31, and a communication unit 32.

[0038] The processing unit 30 includes one or more processors such as a CPU, an MPU, a GPU, an NPU, etc. The processing unit 30 includes a memory that is a temporary storage medium such as an SRAM or a DRAM.

[0039] The storage unit 31 is a relatively large-capacity non-temporary storage medium such as a hard disk, flash memory, etc. The storage unit 31 stores programs (program products) and setting information required for the processing unit 30 to execute the processes described below.

[0040] The program products stored in the storage unit 31 include a server program P31 and a multi-camera tracking program P32. The server program P31 is a program for performing a function of controlling transmission and reception of data with the edge device 1. The multi-camera tracking program P32 is a program for causing the processing unit 30 to perform a function of tracking an object that appears across images captured by multiple cameras 2, as will be described later.

[0041] The server program P31 and the multi-camera tracking program P32 may be server programs P81 and P82 stored in a non-temporary storage medium 8 that is read by the processing unit 10 and stored in the memory unit 11, or they may be downloaded from another download server not shown.

[0042] The storage unit 31 stores a tracked object DB 310 that stores an identification ID assigned to an object detected from an image captured by the camera 2 and a feature amount (feature vector) of the object in association with each other.

[0043] The setting information stored in the storage unit 31 includes a correspondence between data and a name identifying a location where the edge device 1 is installed, associated with the identification data of the edge device 1, and identification data of the camera 2 from which the edge device 1 can acquire images. The setting information includes the identification data of the camera 2 associated with the data identifying the location and data on a map or blueprint of the location where the camera 2 is installed. The setting information may further store, as a whitelist, identification data of locations where captured images that the user can view are taken, associated with the user's account data. This allows the user to operate the client 4, specify the identification data of a location, and specify setting information for the captured images taken at that location. The user can operate the client 4, specify the identification data of a location, and view the results of multi-camera tracking processing for the captured images taken at that location.

[0044] The communication unit 32 is a communication device that establishes a communication connection with the edge device 1 via the network N. The communication unit 32 may also be a communication device that establishes a communication connection with the client 4 via the network N.

[0045] In the multi-camera tracking system 100 configured as above, a person is set as an object to be detected, and processing for appropriately tracking the person captured by the multiple cameras 2 will be described.

[0046] FIG. 4 is a functional block diagram of the multi-camera tracking system 100. The multi-camera tracking system 100 performs the functions shown in FIG. 4 based on an edge-side program P1 in the edge device 1 and a multi-camera tracking program P32 in the cloud server 3. The functions shown in FIG. 4 may be appropriately distributed between the edge device 1 and the cloud server 3 and may be performed by either device. In the following description, the processing unit 10 of the edge device 1 functions as a person detection unit 101 that detects people and a detection result transmission unit 102 based on the edge-side program P1. In the following description, the processing unit 30 of the cloud server 3 functions as a detection result reception unit 301, an ID assignment unit 302, a person determination unit 303, a person exclusion unit 304, a tracking unit 305, and a result output unit 306 based on the multi-camera tracking program P32.

[0047] The person detection unit 101 receives a captured image from the camera 2 via the first communication unit 12. The person detection unit 101 detects a detection target object, i.e., a person, that appears in the received captured image using the detection learning model M11. When the captured image is input, the person detection unit 101 uses the detection learning model M11 to obtain coordinate data of an area in the captured image in which a person appears, feature values ​​(feature vectors) of the area in which the person appears, and a probability that the person is a person. If no person is detected, the person detection unit 101 obtains a result from the detection learning model M11 that there are "zero" people. The area in which a person appears may be an area from the person's head to the toes, or may be an area consisting of only the person's head. If multiple people appear in the captured image, the person detection unit 101 obtains, as detection results, coordinate data, feature values, and a probability from the detection learning model M11, in association with IDs provisionally assigned to each of the multiple people.

[0048] The person detection unit 101 associates a captured image in which a person is detected, time information when the captured image was captured, and a detection result obtained from the detection learning model M11, and stores these in the storage unit 11. The person detection unit 101 may discard (delete) captured images in which a person is not detected, rather than storing them in the storage unit 11.

[0049] The detection learning model M11 disclosed herein uses DeepSORT (Simple Online and Realtime Tracking with a deep association metric) as described in Non-Patent Document 1, and outputs the coordinate data and the above-mentioned time information from the detection results as a track. The person detection unit 101 inputs sequentially received captured images in chronological order to the detection learning model M11 that employs DeepSORT. The detection learning model M11 recognizes objects with the same feature values ​​in consecutively input captured images as the same object. By using the detection learning model M11, the person detection unit 101 associates each detected person with a temporary ID and obtains the coordinate data of the detection results together with time information in the form of a short-term track within the captured image range. The track data includes the start time when the person to be detected first appears within the captured image range, the end time when the person is last seen, and data indicating the person's movement between the start time and the end time. The movement of the person to be detected is time-series data of specific positions such as the center of gravity, center position, and head position of the area detected as the person.

[0050] When an area outside a region of interest (ROI) in the captured image that can be acquired from each camera 2 is set in advance or by processing described below, the person detection unit 101 may mask the area outside the ROI and input the masked captured image to the detection learning model M11. The person detection unit 101 may mask an area where the density of people detected in the captured image is equal to or higher than a set density set in the setting information. For people detected from outside the masked area, the processing unit 10 acquires coordinate data, feature values, and accuracy of the area in which the people are captured.

[0051] The detection result transmission unit 102 transmits, from among the photographed images stored in the storage unit 11, a photographed image in which a person has been detected, together with the identification data of the edge device 1, the identification data of the camera 2, the photographing time, and the detection result to the cloud server 3. The detection result transmission unit 102 may transmit the detection result including the feature amount obtained from the photographed image, without sending the photographed image itself to the cloud server 3.

[0052] The processing unit 30 of the cloud server 3 receives the captured image, the identification data of the edge device 1, the identification data of the camera 2, the capture time, and the detection result from the edge device 1 using the function of the detection result receiving unit 301. The detection result receiving unit 301 stores the received captured image, the identification data of the edge device 1, the identification data of the camera 2, the capture time, and the detection result (the tentative ID, coordinate data, feature amount, and accuracy of the detected person) in the memory unit 31.

[0053] The processing unit 30 assigns a tracking ID to a person detected from a photographed image using the function of the ID assigning unit 302. The ID assigning unit 302 assigns the same tracking ID to the same person appearing in the photographed images, even if the images were taken with different cameras 2 at different times. For this purpose, the ID assigning unit 302 specifies a temporary ID of the person detected from the photographed image of the target, and queries the tracking object DB 310 to see if an assigned tracking ID that corresponds to the feature amount of the person with the specified temporary ID is stored. If the ID assigning unit 302 receives a response indicating that an assigned tracking ID is stored for the feature amount of the person with the specified temporary ID, it assigns the assigned tracking ID to the person detected in the photographed image of the target. When a tracking ID that has already been assigned is not stored for the feature of the person with the specified temporary ID and a new tracking ID is assigned, the ID assigning unit 302 associates the assigned tracking ID with the person detected in the target photographed image. The association between the tracking ID and the feature in the tracking object DB 310 may be reset after a predetermined time has elapsed, except for when, for example, people with the same feature across days need to be considered the same person.

[0054] The processing unit 30 continues to perform the function of the above-mentioned ID assigning unit 302 for persons other than those excluded by the person exclusion unit 304, which will be described later.

[0055] The processing unit 30 uses the function of the identification ID assigning unit 302 to select an image taken by a specific camera 2 from among the multiple cameras 2, and selects multiple images taken within a predetermined time period by the same camera 2 and surrounding cameras 2, based on the shooting time of the selected image and the identification data of the camera 2. The identification ID assigning unit 302 queries the tracking object DB 310 described above for people (objects) that are detected in the detection results of the selected multiple images and have not been excluded, and assigns them tracking identification IDs.

[0056] The processing unit 30 uses the function of the person determination unit 303 to determine which people belong to the majority of people who appear in images captured by the multiple cameras 2. The person determination unit 303 executes the following process before instructing the ID assignment unit 302 to assign a tracking ID to the detected person. The processing unit 30 stores, as the majority ID, the temporary ID of a person who is determined to belong to the majority of people among the temporary IDs of people appearing in each captured image. The processing unit 30 may exclude a person who is determined to belong to the majority by any one of the multiple cameras 2, for example, a specific camera 2, from being assigned a tracking ID, or may exclude a person who is determined to belong to the majority by all cameras 2. In this case, in order to trace the past movements of people who appear in images captured by cameras other than the specific camera 2, it is desirable to assign tracking IDs to people who do not belong to the majority of people by the other cameras 2.

[0057] For example, if the density of multiple people detected by the person detection unit 101 in a target captured image is equal to or greater than a set density within a predetermined range, the person determination unit 303 determines that the person in the predetermined range belongs to the majority. In another example, the person determination unit 303 determines whether the direction of travel of the target person is the majority based on track data in the captured images of multiple people captured by the multiple cameras 2. The person determination unit 303 may determine ex post the range in each captured image where the density is equal to or greater than a set density, or the direction of travel of the majority of people in the captured image, based on the detection results by the person detection unit 101 for images actually captured at the location where the multi-camera tracking system 100 is installed. The range in which people are detected in each captured image where the density is equal to or greater than a set density may be determined for each time period of the capture time, and similarly, the direction of travel of the majority of people in the captured image may be determined for each time period of the capture time. The density of people who make up the majority varies depending on the time of day, such as in a store corridor or inside or outside a station ticket gate, and the movements of people in the majority also vary depending on the time of day, so the criteria for determining who belongs to the majority may change dynamically depending on the time of day.

[0058] In another example, the person determination unit 303 may determine a person belonging to the majority group based on the positional relationship between a person appearing in an image captured by the camera 2 and the user's ROI (region of interest). Specifically, for example, the person determination unit 303 determines that, among people appearing in a captured image of the target, a person who is outside the user's ROI set for that captured image belongs to the majority group. In this case, if the purpose of installing the multi-camera tracking system 100 is crime prevention or the like, the ROI may be a location close to a blind spot within the capture range, or a location that is unnatural for a person to deviate from the flow of people and stagnate. Conversely, the person determination unit 303 may determine, among people appearing in a captured image of the target, a person who is included within the user's ROI set for that captured image as belonging to the majority group. In this case, if the purpose of installing the multi-camera tracking system 100 is crime prevention or the like, the ROI may be a location within the capture range where people tend to pass by or where people tend to gather. In this case, the ROI may be a location where a predetermined percentage or more of the person's body is included within the ROI.

[0059] The processing unit 30 uses the function of the person exclusion unit 304 to exclude people who are determined to belong to the majority by the person determination unit 303 from the people detected in each captured image. The person exclusion unit 304 stores, for the target captured image, a temporary ID that matches the ID of the majority stored based on the determination result of the person determination unit 303, as a temporary ID that is not to be assigned a tracking ID by the ID assignment unit 302. The processing unit 30 does not process the temporary ID that is not to be assigned a tracking ID by the ID assignment unit 302, so that a tracking ID is not assigned to the corresponding person.

[0060] Using the function of the tracking unit 305, the processing unit 30 uses an image taken by a specific camera 2 out of multiple cameras 2 as a reference, and performs multi-camera tracking to track the movement trajectory of people who appear in the image and have been assigned a tracking identification ID by the identification ID assignment unit 302, using the detection results in images taken by other cameras 2.

[0061] 5 is a flowchart showing an example of a processing procedure in the cloud server 3 of the first embodiment. The processing procedure shown in Fig. 5 is executed by the processing unit 30 of the cloud server 3 based on the multi-camera tracking program P32 when detection results are received from the edge devices 1 corresponding to the multiple cameras 2.

[0062] The processing unit 30 receives, via the detection result receiving unit 301, the detection result for the captured image taken by a different camera 2 together with the identification data of the camera 2 (step S311). In step S311, the processing unit 30 may receive the captured image itself and the detection result, or may receive the detection result (including the feature amount) without the captured image.

[0063] The processing unit 30 stores the received detection result in association with the identification data of the camera 2 and the shooting time (step S312). In step S312, if the detection result is track data including time information, the processing unit 30 stores the detection result in association with the identification data of the camera 2, since the detection result has already been associated with the shooting time. The processing unit 30 may continue to sequentially execute the processes of steps S311 and S312 in parallel with the processes of step S313 and thereafter, which will be described later.

[0064] Using the function of the person determination unit 303, the processing unit 30 reads out unprocessed received detection results that are associated with the identification data of a specific reference camera 2 from the detection results stored in the memory unit 31 (step S313).

[0065] Based on the read detection results, the processing unit 30 determines whether or not the person reflected in the captured image corresponding to the detection result belongs to the majority (step S314), and stores the determination result (step S315). In step S314, the processing unit 30 determines for each captured image whether or not the person belongs to the majority based on the coordinate data, feature amount, and / or accuracy of each person in the image associated with the temporary ID assigned to the detected person. The processing in step S314 may be any of a method based on the moving direction of the person, a method based on the density of detected people, and a method based on a preset ROI, as will be described later.

[0066] The processing unit 30 reads out from the storage unit 31 the detection results for the images taken by the cameras 2 other than the specific camera 2 corresponding to the detection result read out in step S313, within a predictable range of shooting times (step S316). In step S316, the processing unit 30 reads out the detection results for the images taken by the other cameras 2, within a realistic time difference range in which a person may be captured across different cameras 2, based on the arrangement of the cameras 2 at the location where the multi-camera tracking system 100 is installed.

[0067] For each of the detection results read out in step S316, the processing unit 30 determines whether the person appearing in the captured image corresponding to each detection result belongs to the majority group using the function of the person determination unit 303 (step S317), and stores the determination result (step S318).

[0068] The processing unit 30, using the function of the person exclusion unit 304, determines (step S319) and stores (step S320) the provisional ID of a person who belongs to the majority and should be excluded for each person identified by a provisional ID, based on the determination result stored in the storage unit 31. In step S319, the processing unit 30 determines the provisional ID assigned to the person determined to belong to the majority as the provisional ID of the person who should be excluded, and stores it.

[0069] The processing unit 30 acquires a tracking identification ID from the tracking object DB 310 using the function of the identification ID assigning unit 302 for each person other than the person excluded by the processing of the person exclusion unit 304, among the people who appear in each captured image stored in the storage unit 31 (step S321). In step S321, the processing unit 30 specifies a temporary ID other than the temporary ID determined in step 319, and determines whether or not a feature that matches the feature corresponding to the specified temporary ID has already been stored in the tracking object DB 310.

[0070] The processing unit 30 associates each captured image with the tracking identification ID acquired in step S321 and stores the associated images in the storage unit 31 (step S322). In step S322, the processing unit 30 associates the tracking identification ID assigned to each person detected in each captured image with the information on the time the captured image was taken, and the identification data of the camera 2 that took the image, and stores the associated results. This makes it possible to identify the time and in which camera 2 the person identified by the tracking identification ID appeared.

[0071] The processing unit 30 executes the processes from step S313 to step S322 on a plurality of captured images obtained from different cameras 2 using the functions of the person determination unit 303, the person exclusion unit 304, and the identification ID assignment unit 302.

[0072] Using the function of the tracking unit 305, the processing unit 30 identifies and stores the movement trajectory for each person identified by the tracking identification ID based on the detection results from each of the images taken by different cameras 2 (step S323), and then terminates the processing.

[0073] 6 to 9 are explanatory diagrams of the processing in the multi-camera tracking system 100. Using Fig. 6 to Fig. 9, the processing procedure shown in Fig. 5 will be explained with specific examples. Each image in Fig. 6 to Fig. 9 shows an example in which the multi-camera tracking system 100 is installed mainly near the entrance and exit of a store.

[0074] FIG. 6 is a line diagram that shows a schematic representation of images captured by multiple cameras 2. The upper center of FIG. 6 shows an example of an image captured by a camera 2 that is set as a specific camera 2 among the multiple cameras 2. In the example of FIG. 6, a large number of people are lined up horizontally in the center of the image captured by the specific camera 2 (indicated by I1 in FIG. 6). These lined people are moving naturally, as they would at the entrance or exit of a store where the camera is installed. In the captured image I1, the majority of people are walking from left to right or right to left in the captured image I1. In the captured image I1, two or so people are seen as being out of line with the large number of people. In the captured image I1 shown in FIG. 6, the multi-camera tracking system 100 of the first embodiment excludes the people in the center who belong to the majority who are behaving similarly from the tracking targets, and performs processing to track the people who are out of line with the majority.

[0075] FIG. 6 shows the location where a specific camera 2 is installed and an image (I2 in FIG. 6) taken from another camera 2 whose shooting range overlaps with that of the specific camera 2. The time of taking the image I2 is almost the same as the time of taking the image I1. FIG. 6 also shows an image (I3 in FIG. 6) taken from a camera 2 installed in a space following the store entrance. FIG. 6 also shows an image (I4 in FIG. 6) taken from a camera 2 installed outside the store entrance. The images I1, I2, I3, and I4 were taken at almost the same time.

[0076] FIG. 7 shows an example of the results of processing a captured image by the person detection unit 101. In FIG. 7, the captured image I1 shown in FIG. 6 and a captured image I1' captured a short time later by the same camera 2 are displayed side by side. In the captured image I1 and the captured image I1', the bounding boxes of the people detected by the person detection unit 101 are indicated by bold lines, and the temporary IDs assigned by the person detection unit 101 are shown for explanatory purposes. In FIG. 7, some of the bounding boxes are omitted for readability. In the captured image I1, people with temporary IDs "01" to "15" are detected, and in the captured image I1', objects with newly assigned temporary IDs are also detected. In the example shown in FIG. 7, the same temporary ID is assigned by the detection learning model M11 to the same person who appears in images captured by the same camera 2 at different times.

[0077] For the captured image I1, the person detection unit 101 obtains the shooting time "2025 / 02 / 01 11:23:45", the temporary IDs "01"-"15" of each detected person, and the coordinate data, feature values, and accuracy of each person.For the captured image 11', the person detection unit 101 obtains the shooting time five seconds later "2025 / 02 / 01 11:23:50", the temporary IDs assigned to the images of each detected person, and the coordinate data, feature values, and accuracy of each person.

[0078] The captured image I1 and the captured image I1' were taken five seconds apart, and the same person appears in each. For example, the person assigned the temporary ID "01" in the captured image I1 is the same as the person assigned the temporary ID "01" in the captured image I1'. The person appearing in the upper center of the image assigned the temporary ID "02" in the captured image I1 is the same as the person appearing in the upper right corner of the image assigned the temporary ID "02" in the captured image I1'. Similarly, the person appearing in the lower left corner of the image assigned the temporary ID "14" in the captured image I1 is the same as the person appearing in the lower left corner of the image assigned the temporary ID "14" in the captured image I1'. If the same person can be identified within the same captured range, the person detection unit 101 acquires the detection result as a track. For example, for a person with the same provisional ID "01" who remains motionless, the coordinate data of the start position of the reflection at the start time "2025 / 02 / 01 11:23:45" and the coordinate data of the end position of the reflection at the end time "2025 / 02 / 01 11:23:50" are obtained, along with the feature values ​​and accuracy. In addition, for the person with provisional ID "02" in the captured image I1 (provisional ID "02" in the captured image I1') who is still visible 5 seconds later, the provisional ID is set to "02," and trajectory data, feature values, and accuracy are obtained between the coordinate data of the reflection start position at the start time "2025 / 02 / 01 11:23:40" before the captured image I1 and the coordinate data of the reflection end position at the end time "2025 / 02 / 01 11:23:55" after the captured image I1'.

[0079] For people with provisional IDs "03" and onward who continue to appear in the image for a predetermined time (for example, 5 seconds) or more at the time they disappear from the shooting range, the person detection unit 101 acquires, each time the predetermined time elapses, coordinate data of the image's start position and coordinate data of the image's end position, feature amount, and accuracy. The process for determining the traveling direction of a person appearing in the captured image I1 is not limited to the change from the captured image I1 to the captured image I1', and may also be determined from the change from an image captured a short time ago to the captured image I1.

[0080] Using either or both of the captured image I1 and I1' shown in Fig. 7, the person determination unit 303 determines the majority of people among those appearing in the captured image I1. Fig. 8 is an explanatory diagram of the processing performed by the person determination unit 303. Fig. 8 explains three patterns of processing performed by the person determination unit 303 for the captured image I1 (I1') shown in Figs. 6 and 7.

[0081] In pattern A, the person determination unit 303 recognizes that the movement of the person assigned the temporary ID "02" in the captured image I1, indicated by the dashed line, from the captured image I1' shown in the actual image is from left to right, as indicated by the arrow in FIG. 8A. The person determination unit 303 recognizes that the movement of the person assigned the temporary ID "14" in the captured image I1, indicated by the dashed line, is from right to left. Similarly, the person determination unit 303 identifies the traveling direction of the majority from the trajectory of the movement of other people from the captured image I1 to the captured image I1' or the trajectory of the movement of other people within the shooting range of the captured image I1 before and after that. The person determination unit 303 stores the temporary ID of a person whose traveling direction matches the traveling direction of the majority of people among the people who appear in the target captured image I1 and are assigned temporary IDs, as the ID of the majority of people appearing in the captured image I1. In the case of FIG. 8A, the person determination unit 303 stores people (provisional IDs) such as "02," "03," "04," and "12" who are moving from left to right as the majority of people.

[0082] In pattern B, the person determination unit 303 extracts a predetermined range indicated by a dashed line in the captured image I1 of pattern B, shifting it from the top left of the image, and determines whether the density of detected people within the extracted predetermined range is equal to or greater than a set density. The person determination unit 303 determines that the density of detected people within the range indicated by a thick line in the captured image I1 of pattern B is equal to or greater than the set density. The person determination unit 303 identifies the range obtained by integrating the ranges determined to have a density equal to or greater than the set density as the range in which the majority of people are captured. In the example of pattern B in FIG. 8, the overlapping portion of the ranges indicated by the thick lines determined to have a density equal to or greater than the set density is indicated by hatching as the integrated range, and is identified as the range in which the majority of people are captured. These ranges may be identified for each captured image, or an average range of the identified ranges may be used after a certain amount of time has passed since the multi-camera tracking system 100 began operation. The person determination unit 303 stores the provisional IDs of people whose detection ranges overlap with the range identified as the range in which the majority of people are captured as the IDs of the majority of people. In the case of FIG. 8B, the person determination unit 303 stores people (provisional IDs) excluding "01," "06," "12," "14," and "15" as the majority of people. In this case, the person with ID "14" in the captured image I1 is not determined to be a majority of people because his / her direction of travel is from right to left.

[0083] In pattern C, the person determination unit 303 may store, as the ID of the majority person, the provisional ID of a person whose head area is included in the detection range shown by the dashed line in a ROI (an area where it is unlikely that a person making unusual movements is present) shown by hatching that is set in advance for the captured image I1. The area of ​​the ROI shown by hatching may be specified for each camera 2 by the user operating the client 4 in the server program P31 of the processing unit 30, or may be set in advance. In the case of FIG. 8C, the person determination unit 303 stores the persons (provisional IDs) excluding "01" and "15" as the majority person.

[0084] The processing unit 30 performs the processing of the above-mentioned person detection unit 101 and person determination unit 303 not only on the captured image I1 and captured image I1' captured by a specific camera 2, but also on the captured images I2, I3, and I4 captured by other cameras 2.

[0085] 8 may be fed back to the processing of the person detection unit 101 of the edge device 1. In this case, the specified range is used as a mask range, and the person detection unit 101 does not need to perform person detection processing on the mask range, thereby reducing the processing load accordingly.

[0086] The processing unit 30 of the cloud server 3 uses the function of the person exclusion unit 304 to store each of the captured images I1, I2, I3, and I4 after executing the processing of the function of the person determination unit 303, excluding the provisional IDs of the majority persons determined by the person determination unit 303. The processing unit 30 executes the processing for the captured images for which provisional IDs remain.

[0087] The processing unit 30 causes the ID assigning unit 302 to assign a tracking ID to a person who is determined not to be the majority of people detected in the images captured by each camera 2. This eliminates the need to perform a process of assigning a tracking ID to people who belong to the majority. FIG. 9 highlights people to whom tracking IDs have been assigned by the ID assigning unit 302 in images captured by multiple cameras 2. As shown in FIG. 9, the same tracking ID is assigned to people who have the same feature amounts and are the same among people detected in images captured by different cameras 2.

[0088] The tracking unit 305 extracts, from the images captured by the multiple cameras 2, images that show people who have not been excluded by the person exclusion unit 304 and who have been assigned the same tracking identification ID. The tracking unit 305 identifies the trajectory of the target person from the shooting time of each of the extracted images, and stores the identified multi-camera tracking data in association with the tracking identification ID in the storage unit 31. The multi-camera tracking data includes identification data for the multiple cameras 2 or data that identifies the shooting range of each of the multiple cameras 2, time information at which the person begins to appear in each shooting range, and time information at which the person ends to appear. For the person with tracking identification ID "G001" in Figure 9, the tracking results show that the person moved from the location where a specific camera 2 was installed to another location but remained there for a long period of time, such as [CAMERA01 2025 / 02 / 01 11:23:45-2025 / 02 / 01 11:34:56, CAMERA02 2025 / 02 / 01 11:23:40-2025 / 02 / 01 11:34:50, CAMERA03 2025 / 02 / 01 11:21:05-2025 / 02 / 01 11:22:55, CAMERA04 2025 / 02 / 01 11:37:05-2025 / 02 / 01 11:39:56].

[0089] 9 can be output to the display unit of the client 4 in response to an operation from the client 4 by the function of the result output unit 306 of the cloud server 3. When the user selects one from the list of tracking identification IDs, the result output unit 306 can output to the client 4, as shown in FIG.

[0090] The multi-camera tracking system 100 of the first embodiment can reduce the amount of calculation required for processing multi-camera tracking when the object (person) to be tracked can be limited to an object (person) that moves in a special way that does not belong to the majority, thereby preventing waste of computer resources required for processing multi-camera tracking.

[0091] [Second embodiment] In the second embodiment, in a facility where the multi-camera tracking system 100 is installed, a tracking identification ID is stored so that tracking of a person detected in an image captured by a camera 2 whose capture range includes an entrance / exit to a specific space where photography is not possible is not interrupted.

[0092] The configuration of the multi-camera tracking system 100 of the second embodiment is the same as that of the multi-camera tracking system 100 of the first embodiment, except for the functions and processing procedures on the side of the cloud server 3, which will be described later. Therefore, among the configurations of the multi-camera tracking system 100 of the second embodiment described below, the configurations common to the multi-camera tracking system 100 of the first embodiment are assigned the same reference numerals, and detailed description thereof will be omitted.

[0093] FIG. 10 is a functional block diagram of a multi-camera tracking system 100 according to the second embodiment. The functions shown in FIG. 10 may be appropriately distributed between the edge device 1 and the cloud server 3, and may be performed by either device. The processing unit 10 of the edge device 1 functions as a person detection unit 101 and a detection result transmission unit 102 based on an edge-side program P1. The processing unit 30 of the cloud server 3 according to the second embodiment functions as a detection result reception unit 301, an identification ID assignment unit 302, a specific space definition unit 307, an ID storage unit 308, an ID selection unit 309, a tracking control unit 311, and a warning unit 312 based on a multi-camera tracking program P32. Detailed description of functions common to the first embodiment will be omitted.

[0094] The processing unit 30, using the function of the specific space definition unit 307, defines an area within the captured image of each camera 2 that corresponds to a specific space, which is a space outside the capture range of the multiple cameras 2 and from which a person who enters will always come out. The specific space definition unit 307 defines an area that corresponds to a specific space for each of the captured images of the multiple cameras 2 using pre-settings. A specific space may be a place where it is not possible to install a camera 2 for privacy reasons, such as a restroom, a fitting room, a changing room, or a nursing room, or a place with a blind spot that ends in a dead end. An area that corresponds to a specific space is defined as a range that is always reflected in the captured image both before entering and after exiting. A range that can be moved to outside the capture range of the camera 2 (such as an exit) after entering the specific space cannot be specified. The specific space definition unit 307 may read out the setting information stored in the memory unit 31 and set it as a specific space, or may accept the specification of the range of the specific space by the user of the client 4 specifying an area with a pointing device for the images captured by each camera 2, or by inputting coordinates.

[0095] The processing unit 30 uses the function of the ID storage unit 308 to identify a person who has entered the specific space defined by the specific space definition unit 307, and separately stores the tracking ID assigned to the identified person by the ID assignment unit 302 in the ID storage area of ​​the tracked object DB 310. The ID storage unit 308 may exclude people other than those who have entered the specific space from those to whom tracking IDs are assigned, thereby reducing the processing load of the tracking ID assignment process. In this case, in order to trace past movements of people captured in images captured by cameras 2 other than the camera 2 capturing an area corresponding to the entrance / exit of the specific space in its shooting range, tracking IDs are assigned to people, including those who have not entered the specific space, by the other cameras 2. Note that the processing unit 30 can detect the movement of the person from the captured images of the person around the area corresponding to the entrance / exit of the specific space, and determine whether the person has entered or exited the specific space.

[0096] The processing unit 30 functions as an ID assignment determination unit that determines whether a tracking ID has been assigned to a person identified by the function of the ID storage unit 308 as having entered a specific space, and may further function as an ID addition assignment unit that assigns a tracking ID to the person to be determined by the ID assignment unit 302 when it is determined that a tracking ID has not been assigned. For example, if the specific area is near the entrance of a facility, it may be the case that a tracking ID has not yet been assigned to a person identified as having entered the specific space. In such a case, as described above, by determining whether a tracking ID has been assigned to a person identified as having entered the specific space and determining that a tracking ID has not been assigned, a tracking ID is assigned to the person to be determined and stored in the ID storage unit 308, the ID selection unit 309, which will be described later, can select a tracking ID for a person who has entered a specific area near the entrance of a facility, such as the one described above, from the tracking IDs stored in the ID storage unit 308 when the person leaves the specific area.

[0097] The processing unit 30 uses the function of the ID selection unit 309 to identify a person who has exited the specific space defined by the specific space definition unit 307, and matches the identified person with a person who has entered the specific space. The ID selection unit 309 determines whether the feature values ​​of a person who was not detected in the image of the previous frame captured by the same camera 2 and who was detected in an area corresponding to the entrance / exit of the specific space in the target captured image (i.e., the person who has exited the specific space) are substantially identical to the feature values ​​corresponding to the tracking ID stored in the ID storage area (the difference in the feature values ​​is within a predetermined range), and determines that a match has occurred if the feature values ​​are substantially identical. Based on the matching result, the ID selection unit 309 selects the tracking ID stored for the person determined to have matched (selects the tracking ID stored for the person determined to have matched as the tracking ID for the person who has exited the specific space).

[0098] Even if the ID selection unit 309 is unable to perform the above-mentioned matching (matching is unsuccessful) between the feature amount of a person who has left the specific space in the image captured at the time the person left the specific space and the feature amount corresponding to the tracking ID stored in the ID storage area (the feature amount of the person who entered the specific space), it is desirable to continue multi-camera tracking of the person who has left the specific space and continue matching between the person who has entered the specific space and the person who has left the specific space using the feature amount of the person in the image captured after the time the person left the specific space and the feature amount corresponding to each tracking ID stored in the ID storage area (the feature amount of each person who has entered the specific space). This is because the facial orientation of the person in the image captured at the time the person left the specific space and the image captured at the time the person entered the specific space often differ, and the difference in the feature amount of the person in these captured images will be somewhat large. Therefore, it is important that the ID selection unit 309 continues the above matching process for a predetermined time (buffer time) until the difference between the feature amount of the person in the image captured after the person exits the specific space and the feature amount corresponding to each tracking identification ID stored in the identification ID storage area (the feature amount of each person who entered the specific space) falls within a predetermined range. This makes it easier for the ID selection unit 309 to successfully match the person who entered the specific space with the person who exited the specific space and to select the tracking identification ID of the person who exited the specific space from the tracking identification IDs stored in the identification ID storage area.

[0099] The processing unit 30, by the function of the tracking control unit 311, uses the tracking identification ID selected by the ID selection unit 309 to perform multi-camera tracking that tracks the trajectory of movement of a person who has left the specific space, if the person is also captured on other cameras 2, using the detection results in the images captured by those other cameras 2. In other words, the tracking control unit 311 performs control to continue tracking of the person who has left the specific space, using the tracking identification ID selected by the ID selection unit 309.

[0100] The processing unit 30 uses the function of the warning unit 312 to warn the user when a person who has entered the specific space does not come out of the specific space for a predetermined time or longer. Specifically, the warning unit 312 outputs a warning to the corresponding client 4 when a person who has entered the specific space is detected in an area corresponding to the entrance / exit of the specific space and then the person who entered the specific space is not detected in the vicinity of the specific space for a predetermined time or longer after the tracking identification ID of this person is saved in the ID saving area. The results of the processing by the tracking control unit 311 are saved in the memory unit 31 even if they are not subject to a warning, and it is desirable that they can be traced from the client 4, similar to the function of the result output unit 306 in the first embodiment.

[0101] Each of the above-mentioned functions will be explained using a flowchart and a specific example. Fig. 11 is a flowchart showing an example of a processing procedure in the cloud server 3 of the second embodiment. The processing procedure shown in Fig. 11 is executed by the processing unit 30 of the cloud server 3 based on the multi-camera tracking program P32 when detection results are received from the edge devices 1 corresponding to the multiple cameras 2.

[0102] The processing unit 30 receives, via the detection result receiving unit 301, the detection results for the images captured by different cameras 2 together with the identification data of the cameras 2 (step S331). In step S331, the processing unit 30 may receive the captured images themselves and the detection results, or may receive the detection results (including feature amounts) without the captured images.

[0103] The processing unit 30 stores the received detection result in association with the identification data of the camera 2 and the shooting time (step S332). In step S332, if the detection result is track data including time information, the processing unit 30 stores the detection result in association with the identification data of the camera 2, since the detection result has already been associated with the shooting time. The processing unit 30 may continue to sequentially execute the processes of steps S331 and S332 in parallel with the processes of step S333 and thereafter, which will be described later.

[0104] The processing unit 30 acquires a tracking ID from the tracking object DB 310 for each person appearing in each captured image indicated by the stored detection results, using the function of the ID assigning unit 302 (step S333). In step S333, if a feature amount that matches (the difference is within a predetermined range) with a feature amount included in the detection results is already stored in the tracking object DB 310, the processing unit 30 assigns the tracking ID corresponding to the already stored feature amount to the target person; if not, the processing unit 30 assigns a new tracking ID and stores it in association with the feature amount. The processing unit 30 executes the process of step S333 for captured images also captured by other cameras 2 within the time that the person can realistically move.

[0105] The processing unit 30 uses the coordinate data included in the detection result of each captured image to identify people who are in an area corresponding to an entrance / exit of the specific space among people who appear in the captured image (step S334). The processing unit 30 determines whether or not each person who appears in the captured image is in an area corresponding to an entrance / exit of the specific space.

[0106] The processing unit 30 determines whether the tracking ID assigned to the person identified in step S334 is stored in the ID storage area for the specific space (step S335). If there are multiple people identified as being in the area corresponding to the entrance / exit of the specific space, the processing unit 30 executes the following process for each person.

[0107] If the processing unit 30 determines that the tracking ID is stored in the ID storage area (S335: YES), it determines that the target person has left the specific space, and deletes the tracking ID assigned to this person from the ID storage area (step S336).The processing unit 30 uses this tracking ID as the tracking ID of the person who has left the specific space, continues tracking the person who has left (step S337), and ends the process.

[0108] If the processing unit 30 determines in step S335 that the tracking identification ID of the identified person is not stored in the identification ID storage area (S335: NO), it determines that the target person has entered the specific space (step S338). The processing unit 30 causes the ID storage unit 308 to store the tracking identification ID of the person determined to have entered the specific space in the identification ID storage area for the specific space in association with the shooting time of the captured image indicating that the person has entered (step S339). In step S339, the processing unit 30 may assign the tracking identification ID only after it has been determined that the person has entered the specific space.

[0109] Based on the tracking ID for each captured image acquired in S333, the processing unit 30 determines whether a predetermined time or more has passed since the capture time of the captured image that served as the basis for determining that the person entered the specific space, without deleting the tracking ID of the person (step S340). If it is determined that the predetermined time or more has passed (S340: YES), the processing unit 30 uses the function of the warning unit 312 to output a warning to the corresponding client 4 (step S341), waits until the warning is confirmed by the user, and then terminates the processing.

[0110] If it is determined in step S340 that the predetermined time or more has not elapsed (S340: NO), the processing unit 30 ends the processing of this flowchart and continues processing the next received detection result.

[0111] FIG. 12 is an explanatory diagram of processing in the multi-camera tracking system 100 of the second embodiment. Similar to the first embodiment, FIG. 12 is a diagrammatic representation of a captured image I1, a captured image I1′, and a captured image I1″ captured by a camera 2 installed near the entrance of a store. The captured image I1′ is captured a few minutes after the captured image I1, and the captured image I1″ is captured a few seconds after the captured image I1′. The entrance to a restroom is visible in the background of the captured images I1, I1′, and I1″, and the hatched area defines the area defined by the specific space definition unit 307. The hatched person in the captured image I1 and the captured image I1′ is the same person, and the same tracking identification ID “G002” is assigned to them. This tracking identification ID “G002” need not be assigned if it is not determined that the target person has entered the specific space. However, since there is a possibility that the target person will subsequently enter the specific space, the tracking identification ID “G002” may be stored in the tracking object DB 310.

[0112] In the captured image I1, a person assigned the tracking identification ID "G001" is detected in an area defined as an area corresponding to the specific space. Therefore, the target person is identified as being in an area corresponding to the entrance / exit of the specific space (S334). The tracking identification ID "G001" is saved in the ID saving area. In the next captured image I1', a person assigned the tracking identification ID "G001" is not detected, so unless the tracking identification ID "G001" is saved, tracking will be lost after a few minutes. In the multi-camera tracking system 100 of the second embodiment, the tracking identification ID "G001" of a person who has entered the specific space is saved in the ID saving area. If a person detected near the specific space after a while is determined to be the same person as the person identified by the tracking identification ID "G001," tracking using the tracking identification ID "G001" will be possible again. Furthermore, whether or not a person who is supposed to appear does not appear can be determined by whether or not the storage period of the tracking identification ID in the identification ID storage area is longer than a predetermined time, so that if a person who is supposed to appear does not appear, a warning can be output from the warning unit 312.

[0113] In the multi-camera tracking system 100 and the multi-camera tracking program P32 of the second embodiment, the tracking ID of a person who enters a specific space is stored in an ID storage area, and the tracking ID of a person who leaves the specific space is selected from the tracking IDs stored in the ID storage area. This makes it easy to continue tracking (multi-camera tracking) of a person who is the target of tracking even if the person enters a specific space such as a restroom or a changing room.

[0114] The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0115] 100 Multi-camera Tracking System 1. Edge Devices 10 Processing section 101 Person detection unit 11 Storage section P1 Edge side program 3. Cloud Server 30 Processing section 301 Detection result receiving unit 302 Identification ID assignment unit 303 Person identification section 304 Person Exclusion Section 305 Tracking Department 31 Storage section P32 Multi-camera tracking program

Claims

1. A multi-camera tracking system includes an ID assigning unit that assigns the same tracking ID to the same person who appears in each of the images captured by the plurality of cameras based on the images captured by the plurality of cameras, and performs multi-camera tracking, which is a process of tracking the person assigned the tracking ID by the ID assigning unit across the capture ranges of the plurality of cameras, a person determination means for determining, from among the people captured in the images captured by the plurality of cameras, a person belonging to the majority who moves in the same manner as many other people; a person excluding means for excluding a person determined by the person determining means to belong to a majority group from the people to be targeted by the multi-camera tracking; the identification ID assigning means continues the process of assigning the tracking identification ID only to persons who have not been excluded by the person excluding means among persons who appear in each of the images captured by the plurality of cameras, A multi-camera tracking system in which, when the user's area of ​​interest is close to a blind spot within the shooting range or a place where the person does not stagnate and deviates from the flow of people, the person determination means determines that, among the people captured in images from the multiple cameras, those who are outside the user's area of ​​interest are people belonging to the majority, and when the user's area of ​​interest is a place within the shooting range where people tend to pass through or where people tend to gather, the person determination means determines that, among the people captured in images from the multiple cameras, those who are within the user's area of ​​interest are people belonging to the majority.

2. 2. The multi-camera tracking system according to claim 1, wherein the person determination means determines the person belonging to the majority group based on the direction of travel of the person captured in the images captured by the plurality of cameras.

3. The multi-camera tracking system according to claim 1, characterized in that the person determination means determines the person belonging to the majority group based on the density of people in the surrounding area of ​​the person captured in the images taken by the multiple cameras.

4. A multi-camera tracking program for performing multi-camera tracking, which is a process of assigning the same tracking identification ID to the same person who appears in each of the images taken by a plurality of cameras, based on the images taken by the plurality of cameras, and tracking the person to which the tracking identification ID is assigned across the shooting ranges of the plurality of cameras, Computer, an identification ID assigning means for assigning the same tracking identification ID to the same person appearing in each of the images captured by the plurality of cameras, based on the images captured by the plurality of cameras; a person determination means for determining, from among the people captured in the images captured by the plurality of cameras, a person belonging to the majority who moves in the same manner as many other people; functioning as a person exclusion means for excluding a person determined by the person determination means to belong to the majority from the people to be targeted by the multi-camera tracking; the identification ID assigning means continues the process of assigning the tracking identification ID only to persons who have not been excluded by the person excluding means among persons who appear in each of the images captured by the plurality of cameras, A multi-camera tracking program in which, when the user's area of ​​interest is close to a blind spot within the shooting range or a place where the person does not stagnate and deviates from the flow of people, the person determination means determines that, among the people reflected in the images taken from the multiple cameras, those who are outside the user's area of ​​interest are people belonging to the majority, and when the user's area of ​​interest is a place within the shooting range where people tend to pass through or where people tend to gather, the person determination means determines that, among the people reflected in the images taken from the multiple cameras, those who are within the user's area of ​​interest are people belonging to the majority.

Images