Watching service management method and management system

Abstract

To properly operate a watching service by reliably acquiring full-body images and face images of a target to be watched, which are necessary for daily tracking of the target to be watched.SOLUTION: A system for managing a watching service that provides a user with a video including a target to be watched that is captured by using an infrastructure camera comprises a first infrastructure camera, a second infrastructure camera, and a processing device. The first and second infrastructure cameras are included in the infrastructure camera. The first infrastructure camera captures a face image of a person passing through a doorway of a building. The second infrastructure camera captures a full-body image of a person passing through the same doorway as the doorway captured by the first infrastructure camera. The processing device performs watch processing of the target to be watched using a first frame group from the first infrastructure camera and a second frame group from the second infrastructure camera.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a method and system for managing a service that provides users with video of a person being monitored that has been captured using an infrastructure camera. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2022-030846 discloses a person tracking system. This system periodically acquires surveillance images using multiple surveillance cameras and detects at least one of a facial image similar to a registered facial image of the tracking target and a full-body image similar to a registered full-body image of the tracking target. If an image similar to at least one of the registered facial image and registered full-body image of the tracking target is detected, the detected image is output.

[0003] In addition to JP 2022-030846 A, examples of documents showing the technical state of the art in the technical field related to the present disclosure include JP 2020-187167 A and JP 2007-329627 A. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-030846 [Patent Document 2] Japanese Patent Publication No. 2020-187167 [Patent Document 3] Japanese Patent Application Laid-Open No. 2007-329627 Summary of the Invention [Problem to be solved by the invention]

[0005] In a service that provides users with video of a monitored target captured using an infrastructure camera (hereinafter referred to as a "monitoring service"), a camera system including the infrastructure camera tracks the monitored target. To perform this tracking, the camera system must have a grasp of the image of the monitored target identified by face recognition.

[0006] Here, it is considered that the clothing of the monitored subject changes every day. Therefore, in order to obtain a figure image of the monitored subject, it is considered to install a camera at the entrance of the home in addition to the infrastructure camera. In this case, a figure image of the monitored subject can be acquired when the monitored subject goes out each day. However, in this case, there is a possibility that a face image cannot be acquired by the camera for capturing figure images, or that face authentication using this face image will fail. Therefore, an improvement is desired to reliably acquire a figure image, including the clothing of the monitored subject, and a face image required for face authentication.

[0007] One object of the present disclosure is to provide a technology that enables proper operation of a monitoring service by reliably acquiring images of the person's appearance and face, which are necessary for daily tracking of the person being monitored. [Means for solving the problem]

[0008] A first aspect of the present disclosure is a system for managing a monitoring service that provides a user with video that includes a target to be monitored and that is captured using an infrastructure camera, and has the following features. The system includes a first infrastructure camera, a second infrastructure camera, and a processing device. The first and second infrastructure cameras are included in the infrastructure camera. The first infrastructure camera captures a facial image of a person passing through an entrance / exit of a building. The second infrastructure camera captures a figure image of the person passing through the entrance / exit. The processing device performs a monitoring process for the monitoring target using a first group of frames from the first infrastructure camera and a second group of frames from the second infrastructure camera.

[0009] A second aspect of the present disclosure is a method for managing a monitoring service that provides a user with video that includes a target being monitored and that is captured using an infrastructure camera, and has the following features. The method includes acquiring a first group of frames from a first camera included in the infrastructure camera for capturing facial images of a person passing through an entrance / exit of a building, acquiring a second group of frames from a second camera included in the infrastructure camera for capturing facial images of a person passing through the entrance / exit, and performing monitoring processing of the person being monitored using the first and second group of frames. [Effects of the Invention]

[0010] According to the present disclosure, a first infrastructure camera captures a facial image of a person passing through a building entrance. In addition, a second infrastructure camera captures a figure image of the person passing through the building entrance. Therefore, when the person being monitored passes through the building entrance, it is possible to reliably obtain figure and facial images of the person being monitored, which are necessary for daily tracking of the person being monitored.

[0011] According to the present disclosure, a monitoring process is also performed using the first and second frame groups from the first and second infrastructure cameras. The monitoring process using the first frame group makes it possible, for example, to identify a face image of a person that matches the face image of the person being monitored. Furthermore, the monitoring process using the second frame group makes it possible, for example, to identify a face image of the person being monitored and a figure image of the person whose face image matches the face image of the person being monitored. In other words, the monitoring process using the first and second frame groups makes it possible to identify a figure image of the person being monitored. Once the figure image of the person being monitored can be identified, it becomes possible to track the person being monitored based on this figure image and frame groups from infrastructure cameras other than the first and second infrastructure cameras. Therefore, according to the present disclosure, the monitoring service can also be operated appropriately. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an overview of a monitoring service. [Figure 2]10 is a flowchart illustrating an example of information processing performed when a group of frames including an image of a monitoring target is provided to a user terminal. [Figure 3] 10A and 10B are diagrams illustrating features of the monitoring process of the monitored subject performed in the embodiment. [Figure 4] FIG. 10 is a diagram illustrating a first example in which the monitoring process is shared by two or more data processing devices. [Figure 5] FIG. 10 is a diagram illustrating a second example in which the monitoring process is shared by two or more data processing devices. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0014] 1. Monitoring service FIG. 1 is a diagram illustrating an overview of the monitoring service. The monitoring service is a service that identifies a frame group FR_CA containing an image IMG_TG of a monitoring target TG from among frame groups FR_CA acquired by multiple infrastructure cameras that make up an infrastructure camera group 20 (in the example shown in FIG. 1, a figure image (full-body image) IMG_TGS), and provides the identified frame group FR_CA to a communication terminal (hereinafter also referred to as a "user terminal") 30 of a user US. Here, the user US is a person who uses the monitoring service. The monitoring target TG is a person that the user US wishes to monitor (for example, a family member of the user US, a friend of the user US, or a person being cared for by the user US).

[0015] Fig. 1 is also a diagram showing an example of the overall configuration of a management system for a monitoring service according to an embodiment. In the example shown in Fig. 1, the management system includes a management server 10, a group of infrastructure cameras 20, a user terminal 30, and a communication terminal of a TG to be monitored (hereinafter also referred to as a "target terminal") 40. The group of infrastructure cameras 20, the user terminal 30, and the target terminal 40 communicate with the management server 10 via a communication network (not shown). The communication network is not particularly limited, and wired and wireless networks may be used.

[0016] The management server 10 includes a data processing device 11 and a database 12. The data processing device 11 includes at least one processor and at least one memory. Examples of the processor include a general-purpose processor, a specific-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an integrated circuit, and / or a combination thereof. The memory is a volatile memory such as a DDR memory, and is used to expand various programs used in various processes performed by the processor and temporarily store various data. The various data used by the processor includes data stored in the database 12.

[0017] The database 12 is formed in a predetermined storage device (for example, a hard disk or flash memory). The database 12 stores user data USR and camera data CAM. The user data USR is transmitted from the user terminal 30 to the management server 10. The user data USR includes identification information of the user US, identification information of the monitoring target TG, etc. The camera data CAM includes identification information of each of the multiple infrastructure cameras that make up the infrastructure camera group 20, location information of these infrastructure cameras, and frame groups FR_CA acquired by each of these infrastructure cameras, etc.

[0018] The infrastructure camera group 20 includes multiple infrastructure cameras. These infrastructure cameras include not only infrastructure cameras installed outdoors but also infrastructure cameras installed indoors. The shooting ranges of two or more infrastructure cameras may partially or completely overlap. Each infrastructure camera acquires a frame group FR_CA. The frame group FR_CA is a collection of still images (frames) that make up the video captured by the infrastructure camera (i.e., camera video). Each infrastructure camera also transmits the acquired frame group FR_CA to the management server 10 together with its own identification information.

[0019] The user terminal 30 is a terminal with communication capabilities, such as a smartphone, tablet, or laptop carried by the user US. The user terminal 30 is used when the user US uses the monitoring service for the first time. At the first use, an application AFU (Application For Use) for using the monitoring service is sent from the user terminal 30 to the management server 10. The application AFU includes user data USR, which includes identification information of the user US and identification information of the TG to be monitored. Once the user data USR is registered in the database 12, the monitoring service can be used.

[0020] Examples of the identification information of the user US include attribute information of the user US (e.g., name, gender, age) and identification information of the user terminal 30. Examples of the identification information of the monitored TG include a facial image IMG_TGF of the monitored TG and attribute information of the monitored TG (e.g., name, gender, age). The identification information of the monitored TG may include identification information of the target terminal 40 and relationship information between the user US and the monitored TG (e.g., family, friends, people cared for by the user US).

[0021] The user terminal 30 is also used when the user US uses the monitoring service for the second or subsequent time. When using the service for the second or subsequent time, a monitoring request RFW (Request For Watch) is sent from the user terminal 30 to the management server 10. The monitoring request RFW includes, for example, login information for the user US for accessing the database 12. The monitoring request RFW may also include information for updating some or all of the data in the user data USR registered in the database 12. When such update information is included in the monitoring request RFW, the user data USR registered in the database 12 is updated.

[0022] The target terminal 40 is a terminal with a communication function, such as a smartphone or a wearable device carried by the TG to be monitored. The target terminal 40 also has a GPS (Global Positioning System) function. By having the GPS function, the target terminal 40 transmits location information of the target terminal 40 to an external device (e.g., a management server 10 or a user terminal 30). The target terminal 40 is an arbitrary component of the management system according to the present disclosure. In other words, the management system according to the present disclosure may be composed of the management server 10, the infrastructure camera group 20, and the user terminal 30.

[0023] 2. Provision of camera footage 2 is a flowchart showing an example of information processing performed when a frame group FR_CA including an image IMG_TG is provided to the user terminal 30. The routine shown in FIG. 2 is repeatedly executed by, for example, the data processing device 11 shown in FIG.

[0024] 2, first, in the process of step S11, it is determined whether or not a monitoring request RFW has been received. As described above, the monitoring request RFW includes login information of the user US for accessing the database 12. If the determination result of step S11 is positive, the process of step S12 is performed.

[0025] In the processing of step S12, a search is performed for the TG to be monitored using the frame group FR_CA stored in the database 12. In this search, the database 12 is referenced using the login information received in the processing of step S11 as a key, and the feature FTG of the TG to be monitored that is included in the user data USR corresponding to the login information is identified.

[0026] Here, the feature FTG of the monitoring target TG is extracted based on the image IMG_TGS of the figure of the monitoring target TG. The feature FTG is used to search for the monitoring target TG. The feature FTG is also used to re-identify the monitoring target TG. The feature FTG is an example of the feature FPS of the person PS. The feature FPS is extracted, for example, by applying a group of bounding boxes representing the same person in multiple time steps to a Re-ID model based on machine learning. Note that extraction of the feature FPS itself is a well-known technique, and the extraction method applied to the processing of step S15 is not particularly limited.

[0027] Once the feature FTG is identified, a figure image having a feature that matches this feature FTG is identified. Then, a frame group FR_CA including this figure image and the infrastructure camera that acquired this frame group FR_CA are identified. The frame group FR_CA including the figure image is identified, for example, by comparing the feature FPS extracted from the frame group FR_CA acquired by each infrastructure camera with the feature FTG. For example, a frame group FR_CA including a figure image from which a feature FPS whose similarity to the feature FTG is equal to or greater than a threshold is identified as a frame group FR_CA including a figure image having a feature that matches the feature FTG. The identified frame group FR_CA is a frame group that includes a frame closest to the current time t.

[0028] The processing of step S12 is performed for a predetermined time. After the predetermined time has elapsed since the start of the processing of step S12, the processing of step S13 is performed. In the processing of step S13, it is determined whether or not the monitoring target TG has been identified. That is, it is determined whether or not a frame group FR_CA including a figure image having a feature that matches the feature FTG and the infrastructure camera that acquired this frame group FR_CA have been identified. If the determination result of step S13 is negative, the processing of step S14 is performed.

[0029] In the processing of step S14, a re-search for the TG to be monitored is performed using the frame group FR_CA stored in the database 12. The method of this re-search is basically the same as the method described in the processing of step S12. Like the processing of step S12, the processing of step S14 is performed over a predetermined period of time. However, while the processing of step S12 performs a search focusing on frames at the current time t, the processing of step S14 performs a search focusing on frames at the current time t and time tk (k≧1).

[0030] If the determination result in step S13 is positive, the processes of steps S15 and S16 are performed. In the process of step S15, the monitoring target TG is tracked. Tracking is a technology that automatically tracks the same person included in a group of frames based on a tracking algorithm. Tracking in one infrastructure camera is performed, for example, by inferring that people PS with matching feature values ​​FPS extracted from the group of frames FR_CA are the same person. Tracking in two or more infrastructure cameras is performed, for example, by comparing the feature values ​​FPS between the infrastructure cameras and inferring that people PS with matching feature values ​​FPS between the infrastructure cameras are the same person.

[0031] The tracking of the watching target TG is performed by using the feature FTG to track a person who can be estimated to be the same person as the watching target TG. By tracking the watching target TG, an image IMG_TG (figure image IMG_TGS) of the watching target TG is identified. In the processing of step S16, a frame group FR_CA including the image IMG_TG identified in this way is transmitted to the terminal that transmitted the watching request RFW (i.e., the user terminal 30).

[0032] Following the process of step S16, the process of step S17 is performed. In the process of step S17, it is determined whether or not a monitoring termination request RFT (Request For Termination) has been received. If the determination result of step S17 is positive, the transmission of the frame group FR_CA including the image IMG_TG is terminated. If not, the processes of steps S15 and S16 are performed. In other words, the processes of steps S15 to S17 are repeatedly executed until a termination request RFT is received.

[0033] 3. Features of the embodiment Tracking the monitored TG using the feature FTG requires a figure image IMG_TGS of the monitored TG to extract the feature FTG. However, the appearance (clothing) of the monitored TG changes daily. Also, if the monitored TG changes clothes during the day, their appearance changes, and even if they take off and put on their jacket, their appearance changes. Therefore, the tracking accuracy cannot be guaranteed with a figure image IMG_TGS registered in advance.

[0034] Therefore, in this embodiment, two infrastructure cameras, one for capturing a face image and one for capturing a figure image, are installed at the entrance and exit of a building to acquire the latest figure image IMG_TGS of the person TG being watched over. Then, a face recognition process is performed using the face image IMG_PSF of the person PS acquired using the face-capturing infrastructure camera and a pre-registered face image IMG_TGF to identify the person TG being watched over. Then, of the figure images IMG_PSS of the person PS acquired using the figure-capturing infrastructure camera, the figure image IMG_PSS of the person PS identified as the person TG being watched over by the face recognition process is estimated to be the figure image IMG_TGS of the person TG being watched over.

[0035] Fig. 3 is a diagram illustrating the characteristics of the monitoring process for the monitoring target TG performed in an embodiment. Fig. 3 depicts infrastructure cameras 21, 22, and 23. These infrastructure cameras are all cameras belonging to infrastructure camera group 20. Infrastructure camera 21 is an example of a "first infrastructure camera" in the present disclosure, infrastructure camera 22 is an example of a "second infrastructure camera" in the present disclosure, and infrastructure camera 23 is an example of a "third infrastructure camera" in the present disclosure.

[0036] The infrastructure cameras 21 and 22 are installed separately, for example, at entrances / exits 60 of buildings (for example, residential houses, public facilities such as schools and hospitals, and commercial facilities such as stores and offices). The infrastructure cameras 21 and 22 are installed in sets of two. The total number of sets of the infrastructure cameras 21 and 22 is one or more.

[0037] While infrastructure camera 21 is a camera for capturing images of faces, infrastructure camera 22 is a camera for capturing images of figures. For example, infrastructure camera 21 is installed at a position and height that allows it to capture an image of the vicinity of the face of a person passing through entrance / exit 60. The focal length of infrastructure camera 21 may be adjusted so as to zoom in and capture an image of the vicinity of the face of a person passing through entrance / exit 60. Infrastructure camera 22 is installed at a position and height that allows it to capture an image of the entire figure of a person passing through entrance / exit 60. Infrastructure camera 22 may use a wide-angle lens or a fisheye lens that captures the entire figure of a person passing through entrance / exit 60.

[0038] The infrastructure camera 23 is installed at a location other than the entrance / exit 60. Examples of locations other than the entrance / exit 60 include indoor structures of the building where the infrastructure cameras 21 and 22 are installed (for example, interior walls such as the walls of a corridor or a room) and outdoor structures of the building (for example, the exterior walls of structures surrounding the building). The total number of infrastructure cameras 23 installed at the same location is one or more. The configuration of the infrastructure camera 23 is the same as that of the infrastructure camera 22. In other words, the infrastructure camera 23 is a camera for capturing images.

[0039] The monitoring process includes (I) face recognition processing, (II) association processing, and (III) search processing. In (I) face recognition processing, a face image IMG_PSF extracted from a frame group FR_CA from the infrastructure camera 21 is compared with a face image IMG_TGF of the monitoring target TG that has been registered in advance. If the comparison result indicates that these face images match, the face image IMG_PSF is identified as the face image of the monitoring target TG.

[0040] (II) In the association process, a figure image IMG_PSS extracted from the frame group FR_CA22 of the infrastructure camera 22 is associated with a face image IMG_PSF extracted from the frame group FR_CA21 of the infrastructure camera 21. Since the installation positions and angles of view of the infrastructure cameras 21 and 22 are known, it is possible to identify the person PS whose figure image IMG_PSS is located on the coordinates (x, y) of the frame acquired by the infrastructure camera 22 at the time the infrastructure camera 21 acquired the face image IMG_PSF. If multiple face images IMG_PSF are acquired on the coordinates (x, y) at the same time, it is sufficient to identify the figure image IMG_PSS to be associated with the face image IMG_PSF based on the positional relationship between the infrastructure cameras 21 and 22.

[0041] (II) In the association process, a frame group FR_CA22 including a figure image IMG_PSS associated with the face image IMG_PSF and a frame group FR_CA21 including the face image IMG_PSF are recorded in combination with timestamp information, information on the position coordinates on the frame of the face image IMG_PSF, and information on the position coordinates on the frame of the figure image IMG_PSS. (II) In the association process, information on the feature FPS extracted from the figure image IMG_PSS may be further combined.

[0042] By performing (I) face recognition processing and (II) association processing, it is possible to identify the figure image IMG_PSS associated with the face image IMG_TGF of the person being watched over TG from among the figure images IMG_PSS. The identified figure image IMG_PSS can be estimated to be the latest figure image IMG_TGS of the person being watched over TG. Furthermore, the feature value FPS extracted from the identified figure image IMG_PSS can be estimated to be the latest feature value FTG of the person being watched over TG.

[0043] The (III) search process is performed when the data processing device 11 shown in FIG. 1 receives a monitoring request RFW. The (III) search process may be performed even if the monitoring request RFW has not been received. In the (III) search process, a figure image IMG_PSS is extracted from the frame group FR_CA23 of the infrastructure camera 23, and a feature value FPS is extracted from this figure image IMG_PSS. In the (III) search process, a figure image IMG_TGS is also extracted from the frame group FR_CA22 that includes the figure image IMG_TGS, and a feature value FTG is extracted from this figure image IMG_TSG. Note that if the feature value FTG is extracted in the (II) association process, the feature value FTG is not extracted in the (III) search process.

[0044] (III) In the search process, the feature FPS and the feature FTG are also compared. If a feature FPS that matches the feature FTG is detected, a frame group FR_CA including a figure image IMG_PSS having this feature FPS and the infrastructure camera 23 that acquired this frame group FR_CA are identified. If the data processing device 11 receives a watching request RFW, the frame group FR_CA23 of the infrastructure camera 23 identified in this way is transmitted to the terminal that transmitted the watching request RFW (i.e., the user terminal 30).

[0045] 4. Distributed monitoring processing 4 and 5 are diagrams illustrating an example in which the monitoring process is shared by two or more data processing devices. In the first example shown in FIG. 4, the management server 10 shown in FIG. 1 is composed of a local server 10A and a remote server 10B. The local server 10A is an example of the "first processing device" of the present disclosure, and the remote server 10B is an example of the "second processing device" of the present disclosure. The local server 10A is connected to infrastructure cameras 21 and 22. Meanwhile, the remote server 10B is connected to infrastructure camera 23. The local server 10A performs part of the monitoring process. The remote server 10B manages the entire monitoring service.

[0046] The local server 10A includes a data processing device 11A and a database 12A. An example of the configuration of the data processing device 11A is the same as that of the data processing device 11 described in FIG. 1. The data processing device 11A performs processing related to (II) association processing. That is, the data processing device 11A extracts a face image IMG_PSF from a frame group FR_CA21 stored in the database 12A, and extracts a figure image IMG_PSS from a frame group FR_CA22 stored in the database 12A. The data processing device 11A also associates the face image IMG_PSF with the face image IMG_PSF. The frame groups FR_CA21&CA22 shown in the database 12A indicate the data set of the frame groups after association.

[0047] The remote server 10B includes a data processing device 11B and a database 12B. An example of the configuration of the data processing device 11B is the same as that of the data processing device 11 described in FIG. 1. The data processing device 11B performs (I) processing related to face recognition processing and (III) processing related to search processing. That is, in the processing related to face recognition processing (I), a frame group FR_CA21&CA22 is received from the local server 10A. Furthermore, a face image IMG_PSF included in the frame group FR_CA21 is identified from the frame group FR_CA21&CA22 and information on the position coordinates of the face image IMG_PSF on the frame. Then, the identified face image IMG_PSF is compared with a face image IMG_TGF included in the user data USR, and a face image IMG_PSF that matches the face image IMG_TGF is identified.

[0048] When a face image IMG_PSF that matches the face image IMG_TGF is identified, processing related to (III) search processing is performed. That is, the data processing device 11B determines whether or not a monitoring request RFW has been received, and if the determination result is positive, the (III) search processing is performed. Alternatively, the data processing device 11B performs the (III) search processing without determining whether or not a monitoring request RFW has been received. In the (III) search processing, the face image IMG_PSF is extracted from the frame group FR_CA23 included in the camera data CAM, and a feature amount FPS is extracted from this face image IMG_PSF.

[0049] (III) In the search process, the figure image IMG_PSS associated with the face image IMG_PSF that matches the face image IMG_TGF is deemed to be the figure image IMG_TGS of the monitoring target TG, and the figure image IMG_PSS deemed to be the figure image IMG_TGS is identified based on the frame groups FR_CA21&CA22 and the information on the position coordinates of the figure image IMG_PSS on the frames. Then, a feature FTG is extracted from the identified figure image IMG_PSS and compared with the feature FPS extracted from the frame group FR_CA23. If the comparison results in a feature FPS that matches the feature FTG, the frame group FR_CA23 containing the figure image IMG_PSS having this feature FPS and the infrastructure camera 23 that acquired this frame group FR_CA23 are identified.

[0050] As described above, in the first example, the (II) association process is performed in the local server 10A. Therefore, the processing load on the remote server 10B can be reduced compared to when the (II) association process is performed in the remote server 10B. Furthermore, when the frame groups FR_CA21 and FR_CA22 are transmitted separately to the remote server 10B, failure to transmit one of the frame groups may cause problems in the (II) association process or the (III) search process. In this regard, according to the first example, the frame groups FR_CA21&CA22 are transmitted to the remote server 10B, so that such problems can be prevented.

[0051] As in the first example, in the second example shown in Fig. 5, the management server 10 shown in Fig. 1 is composed of a local server 10A and a remote server 10B. Unlike the first example, in the second example, the data processing device 11A performs processing related to (I) face authentication processing in addition to processing related to (II) association processing, and the data processing device 11B performs only processing related to (III) search processing.

[0052] (I) When the data processing device 11A performs processing related to face recognition processing, the face image IMG_TGF of the watching target TG needs to be acquired by the data processing device 11A. Therefore, it is required to store the face image IMG_TGF in the database 12B in advance, or (I) acquire the face image IMG_TGF from the database 12A each time face recognition processing is performed.

[0053] However, in the second example, (I) the face recognition process and (II) the association process allow the local server 10A to identify the figure image IMG_PSS associated with the face image IMG_PSF that matches the face image IMG_TGF. Therefore, instead of the frame group FR_CA21&CA22, it is possible to transmit to the remote server 10B only the frame group FR_CA22 that includes the figure image IMG_PSS associated with the face image IMG_PSF that matches the face image IMG_TGF. This leads to a reduction in the amount of communication between the local server 10A and the remote server 10B, which can be said to be an advantage of the second example. [Explanation of symbols]

[0054] 10... management server, 10A... local server, 10B... remote server, 11, 11A, 11B... data processing device, 12, 12A, 12B... database, 20... infrastructure camera group, 21, 22, 23... infrastructure camera, 30... user terminal, 40... target terminal, TG... monitoring target, US... user of monitoring service, AFU... application for use, CAM... camera data, FPS, FTG... feature amount, RFW... monitoring request, RFT... monitoring end request, FR_CA, FR_CA21, FR_CA22, FR_CA23... frame group, IMG_PSF... face image of person PS, IMG_PSS... figure image of person PS, IMG_TGF... face image of monitoring target TG, IMG_TGS... figure image of monitoring target TG

Claims

1. A system for managing a monitoring service that provides a user with a video including a monitoring target captured using an infrastructure camera, a first infrastructure camera included in the infrastructure camera and configured to capture a facial image of a person passing through an entrance / exit of a building; a second infrastructure camera included in the infrastructure camera and configured to capture an image of a person passing through the entrance; a processing device that performs a monitoring process for the target being monitored using a first frame group from the first infrastructure camera and a second frame group from the second infrastructure camera; A management system for a monitoring service comprising:

2. 10. The system of claim 1, a third infrastructure camera included in the infrastructure camera and configured to acquire a third frame group; The monitoring process includes: a process of associating a face image of a person extracted from the first frame group with a figure image of a person extracted from the second frame group; a face recognition process using a face image of a person extracted from the first frame group, the face recognition process identifying a face image of a person that matches the face image of the monitoring target; a search process in which a figure image associated with a face image that matches the face image of the target being watched over is estimated as the figure image of the target being watched over using the results of the association process and the face authentication process, and the figure image is compared with the figure image of the person extracted from the third frame group to search for the target being watched over; A management system for a monitoring service comprising:

3. 3. The system of claim 2, The processing devices include a first processing device connected to the first and second infrastructure cameras and performing a part of the monitoring process, and a second processing device connected to the first processing device and the third infrastructure camera and managing the entire monitoring service, The monitoring process performed by the first processing device includes the association process, The monitoring process performed by the second processing device includes the face authentication process and the search process. A monitoring service management system characterized by:

4. 3. The system of claim 2, The processing devices include a first processing device connected to the first and second infrastructure cameras and performing a part of the monitoring process, and a second processing device connected to the first processing device and the third infrastructure camera and managing the entire monitoring service, the monitoring process performed by the first processing device includes the association process and the face authentication process, The monitoring process performed by the second processing device includes the search process. A monitoring service management system characterized by:

5. A method for managing a monitoring service that provides a user with a video including a monitoring target captured using an infrastructure camera, comprising: acquiring a first group of frames from a first infrastructure camera included in the infrastructure camera and configured to capture a face image of a person passing through an entrance / exit of a building; acquiring a second frame group from a second infrastructure camera included in the infrastructure camera and configured to capture an image of a person passing through the entrance; performing a monitoring process for the monitoring target using the first and second frame groups; A monitoring service management method comprising:

Images