Monitoring system

Abstract

A monitoring device comprises: an acquisition unit that acquires area identification information enabling identification of a target area for monitoring a tracking target, and acquires appearance information of the tracking target; and a processing unit that, when the appearance information of the tracking target has been acquired, causes an image processing means to identify the tracking target in the target area by using a captured image of the target area identified by using the area identification information and the appearance information of the tracking target, and executes predetermined processing if the tracking target satisfies a predetermined condition.

Description

surveillance system

[0001] The present invention relates to a surveillance system.

[0002] An example of a monitoring system is described in Patent Document 1. In the monitoring system described in Patent Document 1, when a person determination unit determines that facial image information of a person included in surrounding image information generated by capturing an image of the surroundings of a person being watched over with an imaging unit does not match facial image information of a pre-registered person who does not need to be watched over, the system issues a warning to the person being watched over and those around them, and also generates warning information including current location information and surrounding image information of the person being watched over and transmits it to a monitoring server via a network.

[0003] Japanese Patent Application Laid-Open No. 2021-114228

[0004] In the monitoring system described in Patent Document 1, facial image information of people who do not need to be watched is pre-registered in the system, but if such monitoring is not necessary, it is desirable from the perspective of protecting personal information that such facial image information not be registered in the system.

[0005] A monitoring device according to the present disclosure comprises: an acquisition means for acquiring area identification information capable of identifying a target area in which a tracking target is to be monitored, and acquiring appearance information of the tracking target; and a processing means for, upon acquiring the appearance information of the tracking target, causing an image processing means to identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target, and for executing a predetermined process if the tracking target satisfies a predetermined condition.

[0006] A first program in the present disclosure causes a computer realizing a communication device to execute a storage process for storing appearance information of a tracked target in a storage means, a location information acquisition process for acquiring location information of the communication device, and an information provision process for providing the appearance information of the tracked target to a monitoring device that performs monitoring processing of a target area for monitoring the tracked target, if the location information of the communication device satisfies a predetermined criterion.

[0007] The monitoring system of the present disclosure comprises a communication device used by a user who is a tracking target, a camera that captures images of a target area in which the tracking target is to be monitored, and a monitoring device that monitors the tracking target using the captured images of the target area generated by the camera, wherein the monitoring device comprises: an acquisition means that acquires area identification information that can identify the target area and acquires appearance information of the tracking target; and a processing means that, upon acquiring the appearance information of the tracking target, causes an image processing means to identify the tracking target within the target area using the captured images of the target area identified using the area identification information and the appearance information of the tracking target, and executes a predetermined process if the tracking target satisfies a predetermined condition.

[0008] The monitoring method in the present disclosure includes having one or more computers acquire area identification information capable of identifying a target area in which a tracking target is to be monitored, and acquire appearance information of the tracking target; upon acquiring the appearance information of the tracking target, having an image processing means identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target; and if the tracking target satisfies a predetermined condition, executing a predetermined process.

[0009] A second program in the present disclosure causes a computer to execute an acquisition process of acquiring area identification information capable of identifying a target area in which a tracking target is to be monitored and acquiring appearance information of the tracking target, and an execution process of, upon acquiring the appearance information of the tracking target, causing an image processing means to identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target, and to execute a predetermined process if the tracking target satisfies a predetermined condition.

[0010] The communication device in the present disclosure includes a storage means for storing appearance information of a tracking target in a storage means, a location information acquisition means for acquiring location information of the communication device, and an information providing means for providing the appearance information of the tracking target to a monitoring device that performs monitoring processing of a target area for monitoring the tracking target when the location information of the communication device satisfies a predetermined criterion.

[0011] The control method for a communication device in the present disclosure includes one or more computers storing appearance information of a tracking target in a storage means, acquiring location information of the communication device, and, if the location information of the communication device satisfies a predetermined criterion, providing the appearance information of the tracking target to a monitoring device that performs monitoring processing of a target area for monitoring the tracking target.

[0012] According to the present disclosure, it is possible to appropriately handle personal information used during monitoring.

[0013] FIG. 1 is a block diagram showing an example configuration of a monitoring device according to the present disclosure. FIG. 2 is a flowchart showing an example processing operation of a monitoring device according to the present disclosure. FIG. 3 is a block diagram showing an example configuration of a communication device according to the present disclosure. FIG. 4 is a flowchart showing an example processing operation of a communication device according to the present disclosure. FIG. 5 is a conceptual diagram showing an example configuration of a monitoring system according to the present disclosure. FIG. 6 is a block diagram showing an example configuration of a computer realizing each device in the present disclosure. FIG. 7 is a diagram showing an example data structure of appearance information of a communication device according to the present disclosure. FIG. 8 is a diagram showing an example data structure of appearance information of a monitoring device according to the present disclosure. FIG. 9 is a diagram showing an example data structure of itinerary information of a monitoring device according to the present disclosure. FIG. 10 is a diagram showing an example data structure of camera information of a monitoring device according to the present disclosure. FIG. 11 is a flowchart showing an example processing operation of a monitoring system according to the present disclosure. FIG. 12 is a block diagram showing an example configuration of a communication device according to the present disclosure. FIG. 13 is a flowchart showing an example processing operation of a monitoring system according to the present disclosure. FIG. 14 is a block diagram showing an example configuration of a communication device according to the present disclosure. FIG. 15 is a flowchart showing an example processing operation of a monitoring system according to the present disclosure. FIG. 16 is a block diagram showing an example configuration of a monitoring device according to the present disclosure. FIG. 17 is a diagram showing an example data structure of group information of a monitoring device according to the present disclosure. FIG. 18 is a flowchart showing a detailed example of processing operation of a monitoring device according to the present disclosure. FIG. 19 is a flowchart showing a detailed example of processing operation of a monitoring device according to the present disclosure. 1 is a flowchart illustrating a detailed example of a processing operation of a monitoring device according to the present disclosure;

[0014] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are denoted by similar reference numerals, and descriptions thereof will be omitted as appropriate. In each of the following drawings, configurations of parts that are not related to the essence of this disclosure are omitted and are not shown.

[0015] In the present disclosure, "acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition); and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a response, and accessing and reading information from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0016] <Example of Functional Configuration> First, the monitoring device 200 will be described. As shown in Fig. 1, the monitoring device 200 includes an acquisition unit 202 and a processing unit 204. The acquisition unit 202 acquires area identification information capable of identifying a target area in which a tracking target is to be monitored, and also acquires appearance information of the tracking target. Upon acquiring the appearance information of the tracking target, the processing unit 204 causes image processing means (for example, an image processing device 300 described below) to identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target, and if the tracking target satisfies a predetermined condition, executes a predetermined process.

[0017] The tracking target is, for example, at least one of a person and an object (such as the person's possessions). The predetermined condition is, for example, a condition that can identify that the tracking target may be involved in a crime. In this case, the predetermined process that is executed when the condition is met is, for example, outputting an alert.

[0018] <Operation Example> The processing operation of the monitoring device 200 will be described below with reference to Fig. 2. The acquisition unit 202 acquires area identification information capable of identifying a target area in which a tracking target is to be monitored (step S201). Then, upon acquiring appearance information of the tracking target (step S203), the processing unit 204 causes the image processing means to identify the tracking target within the target area using the captured image of the target area identified using the area identification information and the appearance information of the tracking target (step S205). Then, if the tracking target satisfies a predetermined condition (YES in step S207), the processing unit 204 executes a predetermined process (step S209).

[0019] This monitoring device 200 acquires, via the acquisition unit 202, area identification information capable of identifying a target area in which a tracking target is to be monitored, and also acquires appearance information of the tracking target. Then, upon acquiring the appearance information of the tracking target, the processing unit 204 causes the image processing means to identify the tracking target within the target area using the captured image of the identified target area and the appearance information of the tracking target. Then, if the tracking target satisfies a predetermined condition, the processing unit 204 executes a predetermined process. Thus, according to the monitoring device 200, upon acquiring the appearance information of the tracking target, which is personal information used for monitoring, a monitoring process using the appearance information is initiated. Therefore, since the monitoring device 200 does not retain the appearance information of the tracking target, which is personal information, before monitoring, it is possible to appropriately handle the personal information used during monitoring.

[0020] <Example of Functional Configuration> Next, the communication device 100 will be described. As shown in Fig. 3, the communication device 100 includes an appearance information storage processing unit 102, a position information acquisition unit 104, and an information providing unit 106. The appearance information storage processing unit 102 stores appearance information of the tracking target in the storage device 120. The position information acquisition unit 104 acquires position information of the communication device 100. If the position information of the communication device 100 satisfies a predetermined criterion, the information providing unit 106 provides the appearance information of the tracking target to a monitoring device 200 that performs monitoring processing of a target area for monitoring the tracking target.

[0021] <Operation Example> The processing operation of the communication device 100 will be described below with reference to Fig. 4. The appearance information storage processing unit 102 stores appearance information of the tracking target in the storage device 120 (step S101). The position information acquisition unit 104 acquires position information of the communication device 100 (step S103). If the position information of the communication device 100 satisfies a predetermined criterion (YES in step S105), the information providing unit 106 provides the appearance information of the tracking target to the monitoring device 200 that performs monitoring processing of the target area for monitoring the tracking target (step S107). If the position information of the communication device 100 does not satisfy the predetermined criterion (NO in step S105), the process returns to step S103.

[0022] In this communication device 100, the appearance information storage processing unit 102 stores the appearance information of the tracking target in the storage device 120, and when the acquired position information of the communication device 100 satisfies a predetermined criterion, the information providing unit 106 provides the appearance information of the tracking target to the monitoring device 200. In this way, according to the communication device 100, when the position of the communication device 100 satisfies the predetermined criterion, the appearance information, which is personal information of the tracking target, is provided to the monitoring device 200. In other words, when the position of the communication device 100 does not satisfy the predetermined criterion, the appearance information, which is personal information of the tracking target, is not provided to the monitoring device 200. Therefore, according to the communication device 100, it is possible to appropriately handle personal information used during monitoring.

[0023] A detailed example of the monitoring system 1 including the monitoring device 200 and the communication device 100 will be described below.

[0024] 5, the monitoring system 1 includes a communication device 100, a camera 5, and a monitoring device 200. The monitoring device 200 and the communication device 100 are connected via a communication network 3.

[0025] The communication device 100 is a device used by a user (hereinafter also referred to as user U) who is the tracking target. The camera 5 captures an image of a target area 30 in which the tracking target is monitored. The monitoring device 200 monitors the tracking target using the captured image of the target area 30 generated by the camera 5. Furthermore, the monitoring system 1 includes an image processing device 300.

[0026] However, the image processing device 300 may be included in the monitoring device 200. In other words, the image processing device 300 may be provided inside or outside the monitoring device 200. In other words, the image processing device 300 may be hardware that is integrated with the monitoring device 200, or may be hardware that is separate from the monitoring device 200. For example, the image processing device 300 may be included in the second server device 500 of Fig. 25 in an embodiment described later.

[0027] The monitoring system 1 monitors the person to be tracked and the belongings 10 that the person possesses in a target area 30 that the person to be tracked visits, using images captured by a camera 5, etc.

[0028] The monitoring device 200 is a computer such as a server computer or a personal computer. The monitoring device 200 may be a cloud server. The monitoring device 200 includes a storage device 220. The storage device 220 may be provided inside the monitoring device 200 or may be provided externally. In other words, the storage device 220 may be hardware that is integrated with the monitoring device 200, or may be hardware that is separate from the monitoring device 200.

[0029] The monitoring device 200 is further connected to at least one camera 5 installed in the target area 30 via a communication network 3. The camera 5 includes an imaging element such as a lens and a charge-coupled device (CCD) image sensor, and is, for example, a network camera such as an Internet Protocol (IP) camera. The network camera has, for example, a wireless local area network (LAN) communication function and is connected to the monitoring device 200 via the communication network 3, i.e., a relay device (not shown) such as a router. These cameras 5 may be so-called surveillance cameras installed in towns, stores, facilities, etc. The camera 5 may also include a mechanism for controlling the orientation of the camera body and lens, zooming, focusing, etc., in accordance with the movement of the tracking target.

[0030] In the figure, the camera 5 and the monitoring device 200 are directly connected. However, as will be described in the embodiment below, the camera 5 may be connected to a computer (e.g., a second server device 500 in FIG. 25 ) separate from the monitoring device 200, and images generated by the camera 5 may be transmitted from that computer to the monitoring device 200. Alternatively, the camera 5 may be directly connected to the image processing device 300. Then, in accordance with instructions from the monitoring device 200, the image processing device 300 may acquire images generated by the camera 5 and perform analysis processing.

[0031] The images generated by the camera 5 are preferably taken in real time and transmitted to the monitoring device 200. However, the images transmitted to the monitoring device 200 do not have to be transmitted directly from the camera 5, but may be images delayed by a predetermined time. The images taken by the camera 5 may be temporarily stored in another storage device, and the monitoring device 200 may read them out from the storage device sequentially or at predetermined intervals. Furthermore, the images transmitted to the monitoring device 200 are preferably moving images, but may also be frame images taken at predetermined intervals or still images.

[0032] The communication device 100 is a device used by the person (user U) to be tracked, and is, for example, a computer such as a smartphone, tablet terminal, or personal computer. The user U performs user registration in advance to use the monitoring service provided by the monitoring system 1. Specifically, first, a predetermined application that enables the user U to use the service provided by the monitoring system 1 is installed and launched on the communication device 100 used by the user U. Alternatively, the user U uses the communication device 100 to access a predetermined website that provides the service of the monitoring system 1 via a browser.

[0033] Next, the user U creates an account on the application or website and performs the user registration procedure. After user registration, when using a service, the user U uses the account on the application or website to log in to the monitoring system 1 using the communication device 100. This allows the user U to use the services provided by the monitoring system 1.

[0034] When the monitoring system 1 is used via a website, the pre-registration of the user U's appearance information, which will be described later, is performed by storing the appearance information in a predetermined area of ​​the storage device 120 of the communication device 100 of the user U, rather than uploading the appearance information to the website. For example, the website may output guidance on how to register the appearance information in the communication device 100.

[0035] <Hardware Configuration Example> The hardware configuration of a computer 1000 that realizes the monitoring device 200 and communication device 100 according to the present disclosure will be described with reference to Fig. 6. The image processing device 300 is also realized by the computer 1000. Furthermore, a first server device 400 and a second server device 500, which will be described later, are also realized by the computer 1000.

[0036] The computer 1000 has a bus 1010 , a processor 1020 , a memory 1030 , a storage device 1040 , an input / output interface 1050 , and a network interface 1060 .

[0037] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0038] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0039] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0040] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the monitoring device 200 (e.g., the acquisition unit 202, the processing unit 204, the storage processing unit 206 described later, the group registration unit 208, etc.) and each function of the communication device 100 (e.g., the appearance information storage processing unit 102, the location information acquisition unit 104, the information providing unit 106, the deletion request unit 108 described later, the reception unit 110, etc.). The processor 1020 loads each of these program modules into the memory 1030 and executes them to realize each function corresponding to the program module. The storage device 1040 may also store each data of the storage device 220 of the monitoring device 200 and the storage device 120 of the communication device 100.

[0041] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded may include a non-transitory, tangible medium usable by the computer 1000, and the program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

[0042] The input / output interface 1050 is an interface for connecting the computer 1000 with various input / output devices. The input / output interface 1050 also functions as a communication interface for performing short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication).

[0043] The network interface 1060 is an interface for connecting the computer 1000 to the communication network 3. This communication network 3 is, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 may be connected to the communication network 3 wirelessly or by wire.

[0044] The computer 1000 is connected to necessary devices (e.g., a display, touch panel, operation buttons, touch pad, keyboard, mouse, speaker, microphone, camera, printer, etc. of the monitoring device 200, and a display, touch panel, touch pad, operation buttons, keyboard, mouse, speaker, microphone, camera, printer, etc. of the communication device 100) via the input / output interface 1050 or the network interface 1060. The display is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

[0045] Each component of the monitoring device 200 and communication device 100 according to the present disclosure in each figure is realized by any combination of hardware and software of the computer 1000 in Figure 6. Those skilled in the art will understand that there are various variations in the realization methods and devices. The functional block diagrams showing the monitoring device 200 and communication device 100 according to the present disclosure in each figure show blocks of logical functional units, rather than configurations in hardware units.

[0046] <Example of Functional Configuration> The following describes the details of the monitoring device 200 and the communication device 100. First, the communication device 100 will be described. As shown in FIG. 3 , the communication device 100 includes an appearance information storage processing unit 102, a position information acquisition unit 104, and an information providing unit 106.

[0047] The appearance information storage processing unit 102 stores the appearance information of the tracked object in the storage device 120 , or in the memory 1030 or storage device 1040 of the computer 1000 that implements the communication device 100 .

[0048] 7 is a diagram showing an example data structure of appearance information 130 of communication device 100. Appearance information 130 is feature information about the appearance of a person (user U) who is the tracking target. As an example, the appearance information is biometric information of user U. The biometric information is, for example, at least one feature among facial feature information (e.g., a facial image or features extracted from an image), physical feature information (e.g., images of the body taken from the front, back, and left and right sides or features extracted from an image), iris, veins, auricles, fingerprints, gait, and physique (height, shoulder width, body length, bone structure), etc.

[0049] Furthermore, the appearance information is characteristic information of the object to be tracked. The object is characteristic information (for example, an image of the possession 10 or a feature extracted from the image) of the person (user U) to be tracked. The possession 10 includes personal belongings (bags, luggage, umbrellas, walking sticks, etc.), clothing (shape, color, material, etc.), hairstyle (including hair color), ornaments (hat, glasses, accessories, etc.), etc.

[0050] 7, the appearance information 130 is stored in association with identification information (shown as a user ID in the figure) that can identify the user U, but this is not necessarily required. However, when the appearance information 130 is transmitted to the monitoring device 200, the appearance information may be linked to the user ID and transmitted.

[0051] The appearance information 130 may be image data of a captured image of the tracking target, or may be feature amounts extracted from the image data using a general-purpose algorithm.

[0052] The location information acquisition unit 104 acquires location information of the communication device 100. The location information acquisition unit 104 acquires the current location information of the communication device 100 from, for example, a GPS (Global Positioning System) receiving function that the communication device 100 has.

[0053] However, the method of acquiring the location of the communication device 100 is not limited to this, and various methods are conceivable, such as acquiring the IP address or the like of the access point accessed by the communication device 100 during wireless communication as location information and specifying the location. Furthermore, the location information acquisition unit 104 may acquire the location information of the property 10 to be tracked by wireless communication from a device (such as a tag) that has a GPS receiving function and a location information transmitting function and is attached to the property 10 of the user U.

[0054] If the location information of the communication device 100 satisfies a predetermined standard, the information providing unit 106 provides the appearance information of the tracking target to the monitoring device 200 that performs monitoring processing of the target area 30 that monitors the tracking target.

[0055] As an example of the predetermined criterion, the location information of communication device 100 indicates that the user has passed through a departure gate at an airport. Alternatively, the predetermined criterion is that the location information of communication device 100 indicates that the user has passed through an entry gate at a visited airport. In this example, the location information may be, in addition to the GPS location information described above, information indicating that the user has passed through a gate such as a boarding gate when using communication device 100 for ticketless check-in.

[0056] The predetermined criterion may also be that the tracking target (user U) has reached the arrival time (entry time) in the target area 30 indicated in the travel itinerary information of the tracking target (user U). Alternatively, the predetermined criterion may be that the location information of the communication device 100 of the user U indicates that the user U has entered the target area 30.

[0057] As another example, the predetermined criterion is that the location information of the communication device 100 is included in the target area 30. The target area 30 is an area where tracking targets (including the user U) are monitored, and is, for example, a predetermined area such as a tourist spot that the user U visits.

[0058] According to this configuration, appearance information is not transmitted to the monitoring device 200 until the user U visits the target area 30. In other words, appearance information is not transmitted to the monitoring device 200 until monitoring is started by the image processing device 300. Therefore, according to this configuration, it is possible to appropriately handle personal information used during monitoring.

[0059] In the monitoring system 1, the ranges of the target areas 30 can be set in various ways, such as by country, state, city, town, village, region, or facility. For example, if a city in a country has multiple tourist destinations, a target area 30 may be set for each tourist destination. For example, in Paris, France, a target area 30 may be set for each district, such as the Saint-Germain-des-Prés district, the Opera district, the Invalides district, the Champs-Élysées district, and the Île de la Cité / Île Saint-Louis district. Alternatively, two regions, the left bank region and the right bank region, with the Seine River as the border, may each be set as a target area 30.

[0060] In the monitoring system 1, at least one camera 5 is installed in each target area 30. In the monitoring system 1, the target areas 30 and the cameras 5 installed in the target areas 30 are associated with each other and stored in advance, as will be described later.

[0061] The communication device 100 stores information indicating the range of each target area 30 in advance in the storage device 120. The information providing unit 106 determines whether the location indicated by the current location information of the communication device 100 is within the range of each target area 30. For example, the information providing unit 106 may preferentially select a target area 30 that includes a place that the user U plans to go to at a certain time based on information indicating a travel itinerary that the user U has registered in advance, and perform a determination process for the target area 30.

[0062] Furthermore, the information providing unit 106 provides the appearance information 130 of the tracked target to the monitoring device 200 on the condition that the information is provided for a predetermined period. The predetermined period is, for example, the travel period of the user U. Details of this configuration will be described later. For example, the information providing unit 106 determines whether the location information of the communication device 100 satisfies a predetermined criterion if the location information is for the travel period of the user U. In an embodiment described later, the information providing unit 106 can determine whether the location information is for the predetermined period, i.e., the travel period, by acquiring travel itinerary information of the user U using the receiving unit 110.

[0063] However, the predetermined period is not limited to this. For example, the predetermined period may be the period from the time when the traveler passes through the departure gate at the airport until the time when the traveler passes through the departure gate at the airport when returning home from the travel destination.

[0064] Next, a description will be given of the monitoring device 200. As shown in Fig. 1, the monitoring device 200 includes an acquisition unit 202 and a processing unit 204. The acquisition unit 202 acquires area identification information capable of identifying the target area 30 in which the tracking target is to be monitored.

[0065] A tracking target is, for example, at least one of a person and an object (such as the person's possessions 10) to be monitored within a certain area. In the present disclosure, "monitoring" means tracking the footsteps of the tracking target by analyzing images of the tracking target, and monitoring the tracking target to prevent the tracking target from becoming involved in a crime, getting lost, or being lost, or detecting when the tracking target has fallen into any of these situations. Crimes to be monitored are, for example, crimes that can be estimated from image analysis or location information, and are at least one of theft, baggage snatching, pickpocketing, kidnapping, robbery, murder, and violence.

[0066] The area specifying information that can specify the target area 30 is at least one of location information that indicates the location of the tracking target and information that indicates the itinerary of the tracking target (e.g., travel itinerary information). The location information that indicates the location of the tracking target is, for example, location information obtained using a GPS receiving function or the like of the communication device 100 used by the person (user U) who is the tracking target.

[0067] However, there are various possible methods for acquiring the location information of a tracking target, including but not limited to the following examples: (Example 1) Identifying using a communication record between an airport gate and the communication device 100. (Example 2) Identifying using a communication record between a wireless communication access point and the communication device 100. (Example 3) Identifying using a communication record between a mobile communication network base station and the communication device 100. (Example 4) Acquiring the location information of a tracking target from a device (such as a tag) that has a GPS receiving function and a location information transmitting function and is attached to the object to be tracked.

[0068] The information indicating the itinerary of the tracking target is, for example, information indicating the travel schedule of the person (user U) to be tracked (travel itinerary information 240 described later), and information indicating when and where the person will go. This information may be received by the person to be tracked themselves inputting it via the communication device 100, or may be acquired from a computer (first server device 400 described later) of a travel agency or the like that plans or sells the tour in which the person to be tracked will participate. In the latter case, if multiple people participating in the same tour are to be tracked, the monitoring device 200 may acquire information on the multiple people collectively. The acquired information can be used in a configuration in which people are registered as a group that acts together in the embodiment described later.

[0069] 9, the travel itinerary information 240 stores information indicating the travel itinerary, such as information indicating the date and time (arrival time, departure time, length of stay, etc.) and location for each destination, in association with identification information (shown as user ID in the figure) that can identify user U. Furthermore, the travel itinerary information 240 may include information indicating the purpose and content of the visit to the location (sightseeing, sightseeing, experience, meal, shopping, stroll, etc.), information indicating the means of transportation to the location (walking, taxi, public transportation, etc.), etc.

[0070] The travel itinerary information 240 is stored in the storage device 220, or in the memory 1030 or storage device 1040 of the computer 1000 that implements the monitoring device 200. Note that the various pieces of information described below may also be stored in the storage device 220, or in the memory 1030 or storage device 1040 of the computer 1000. In the following description, "stored in the storage device 220" means "stored in the storage device 220, or in the memory 1030 or storage device 1040 of the computer 1000."

[0071] The acquisition unit 202 acquires appearance information 130 of the tracking target from the communication device 100 (first communication device) used by the user (user U) who is the tracking target. The acquisition unit 202 acquires the above-mentioned area specification information in advance. Meanwhile, the acquisition unit 202 waits for the appearance information 130 to be transmitted from the communication device 100. In other words, the monitoring device 200 does not acquire the appearance information in advance. When the appearance information 130 is transmitted from the communication device 100, the acquisition unit 202 receives the appearance information 130.

[0072] 8, the appearance information 130 acquired from the communication device 100 is temporarily stored in the storage device 220 as appearance information 230. The appearance information 230 stores the appearance information 130 received from each user U in association with information indicating the date and time of reception (or the date and time the appearance information was recorded) and identification information (indicated as a user ID in the figure) that can identify the user U.

[0073] When the processing unit 204 acquires the appearance information 130 of the tracking target, it acquires from the camera 5 a captured image of the target area 30 identified using the area identification information. In other words, after acquiring the area identification information, the processing unit 204 does not start processing until it acquires the appearance information 130 of the tracking target. After acquiring the appearance information 130 of the tracking target, the processing unit 204 acquires from the camera 5 a captured image of the target area 30, which is a monitoring target area of ​​the tracking target. The processing unit 204 identifies the camera 5 and the time corresponding to the target area 30 identified in advance using the area identification information. The processing unit 204 acquires a captured image at the identified time from the identified camera 5.

[0074] 10 , for each target area 30, identification information (shown as a camera ID in the figure) that can identify at least one camera 5 installed in the target area 30 is associated with the target area 30 and stored in the storage device 220 as camera information 250. The camera ID may be, for example, the IP address of the camera 5.

[0075] The processing unit 204 acquires images from the cameras 5 associated with the target area 30. Then, the processing unit 204 causes the image processing device 300 (image processing means) to perform a matching process using the acquired captured images of the target area 30 and the appearance information 130 of the tracking target. Through the matching process, the image processing device 300 searches for, finds, and tracks the tracking target within the captured images in the target area 30. Furthermore, during the tracking process, the image processing device 300 may acquire captured images from multiple cameras 5 along the movement route and perform image analysis processing.

[0076] Specifically, first, the image processing device 300 detects at least one of a person and an object present in the captured image of the target area 30. Then, the image processing device 300 compares the feature amount extracted from the area of ​​the detected person or object with the feature amount indicated by the appearance information of the tracking target registered in the appearance information 230. If the comparison results in a similarity equal to or greater than a threshold, for example, the image processing device 300 identifies the person or object in the captured image as the tracking target.

[0077] The images to be processed are preferably real-time video data (a series of images) capturing the target area 30, but may also be data delayed by a predetermined period. The image processing device 300 tracks the identified tracking target within the series of images using the tracking target appearance information 130.

[0078] If the tracking target satisfies a predetermined condition, the processing unit 204 executes a predetermined process. One example of the predetermined process is a process of outputting an alert. A specific example of the predetermined process will be described later.

[0079] The predetermined condition, for example, is at least one of indicating that the location information of the tracking target has moved outside the reference area and being unable to detect the location information of the tracking target. For example, the reference area is determined for each date and time based on the travel itinerary information 240. In other words, the destination of the tracking target for each date and time is set as the reference area based on the travel itinerary information 240 of the tracking target user U. Furthermore, "being unable to detect the location information of the tracking target" includes being unable to receive location information transmitted from the tracking target and being unable to identify the tracking target through image recognition processing from captured images within the target area 30 or the reference area.

[0080] As another example, the predetermined condition may include that the tracking target has encountered at least one of theft, pickpocketing, kidnapping, robbery, violence, and getting lost. Whether or not the tracking target has encountered a crime or gotten lost can be determined using images captured in the target area 30. Details of the specific determination method and the processing content based on the determination result will be described later.

[0081] Furthermore, at least one of the process of identifying the tracking target and the process of determining whether or not a predetermined condition is satisfied is executed using a method defined for each target area 30 .

[0082] Laws and regulations regarding the handling of personal information vary depending on the region, such as a country, state, city, town, or village. Therefore, the above-mentioned processing using the appearance information of the user U or the property 10 must comply with the laws and regulations of the region. Therefore, the monitoring device 200 predetermines a method (processing content and predetermined conditions) that complies with the laws and regulations of the region for each target area 30 and stores the method in the storage device 220.

[0083] The processing unit 204 reads out the processing content and predetermined conditions corresponding to the identified target area 30 from the storage device 220 and executes the corresponding processing.

[0084] For example, if the target area 30 includes an area where the use of face recognition technology is prohibited, the processing unit 204 does not perform the process of identifying and tracking the user U using facial feature information in the target area 30. The processing unit 204 performs the process of identifying and tracking the user U using feature information such as clothing and belongings.

[0085] This configuration enables the monitoring system 1 to appropriately handle personal information used during monitoring.

[0086] <Operation Example> An example of the processing operation of the monitoring system 1 will be described below with reference to FIG. 11 . First, the communication device 100 acquires the appearance information 130 of the tracking target as described with reference to FIG. 4 (step S101 in FIG. 4 ) and pre-registers the information in the storage device 120. For example, the user U uses the communication device 100 to launch an application for the monitoring system 1 and selects an appearance information registration menu from an operation menu using a UI (User Interface). When the menu is selected, the communication device 100 displays an appearance information registration screen on the display. The communication device 100 then launches the camera of the communication device 100 and outputs guidance on how to capture the appearance of the tracking target. The guidance is displayed on the screen or output as audio. The user U can capture the appearance of at least one tracking target by operating the operation unit (or the UI on the screen) of the communication device 100 according to the guidance. The captured image or feature values ​​extracted from the image are stored in the storage device 120 as appearance information 130 of the appearance.

[0087] Thereafter, the flow of FIG. 11 is started. The timing of starting the flow of FIG. 11 can be various. For example, the start timing may be when an operation button for starting monitoring is pressed in the operation menu of the application of the monitoring system 1. Alternatively, the start timing may be when ticketless check-in is accepted. Alternatively, the start timing may be a predetermined time before the departure date and time (before the time of passing through the departure gate) based on the travel itinerary information 240 of the user U. Alternatively, the first server device 400 may instruct the communication device 100 to start processing based on the travel itinerary information 240 or based on an instruction from a communication device (not shown) of the tour conductor.

[0088] In the communication device 100, the location information acquisition unit 104 acquires the location information of the communication device 100 (step S103). The location information acquisition unit 104 periodically acquires the location information of the communication device 100. Then, the information providing unit 106 determines whether the location information of the communication device 100 satisfies a criterion (step S105). If the criterion is satisfied (YES in step S105), the information providing unit 106 reads the appearance information 130 from the storage device 120 and transmits it to the monitoring device 200 (step S107). If the criterion is not satisfied (NO in step S105), the process returns to step S103.

[0089] As described above, the information providing unit 106 links the appearance information 130 with the identification information of the user U and transmits the linked information to the monitoring device 200 .

[0090] Meanwhile, in the monitoring device 200, the acquisition unit 202 acquires the area identification information (step S201), and then waits to receive appearance information from the communication device 100 (NO in step S203). As described above, the area identification information is travel itinerary information 240 of the user U who is the tracking target. Upon receiving the appearance information from the communication device 100 (YES in step S203), the acquisition unit 202 temporarily stores the received appearance information in the storage device 220 as appearance information 230 in association with the identification information of the user U.

[0091] The processing unit 204 then reads out from the storage device 220 the travel itinerary information 240 corresponding to the identification information of the user U whose appearance information has been acquired, and identifies the target area 30 of the user U. Furthermore, the processing unit 204 reads out from the storage device 220 the camera information 250, and identifies the camera 5 corresponding to the identified target area 30. The processing unit 204 then transmits the images captured by the camera 5 in the identified target area 30 and the appearance information of the tracking target to the image processing device 300, causing the image processing device 300 to perform processing. The image processing device 300 uses the images within the target area 30 and the appearance information of the tracking target to search for, find, and track the tracking target (step S205).

[0092] Furthermore, the image processing device 300 determines whether the tracking target satisfies a predetermined condition (step S207). Here, the predetermined condition is, for example, that the tracking target is involved in a crime. If the predetermined condition is met (YES in step S207), the processing unit 204 executes a predetermined process (step S209). If the predetermined condition is not met (NO in step S207), the process returns to step S205, and the processing unit 204 causes the image processing device 300 to repeat the process. The image processing device 300 continues tracking the tracking target while changing the target area 30 according to the travel itinerary information 240 of the user U.

[0093] As described above, according to this embodiment, the communication device 100 includes an appearance information storage processing unit 102, a position information acquisition unit 104, and an information providing unit 106. The appearance information storage processing unit 102 stores appearance information of the tracking target in the storage device 120. The position information acquisition unit 104 acquires position information of the communication device 100. If the position information of the communication device 100 satisfies a predetermined criterion, the information providing unit 106 provides the appearance information of the tracking target to the monitoring device 200, which performs monitoring processing of the target area for monitoring the tracking target. The monitoring device 200 also includes an acquisition unit 202 and a processing unit 204. The acquisition unit 202 acquires area identification information capable of identifying the target area for monitoring the tracking target, and also acquires appearance information of the tracking target. Upon acquiring the appearance information of the tracking target, the processing unit 204 causes an image processing means to identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target, and executes predetermined processing if the tracking target satisfies a predetermined condition.

[0094] As described above, according to the monitoring system 1 of the present disclosure, the monitoring device 200 does not acquire the appearance information of the tracked target, which is personal information, until the location information of the communication device 100 of the user U satisfies a predetermined standard, that is, before the monitoring process starts. Therefore, according to this embodiment, the personal information used during monitoring can be handled appropriately.

[0095] Second Embodiment <Functional Configuration Example> As shown in Fig. 12 , the communication device 100 of this embodiment is the same as the communication device 100 of Fig. 3 except that it further includes a deletion request unit 108 in addition to the configuration of the communication device 100. Note that the elements of Fig. 12 may be combined with the elements of one or more embodiments as long as no contradictions occur. Also, as shown in Fig. 13 , the monitoring device 200 of this embodiment is the same as the monitoring device 200 of Fig. 1 except that it further includes a storage processing unit 206 in addition to the configuration of the monitoring device 200. Note that the elements of Fig. 13 may be combined with the elements of one or more embodiments as long as no contradictions occur.

[0096] When the location information of the communication device 100 indicates that the communication device 100 has moved outside the target area 30, the deletion request unit 108 of the communication device 100 requests the monitoring device 200 to delete the appearance information of the tracked target from the storage device 220 of the monitoring device 200.

[0097] Furthermore, when the storage processing unit 206 of the monitoring device 200 acquires appearance information of the tracking target from the communication device 100 used by the user U who is the tracking target, the storage processing unit 206 stores the information in the storage device 220. Then, when the storage processing unit 206 receives a request to delete the appearance information of the tracking target from the communication device 100, the storage processing unit 206 deletes the appearance information of the tracking target from the storage device 220.

[0098] <Operation Example> An example of the processing operation of the monitoring system 1 will be described below with reference to Fig. 14. The flow of the communication device 100 is executed in parallel with the flow of step S105 in Fig. 11 after step S103 of the flow described with reference to Fig. 11. The flow of the monitoring device 200 is also executed in parallel with the flow of Fig. 11.

[0099] In communication device 100, location information acquisition unit 104 periodically acquires location information of communication device 100 (step S103). Then, deletion request unit 108 determines whether the location information of communication device 100 indicates that communication device 100 has moved outside of target area 30 (step S111). However, the condition for this determination may be that communication device 100 has moved outside of all target areas 30 included in user U's travel itinerary information 240, or that communication device 100 has passed through the departure gate at the airport upon returning home.

[0100] 11 , when the storage processing unit 206 of the monitoring device 200 receives the appearance information from the communication device 100, the storage processing unit 206 stores the appearance information in the storage device 220 as appearance information 230 in association with the identification information of the user U. The deletion request includes the identification information of the user U.

[0101] 14 , thereafter, in monitoring device 200, storage processing unit 206 waits to receive a deletion request from communication device 100 (NO in step S211). Then, upon receiving a deletion request from communication device 100 (YES in step S211), storage processing unit 206 deletes the appearance information associated with the identification information of user U included in the deletion request from storage device 220 (referred to as storage unit in the figure) (step S213).

[0102] As described above, the communication device 100 of this embodiment has the deletion request unit 108, and the monitoring device 200 has the storage processing unit 206. When the location information of the communication device 100 indicates that the communication device 100 has moved outside the target area 30, the deletion request unit 108 of the communication device 100 requests the monitoring device 200 to delete the appearance information of the tracking target from the storage device 220 of the monitoring device 200. Furthermore, when the storage processing unit 206 of the monitoring device 200 acquires the appearance information of the tracking target from the communication device 100 used by the user U who is the tracking target, the storage processing unit 206 stores the information in the storage device 220. Then, when the storage processing unit 206 receives a request to delete the appearance information of the tracking target from the communication device 100, the storage processing unit 206 deletes the appearance information of the tracking target from the storage device 220.

[0103] As described above, according to the monitoring system 1 of this embodiment, the appearance information of the tracking target temporarily stored in the monitoring device 200 can be deleted when the location information of the communication device 100 indicates that the tracking target has moved outside the target area 30. Therefore, according to this embodiment, the monitoring device 200 does not store the appearance information of the tracking target except during monitoring, and thus, similar to the above embodiment, personal information used during monitoring can be handled appropriately.

[0104] (Third embodiment) <Functional configuration example> As shown in Fig. 15, the communication device 100 of this embodiment is the same as the communication device 100 of Fig. 12 except that it further includes a reception unit 110 in addition to the configuration of the communication device 100 of Fig. 12. Note that the elements of Fig. 15 may be combined with the elements of one or more embodiments to the extent that no contradiction occurs. The monitoring device 200 of this embodiment is the same as that of Fig. 13, and will be described using Fig. 13.

[0105] The receiving unit 110 of the communication device 100 receives input of the travel period of the user to be tracked. The deletion request unit 108 requests the monitoring device 200 to delete the appearance information of the user U after the travel period of the user U.

[0106] The travel period accepted by the accepting unit 110 may be information indicating the date and time when the user U will leave the target area 30. The travel period accepted by the accepting unit 110 may also be information indicating the end date or end date and time of the user U's travel period. Alternatively, the accepting unit 110 may accept pressing of an operation button for canceling (terminating) monitoring in an operation menu of the application of the monitoring system 1. Alternatively, the first server device 400 may instruct the communication device 100 to cancel monitoring based on the travel itinerary information 240 or based on an instruction from a communication device (not shown) of the tour conductor.

[0107] The determination of whether the date and time is after the travel period of the user U can be made by, for example, comparing the date and time received by the receiving unit 110 with the current time. Alternatively, the travel period may be determined to have ended after receiving a press of the monitoring cancellation operation button.

[0108] When the storage processing unit 206 of the monitoring device 200 receives a request to delete the appearance information of the tracking target from the communication device 100 , it deletes the appearance information of the tracking target from the storage device 220 .

[0109] <Operation Example> An example of the processing operation of the monitoring system 1 will be described below with reference to Fig. 16. The flow of the communication device 100 is executed in parallel with the flow of step S105 of Fig. 11 after step S103 of the flow described with reference to Fig. 11. The flow of the monitoring device 200 is also executed in parallel with the flow of Fig. 11. The flow of Fig. 16 may also be executed in parallel with the flow of Fig. 14.

[0110] In the communication device 100, the deletion request unit 108 determines whether the travel period of the user U has ended (step S115). However, the condition for this determination may be that the user U has passed through the departure gate at the airport upon returning home.

[0111] 11 , when the storage processing unit 206 of the monitoring device 200 receives the appearance information from the communication device 100, the storage processing unit 206 stores the appearance information in the storage device 220 as appearance information 230 in association with the identification information of the user U. The deletion request includes the identification information of the user U.

[0112] 14 , thereafter, in monitoring device 200, storage processing unit 206 waits for receipt of a deletion request from communication device 100 (NO in step S211). Then, upon receiving a deletion request from communication device 100 (YES in step S2111), storage processing unit 206 deletes the appearance information associated with the identification information of user U included in the deletion request from storage device 220 (referred to as storage unit in the drawing) (step S213).

[0113] As described above, the communication device 100 of this embodiment includes the receiving unit 110. The receiving unit 110 of the communication device 100 receives input of the travel period of the user U to be tracked. The deletion request unit 108 requests the monitoring device 200 to delete the appearance information of the user U after the user U's travel period ends. As described above, the communication device 100 of this embodiment deletes the appearance information of the user U from the monitoring device 200 after the user U's travel period ends. Therefore, the monitoring device 200 does not retain the appearance information of the tracked target under circumstances where monitoring is not necessary. Therefore, according to this embodiment, as with the above embodiment, it is possible to appropriately handle personal information used during monitoring.

[0114] Fourth Embodiment <Functional Configuration Example> As shown in Fig. 17, the monitoring device 200 of this embodiment is the same as the monitoring device 200 of Fig. 1 except that it further includes a group registration unit 208 in addition to the configuration of the monitoring device 200 of Fig. 1. Note that each element of Fig. 17 may be combined with each element of one or more embodiments to the extent that no contradiction occurs.

[0115] The group registration unit 208 registers a plurality of tracking targets (users U) who act together as a group. As shown in Fig. 18 , group information 260 is stored in the storage device 220 for each group, in such a manner that identification information (shown as a group ID in the drawing) that can identify the group is associated with identification information of the user U, who is a tracking target registered as a member of the group.

[0116] The group registration unit 208 registers, as members of the same group, multiple tracking targets (users U) whose location information within a predetermined period indicates that they have been within a predetermined range for a certain period of time or more. For example, multiple users U whose location information indicates that they have been within 3 meters for 40 minutes or more within one hour are registered as members of the same group.

[0117] The predetermined condition used by the processing unit 204 for the determination includes indicating that the location information of one of the tracking targets (user U and belongings 10 of user U) included in the group is outside the location information of the group. Detailed processing when the predetermined condition is satisfied will be described later.

[0118] Furthermore, the group information 260 may pre-register the identification information of users U who are tour participants as members of the same group based on travel itinerary information acquired from the first server device 400. In this case, the group information 260 may register users U who plan to travel together in groups by travel schedule, in accordance with the travel itinerary. In this case, the group information 260 creates multiple groups by schedule, and multiple members are registered in each group.

[0119] <Operation Example> An example of the processing operation of the monitoring device 200 will be described below with reference to FIG. 19 . The processing of this flow is repeatedly executed for a predetermined period after monitoring begins. The processing of this flow may be executed in parallel with the processing flows of other monitoring devices 200. Furthermore, the processing of this flow may be executed periodically (every half day, etc.) after being repeatedly executed for a predetermined period. For example, even if participating in the same tour, people may split up along the way, so the group registration unit 208 periodically updates the group members. However, removal of a member from a group is executed only when an approval operation by the user U is received in order to distinguish from cases of being lost or kidnapped.

[0120] In the monitoring device 200, the acquisition unit 202 acquires location information of each of a plurality of tracking targets (users U) (step S211). The group registration unit 208 identifies a plurality of tracking targets that have been acting together for a certain period of time or more (step S233). The group registration unit 208 registers the plurality of tracking targets (users U) that have been acting together for a certain period of time or more as members of the same group in the group information 260 by associating the identification information of the user U with the identification information of the group.

[0121] As described above, the monitoring device 200 of this embodiment includes a group registration unit 208. The group registration unit 208 registers multiple tracking targets (users U) who act together as a group. In this way, the monitoring device 200 of this embodiment not only achieves the same effects as the above-described embodiment, but also makes it possible to register people who are estimated to act together for a certain period of time or more as members of the same group, even if they have not been registered as a group in advance. Therefore, according to this embodiment, the membership composition of a group can be updated depending on the situation at the time.

[0122] The following describes a method and processing for identifying a crime that a tracking target is likely to have encountered according to predetermined conditions. However, for operations that do not include group discrimination, the configurations of other embodiments other than the fourth embodiment may be used.

[0123] 20 , first, the processing unit 204 of the monitoring device 200 causes the image processing device 300 to track the belongings 10 of each user U (step S241). If the image processing device 300 detects that the belongings 10 have been taken away by a person outside the group (YES in step S243), the processing unit 204 performs processing to notify (output an alert) that "the belongings have been taken away" (step S245). If the image processing device 300 does not detect that the belongings 10 have been taken away by a person outside the group (NO in step S243), the process returns to step S241, and the processing unit 204 causes the image processing device 300 to continue tracking the belongings 10 of each user U.

[0124] In more detail, for example, the image processing device 300 detects all people included in an image within a predetermined range and distinguishes between group members and non-members. The image processing device 300 monitors whether a non-member is approaching a property 10 belonging to a group member. When a non-member approaches a property 10 belonging to a group member, the image processing device 300 monitors whether the position of the person and the property 10 move together by a predetermined distance or more from the original position of the property 10. When the position of the property 10 and the person move together by a predetermined distance or more, the image processing device 300 transmits information indicating the occurrence of "theft" to the processing unit 204.

[0125] However, if there are multiple people other than the group members, it is conceivable that the multiple people may move the belongings 10 in a relay manner. In this case, since the image processing device 300 may not be able to determine whether the person receiving the belongings 10 is a member of the group, it is not a necessary condition that the person and the belongings 10 move "together." The image processing device 300 may monitor whether the belongings 10 of the group members are moving away from the group members by more than a predetermined distance. If it is detected that the belongings 10 of the group members have moved away from the group members by more than a predetermined distance, the image processing device 300 transmits information indicating the occurrence of "theft" to the processing unit 204.

[0126] The location of the possession 10 may be identified by image processing, or if a tag that transmits location information is attached to the possession 10, the location information may be received from the tag. The tag periodically transmits location information to the communication device 100 of the user U who owns the possession 10 via wireless communication. The communication device 100 may transmit the location information received from the tag to the monitoring device 200. Furthermore, if the possession 10 is taken away, the tag will lose wireless communication when it moves away from the communication device 100 by a predetermined distance or more. Therefore, the communication device 100 can detect that the tag has moved away from the user U by a predetermined distance or more. In step S243, the processing unit 204 may also detect that the location of the possession 10 has moved using the location information of the tag.

[0127] When the processing unit 204 receives the information, it performs a process of notifying the occurrence of "theft" (alert output process). The processing unit 204 notifies a predetermined second communication device that a predetermined condition has been met.

[0128] The notification process notifies the communication device 100 of the user U who owns the property 10 and the communication devices (second communication devices) of members of a group to which the user U belongs of the occurrence of the "item pickpocketing" via an application or to a pre-registered notification destination (email address, SMS, etc.). The second communication device may be, for example, at least one of the communication device of the tour conductor and the first server device 400. The second communication device is a computer 1000 such as a smartphone, tablet terminal, or personal computer.

[0129] The communication device 100 or the second communication device that receives the notification may output an alert indicating that a "purchase pickpocket" has occurred. The method for outputting the alert is not particularly limited, but may include, for example, outputting an alarm sound from the speaker of the communication device or displaying information indicating the occurrence of a "purchase pickpocket" on a display. In this case, an application for outputting the alert is installed and activated in advance on the second communication device. Note that a similar method can also be used for outputting an alert when other crimes, etc. are detected.

[0130] After a theft is detected, the processing unit 204 may perform the processing shown in FIG. 21 in addition to the notification processing. First, the processing unit 204 acquires an image (video) at the time of the theft (an image in which the theft is detected) and an image (video) after the theft (step S271). Then, the processing unit 204 causes the image processing device 300 to perform a tracking process of the property 10 using the acquired image (step S273). The processing unit 204 estimates the escape route of the person who took the property 10 based on the movement path of the property 10 tracked by the image processing device 300.

[0131] The processing unit 204 then uses the estimated escape route to identify the direction of escape and identifies the communication device of the police station or police officer or security guard in the direction of escape (step S275).The processing unit 204 then transmits information notifying the occurrence of the theft, together with the image of the theft acquired in step S271, to the identified communication device (step S277).

[0132] The processing unit 204 may also transmit appearance information of the tracking target (property 10) to a second communication device (for example, a communication device at a police station).

[0133] <Lost Child / Abduction> Next, an example of a method for detecting a lost child or abduction and processing after the detection will be described with reference to Fig. 22. First, the processing unit 204 of the monitoring device 200 causes the image processing device 300 to track each user U (step S251). Then, if the image processing device 300 detects that a group member has disappeared from the image (YES in step S253), the processing unit 204 performs processing to notify the occurrence of "lost child or abduction" (step S257).

[0134] The processing unit 204 notifies a predetermined second communication device that a predetermined condition has been met (the occurrence of "lost or kidnapped"). In this example, the second communication device is, for example, a communication device of a member of a group to which the missing user U belongs, or a communication device of the nearest police station. The processing unit 204 may also transmit appearance information of the tracking target (missing user U) to, for example, the communication device of the police station.

[0135] On the other hand, if the image processing device 300 does not detect that a member of the group has disappeared from the image (NO in step S253), the process proceeds to step S255. If the image processing device 300 detects that a member of the group has left the group (YES in step S255), the processing unit 204 performs processing to notify the occurrence of "lost or kidnapped" (step S257). If the image processing device 300 does not detect that a member of the group has left the group (NO in step S255), the process returns to step S251, and the processing unit 204 causes the image processing device 300 to continue tracking each user U.

[0136] <Robbery> Next, an example of a processing method after robbery detection will be described with reference to FIG. 23 . When a robbery is detected using existing image processing technology (YES in step S281), this processing is initiated. This processing is not initiated until a robbery is detected (NO in step S281). The processing unit 204 of the monitoring device 200 causes the image processing device 300 to process images generated by the camera 5 capturing the location where the robbery occurred. The image processing device 300 identifies at least one of the robber and his / her belongings (the stolen property 10 or a bag brought by the robber) as tracking targets (step S283). The image processing device 300 then tracks at least one of the robber and his / her belongings using images captured by the camera 5 within the target area 30 (step S285).

[0137] The processing unit 204 estimates the criminal's escape route based on the movement path of at least one of the criminal and his / her belongings tracked by the image processing device 300 (step S287). Thereafter, similar to the flow of Fig. 21 , the processing unit 204 identifies the direction of escape using the estimated escape route and identifies the communication device (second communication device) of the police station or police officer or security guard in the direction of escape (step S275). The processing unit 204 then transmits information (alert output) notifying the second communication device of the occurrence of the theft, together with the image of the theft acquired in step S271, to the identified communication device (step S277).

[0138] The processing unit 204 may transmit appearance information of the tracking target (the stolen property 10) to a second communication device (e.g., a communication device at a police station).

[0139] <Emergency Button Report> Next, an example of a processing method when pressing of the emergency button is detected will be described with reference to Fig. 24. Pressing of the emergency button can be detected, for example, by receiving a press of an operation button for emergency reporting provided in an application of the monitoring system 1 executed on the communication device 100 or on a website of the monitoring system 1 viewed via a browser. Alternatively, an emergency situation involving the user U may be detected by receiving a notification of pressing of an emergency button on a security reporting terminal or the like carried by the user U or a tour conductor and configured to share information with the monitoring device 200.

[0140] When the pressing of the emergency button is detected (YES in step S261), this process is started. Until the pressing of the emergency button is detected (NO in step S261), this process is not started.

[0141] When pressing of the emergency button is detected (YES in step S261), the acquisition unit 202 of the monitoring device 200 identifies the user U who pressed the emergency button and acquires the location information of the identified user U (step S263). The processing unit 204 then identifies the target area 30 from the location information (step S265). The processing unit 204 then acquires an image (video) captured by the camera 5 of the target area 30 (step S267). The processing unit 204 then performs processing to submit a damage report to the police station with jurisdiction over the location of the camera 5, along with the acquired image (step S269). The processing unit 204 may also notify the second communication device (output an alert) that the "pressing of the emergency button" has been detected.

[0142] The processing unit 204 prepares a report of crime to the local police station by automatically entering the necessary information using information about the user U in accordance with the prescribed format for a report of crime of the local police station. The information about the user U can be information that has been registered in advance by the user U in a user information input menu of the application or website. If the information registered by the user U is entered in a language other than English or the local language (for example, Japanese), the processing unit 204 translates the information into the local language or English using machine translation, and then enters it into the report of crime form.

[0143] The processing unit 204 may transmit the created report to a communication device of the police station in charge of the crime, or may print out the created report on a printer. In the latter case, the tour conductor or a member of the group may submit the printed report to the police station in charge of the crime.

[0144] The processing unit 204 may also transmit appearance information of the tracking target (at least one of the user U and the property 10) to a second communication device (for example, a communication device at a police station).

[0145] While the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may also be adopted. (Another Embodiment 1) For example, as shown in Fig. 25, the monitoring system 1 may further include a first server device 400 and a second server device 500 in addition to the monitoring device 200. The first server device 400 is a server computer of a travel agency, an airline, or the like. The second server device 500 is a server computer of a local government (smart city) or the like that the user U visits.

[0146] The user U registers the appearance information of the tracking target in advance in the storage device 120 of the communication device 100 via an application or a website.

[0147] The first server device 400 also accepts advance registration of user U's travel itinerary information 240. Alternatively, when user U participates in a tour planned by a travel agency, the first server device 400 may manage user U's travel itinerary information as customer information and register it linked to user U's identification information. The first server device 400 provides user U's travel itinerary information to the monitoring device 200 for information that has been approved in advance by user U. For example, upon detecting user U's departure from the country, the first server device 400 may transmit user U's travel itinerary information to the monitoring device 200. Alternatively, the first server device 400 may transmit user U's travel itinerary information to the monitoring device 200 at the departure date and time of the trip based on user U's travel itinerary information. Alternatively, the communication device 100 or the monitoring device 200 may request the first server device 400 to provide user U's travel itinerary information to the monitoring device 200.

[0148] Then, the communication device 100 provides the monitoring device 200 with appearance information of the user U at least one of the timings when the user U passes through the departure gate and the timing when the user U passes through the entry gate.

[0149] The second server device 500 monitors at least one target area 30 (shown as multiple target areas 30a, 30b, ... in the figure). The second server device 500 receives images generated by at least one camera 5 (shown as cameras 5a, 5b, ... in the figure) installed in each target area 30, identifies the tracking target, and monitors it. The second server device 500 performs an analysis process on the tracking target people (shown as users Ua, Ub, ... in the figure) and their belongings 10 (shown as belongings 10a, 10b, ... in the figure) within each target area 30 using images captured by the cameras 5.

[0150] In this example, the second server device 500 has at least the functions of the image processing device 300 described above. However, the image processing device 300 may be hardware separate from the second server device 500. The monitoring device 200 transmits to the second server device 500 the target area 30 identified based on the area identification information acquired by the acquisition unit 202 and appearance information of the tracking target, and instructs the second server device 500 to start image analysis processing of the tracking target. Then, the monitoring device 200 receives the results of the image analysis processing from the second server device 500.

[0151] Furthermore, the second server device 500 may have the function of the processing unit 204 of the monitoring device 200 described above. In this case, the monitoring device 200 obtains the processing result of the processing unit 204 from the second server device 500.

[0152] (Another Embodiment 2) In the above embodiment, the monitoring system 1 of the present disclosure recognizes and monitors the tracking target by using an image processing device to process images of the tracking target generated by the camera 5. In another embodiment, the monitoring system 1 of the present disclosure may monitor the tracking target by recognizing it using other recognition means. Specifically, an image (three-dimensional point cloud data) generated using a ranging means such as radar or LiDAR (Laser Imaging Detection And Ranging) may be recognized using an image recognition device. Furthermore, sound collected using a microphone may be recognized (authenticated) using a voice authentication device. The monitoring device 200 can recognize the tracking target using at least one of an image captured by the camera 5, an image captured by radar or LiDAR, and sound captured by a microphone.

[0153] When radar or LIDAR is used, an image (point cloud data) of the entire tracking target (in the case of a person, the entire body) is generated using the radar or LIDAR in the pre-registration process of appearance information. An image processing device is used to acquire feature points from the generated image (point cloud data) of the entire tracking target (in the case of a person, the entire body) to an extent that at least the tracking target can be identified (in the case of a person, the body shape, etc. can be identified), and generate appearance information. The image processing device may be included in the communication device 100 or may be an external device to the communication device 100. When the image processing device is an external device to the communication device 100, the communication device 100 acquires the appearance information from the image processing device. When the image processing device is included in the communication device 100, the communication device 100 acquires the image generated by the radar or LIDAR, performs image processing, and generates appearance information of the tracking target. The appearance information storage processing unit 102 of the communication device 100 stores the appearance information of the tracking target (in the case of a person, biometric information) in the appearance information 130.

[0154] The processing unit 204 of the monitoring device 200 acquires images (point cloud data) of the target area 30 using a radar or LIDAR installed in the target area 30, and causes the image processing device to process the images. The image processing device detects a person or object and extracts feature amounts. The processing unit 204 of the monitoring device 200 then causes the image processing device to compare the extracted feature amounts of the person or object with the appearance information 230 (feature amounts) of the tracking target acquired from the communication device 100. In this way, the monitoring device 200 can identify the tracking target using the image processing device.

[0155] The processing unit 204 of the monitoring device 200 then associates the position information of the radar or lidar that generated the image of the identified tracking target and the image capture time with the identification information of the tracking target, and stores the information as tracking information, thereby making it possible to identify the movement path of the tracking target.

[0156] When a microphone is used, in the pre-registration process of appearance information, the appearance information storage processing unit 102 of the communication device 100 uses the microphone of the communication device 100 to have the person to be tracked speak, collects the voice, generates voice data for voice recognition, extracts features from the voice data, and stores them in the appearance information 130 as appearance information of the tracked target.

[0157] The processing unit 204 of the monitoring device 200 uses a microphone installed in the target area 30 to acquire voice data collected in the target area 30, and causes the voice authentication device to process the data. The voice authentication device detects voice and extracts features from the voice data. The processing unit 204 of the monitoring device 200 then causes the voice authentication device to perform a comparison process between the extracted voice features of the person and the appearance information 230 (voice features) of the tracking target acquired from the communication device 100. In this way, the monitoring device 200 identifies the tracking target using the voice authentication device.

[0158] The processing unit 204 of the monitoring device 200 then associates the location information of the microphone that collected the sound of the identified tracking target and the time of sound collection (recording) with the identification information of the tracking target and stores the information as tracking information, thereby making it possible to identify the movement route of the tracking target.

[0159] Another example of the present disclosure may be a program causing at least one computer to execute the above-disclosed method, or a computer-readable recording medium having such a program recorded thereon. This recording medium includes a non-transitory tangible medium. The computer program includes computer program code that, when executed by a computer, causes the computer to implement the monitoring method or control method on a monitoring device or a communication device.

[0160] Any combination of the above components, and conversion of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure.

[0161] Furthermore, the various components of the present disclosure do not necessarily have to be independent entities, but may be formed as a single member by multiple components, one component may be formed from multiple components, one component may be a part of another component, or part of one component may overlap with part of another component, etc.

[0162] Furthermore, although the methods and computer programs disclosed herein describe a plurality of processing steps in a sequential order, the order in which the processing steps are performed does not limit the order in which the processing steps are performed. Therefore, when implementing the methods and computer programs disclosed herein, the order of the processing steps can be changed as long as it does not cause any problems in terms of the content.

[0163] Furthermore, the multiple processing steps of the method and computer program disclosed herein are not limited to being executed at different times, and may be executed while one processing step is being executed, or may be executed partially or entirely overlapping with another processing step, etc.

[0164] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate. Note that, when information about a user (user U) is acquired and used in the present disclosure, this must be done lawfully.

[0165] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps executed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments can be combined as long as the content is not contradictory.

[0166] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1. A monitoring device comprising: acquisition means for acquiring area identification information capable of identifying a target area in which a tracking target is to be monitored, and for acquiring appearance information of the tracking target; and processing means for, upon acquiring the appearance information of the tracking target, causing image processing means to identify the tracking target within the target area using a captured image of the target area identified using the area identification information and the appearance information of the tracking target, and for executing predetermined processing if the tracking target satisfies a predetermined condition. 2. The monitoring device described in 1., wherein the acquisition means acquires the appearance information of the tracking target from a first communication device used by a user who is the tracking target. 3. The monitoring device described in 1. or 2., wherein the processing means outputs an alert as the predetermined processing. 4. The monitoring device described in any one of 1. to 3., further comprising the image processing means. 5. 1. to 4. 5. The monitoring device described in any one of above, wherein at least one of the process of identifying the tracking target and the process of determining whether the predetermined condition is satisfied is executed using a method defined for each of the target areas. 6. The monitoring device described in any one of above 1. to 5., wherein the predetermined condition is at least one of the location information of the tracking target indicating that it has moved outside a reference area and the location information of the tracking target cannot be detected. 7. The monitoring device described in any one of above 1. to 6., further comprising group registration means for registering multiple tracking targets that act together as a group, wherein the predetermined condition includes the location information of one tracking target, of the location information of the multiple tracking targets included in the group, indicating that it has moved outside the location information of the group. 8. The monitoring device described in any one of above 1. to 7., wherein the predetermined condition includes the tracking target having encountered at least one of theft, baggage pickpocketing, kidnapping, robbery, violence, and being lost.9. The monitoring device described in any one of 1. to 8., wherein the processing means notifies a predetermined second communication device that the predetermined condition has been satisfied. 10. The monitoring device described in 9., wherein the processing means further transmits appearance information of the tracking target to the second communication device. 11. The monitoring device described in any one of 1. to 10., further comprising storage processing means for storing the appearance information of the tracking target in storage means when acquiring it from a first communication device used by the user who is the tracking target, and for deleting the appearance information of the tracking target from the storage means when receiving a request to delete the appearance information of the tracking target from the first communication device.

[0167] 12. A program causing a computer implementing a communication device to execute a storage process for storing appearance information of a tracking target in a storage means, a location information acquisition process for acquiring location information of the communication device, and an information provision process for providing the appearance information of the tracking target to a monitoring device that performs monitoring processing of a target area for monitoring the tracking target if the location information of the communication device satisfies a predetermined criterion. 13. A program described in 12., wherein the predetermined criterion is that the location information of the communication device indicates that the communication device has passed through a departure gate at an airport. 14. A program described in 12. or 13., wherein the predetermined criterion is that the location information of the communication device is included in the target area. 15. A program described in any one of 12. to 14., wherein the appearance information of the tracking target includes at least biometric information of a user of the communication device. 16. A program described in any one of 12. to 15., wherein in the information provision process, the appearance information of the tracking target is provided on the condition that it is for a predetermined period of time. 17. A program described in 12. to 16. 18. The program described in any one of items 1 to 17, further causing the computer to execute a deletion request process to request the monitoring device to delete the appearance information of the tracked target from a storage means of the monitoring device when the location information of the communication device indicates that the communication device has moved outside the target area. 19. The program described in item 16, further causing the computer to execute a reception process to receive input of a travel period of the user who is the tracked target, and in the deletion request process, making a request to delete the appearance information of the user after the user's travel period.

[0168] 19. A monitoring system comprising: a communication device used by a user who is the tracking target; a camera that captures images of a target area in which the tracking target is monitored; and a monitoring device that monitors the tracking target using captured images of the target area generated by the camera, wherein the monitoring device comprises: an acquisition means that acquires area identification information capable of identifying the target area and acquires appearance information of the tracking target; and a processing means that, upon acquiring the appearance information of the tracking target, causes an image processing means to identify the tracking target within the target area using the captured images of the target area identified using the area identification information and the appearance information of the tracking target, and executes a predetermined process if the tracking target satisfies a predetermined condition. 20. A monitoring method comprising: one or more computers that acquire area identification information capable of identifying a target area in which the tracking target is monitored and acquires the appearance information of the tracking target; and, upon acquiring the appearance information of the tracking target, causes an image processing means to identify the tracking target within the target area using the captured images of the target area identified using the area identification information and the appearance information of the tracking target, and executes a predetermined process if the tracking target satisfies a predetermined condition. 21. 21. A program that causes a computer to execute: an acquisition process that acquires area specification information that can specify a target area in which a tracking target is to be monitored, and acquires appearance information of the tracking target; and an execution process that, upon acquiring the appearance information of the tracking target, causes an image processing means to identify the tracking target within the target area using a captured image of the target area specified using the area specification information and the appearance information of the tracking target, and executes a predetermined process if the tracking target satisfies a predetermined condition. 22. A communication device comprising: a storage means that stores appearance information of the tracking target in a storage means; a location information acquisition means that acquires location information of a communication device; and an information provision means that, if the location information of the communication device satisfies a predetermined standard, provides the appearance information of the tracking target to a monitoring device that performs monitoring processing of the target area in which the tracking target is to be monitored.23. A method for controlling a communication device, in which one or more computers store appearance information of a tracking target in a storage means, acquire location information of a communication device, and, if the location information of the communication device satisfies a predetermined criterion, provide the appearance information of the tracking target to a monitoring device that performs monitoring processing of a target area for monitoring the tracking target. 24. A computer-readable recording medium that stores a program that causes a computer that realizes a communication device to execute: a storage process for storing appearance information of the tracking target in a storage means, a location information acquisition process for acquiring location information of the communication device, and an information provision process for providing the appearance information of the tracking target to a monitoring device that performs monitoring processing of a target area for monitoring the tracking target, if the location information of the communication device satisfies a predetermined criterion. 25. A computer-readable recording medium storing a program that causes a computer to execute: an acquisition process for acquiring area specification information that can specify a target area in which a tracking target is to be monitored, and acquiring appearance information of the tracking target; and an execution process for, upon acquiring the appearance information of the tracking target, causing an image processing means to specify the tracking target within the target area using a captured image of the target area specified using the area specification information and the appearance information of the tracking target, and to execute a predetermined process if the tracking target satisfies a predetermined condition.

[0169] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 11 that are dependent on the monitoring device of Supplementary Note 1 may also be dependent on the monitoring method of Supplementary Note 20 and the program for the monitoring device of Supplementary Note 21 in the same dependency relationship as Supplementary Notes 2 to 11. Furthermore, some or all of the processing procedures described in Supplementary Notes 13 to 18 that are dependent on the program for the communication device of Supplementary Note 12 may also be dependent on the communication device of Supplementary Note 22 and the control method for the communication device of Supplementary Note 23 in the same dependency relationship as Supplementary Notes 13 to 18. Furthermore, not limited to Supplementary Notes 1, 20, 21, 12, 22, and 23, some or all of the configurations described as the Supplements may also be dependent on various hardware, software, various recording means for recording software (e.g., the recording media of Supplementary Note 24 and Supplementary Note 25), or the system of Supplementary Note 19, within the scope of each of the above-mentioned embodiments.

[0170] This application claims priority based on Japanese Patent Application No. 2023-132000, filed on August 14, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0171] REFERENCE SIGNS LIST 1 Surveillance system 3 Communication network 5, 5a, 5b Camera 10, 10a, 10b Property 30, 30a, 30b Target area 100 Communication device 102 Appearance information storage processing unit 104 Location information acquisition unit 106 Information provision unit 108 Deletion request unit 110 Reception unit 120 Storage device 130 Appearance information 200 Surveillance device 202 Acquisition unit 204 Processing unit 206 Storage processing unit 208 Group registration unit 220 Storage device 230 Appearance information 240 Travel itinerary information 250 Camera information 260 Group information 300 Image processing device 400 First server device 500 Second server device 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input / output interface 1060 Network Interface

Claims

1. A means for acquiring area identification information that can identify the target area to be monitored for tracking, and for acquiring visual information of the tracking target, A monitoring device comprising: an image processing means that, upon acquiring the appearance information of the target to be tracked, causes the image processing means to identify the target to be tracked within the target area using the captured image of the target area identified using the area identification information and the appearance information of the target to be tracked, and if the target to be tracked satisfies predetermined conditions, a processing means that performs predetermined processing.

2. In the monitoring device according to claim 1, The acquisition means is a monitoring device that acquires the appearance information of the tracking target from a first communication device used by the user who is the tracking target.

3. In the monitoring device according to claim 1 or 2, The processing means is a monitoring device that outputs an alert as the predetermined processing.

4. In the monitoring device according to claim 1 or 2, The system further includes a group registration means for registering multiple aforementioned tracking targets acting together as a group, A monitoring device in which the predetermined condition includes indicating that the location information of one of the multiple tracked targets included in the group has deviated from the location information of the group.

5. The communication device used by the user being tracked, A camera that captures images of the target area to monitor the aforementioned tracking target, A monitoring device that uses the image of the target area generated by the camera to monitor the target being tracked, Equipped with, The aforementioned monitoring device is An acquisition means for acquiring area identification information that can identify the target area, and for acquiring visual information of the subject to be tracked, The system includes, upon acquiring the appearance information of the target to be tracked, an image processing means that uses the captured image of the target area identified using the area identification information and the appearance information of the target to identify the target within the target area, and a processing means that performs a predetermined process if the target satisfies predetermined conditions. Monitoring system.

6. On one or more computers, In addition to obtaining area identification information that allows for the identification of the target area to be monitored for tracking, the external appearance information of the tracking target is also obtained. A monitoring method comprising: obtaining visual information of the target to be tracked; having an image processing means identify the target to be tracked within the target area using the captured image of the target area identified using the area identification information and the visual information of the target to be tracked; and executing a predetermined process if the target to be tracked meets predetermined conditions.

7. On the computer, A process to acquire area identification information that allows for the identification of the target area to be monitored, and to acquire visual information of the target being tracked. A program that, upon acquiring the appearance information of the target to be tracked, causes the image processing means to identify the target to be tracked within the target area using the captured image of the target area identified using the area identification information and the appearance information of the target to be tracked, and executes an execution process if the target to be tracked satisfies predetermined conditions.

8. A computer that makes up a communication device, A memory process that stores the external appearance information of the target being tracked into a memory device. Location information acquisition process for acquiring location information of the aforementioned communication device, A program that, when the location information of the communication device meets a predetermined standard, causes a monitoring device to perform monitoring processing for the area to be monitored for the tracking target to perform information provision processing, which provides the appearance information of the tracking target to the monitoring device.

9. In the program described in claim 8, The predetermined criterion is a program in which the location information of the communication device indicates that the device has passed through the departure gate at the airport.

10. In the program according to claim 8 or 9, To the aforementioned computer, A program that, when the location information of the communication device indicates that it has moved outside the target area, causes the monitoring device to further execute a deletion request process that requests the monitoring device to delete the appearance information of the tracked object from the storage means of the monitoring device.