Monitoring device, monitoring method, and program

Abstract

A monitoring device (100) is provided with: a biometric information acquisition unit (102) that acquires biometric information of a subject present in a predetermined space; an image acquisition unit (104) that acquires an image including the subject and surroundings thereof; a determination unit (106) that determines whether the biometric information meets a first criterion and whether the state of others located around the subject meets a second criterion; and an output processing unit (108) that outputs predetermined information when it is determined that both the first criterion and the second criterion are met.

Description

[Technical Field]

[0001] The present invention relates to a monitoring system, a monitoring device, a monitoring method, and a recording medium. [Background technology]

[0002] In recent years, crimes and troubles involving moving vehicles have been occurring one after another, making crime prevention measures an urgent necessity. In addition, in some areas, crimes such as kidnapping of minors have also been occurring frequently.

[0003] An example of a trouble response system for use on a train is described in Patent Document 1. The system in Patent Document 1 includes a first input unit that inputs the detection results of changes in the biometric information (heart rate, etc.) of a target person such as a passenger detected using a sensor installed in the train, a second input unit that inputs the recognition results of a recognition device that recognizes the target person's movements, and a processing unit that determines whether or not a problem has occurred based on the input results of the first and second input units, and performs processing according to the target person's movements (for example, making an announcement on the train).

[0004] Patent Document 2 describes an example of an image processing device that estimates the risk level when an unspecified number of people gather at an event venue, etc. Device The system has a congestion analysis unit that estimates the degree of congestion of people based on the captured image, an attribute analysis unit that estimates the attributes of people based on the captured image, a danger level analysis unit that estimates the danger level based on the congestion degree and the attributes of people, and a display processing unit that controls the display of various information on the display unit, and the display processing unit displays a screen showing the danger level.

[0005] Patent Document 3 describes an information processing method that can use a learning model to determine whether a subject is engaging in suspicious behavior based on a captured image of the subject. Patent Document 4 describes a video surveillance device that includes a camera image acquisition means that acquires surveillance video information, a fault detection means that extracts multiple person areas from the surveillance video information, determines the gaze direction of the person for each person area, and determines that the gaze directions of the multiple people are directed toward a specific position to detect the occurrence of a notable event, and an output device that notifies the occurrence of a notable event.

[0006] Patent Document 5 describes a monitoring system that monitors the interior of a train. The monitoring system in Patent Document 5 detects people present inside the train based on video data captured by a surveillance camera or video data recorded in a recording device, and determines an occupancy rate A based on the proportion of the train occupied by the detected people. The system also measures the number of people getting on and off based on video data captured by a surveillance camera or video data recorded in a recording device, and determines an occupancy rate B based on the number of people currently on board relative to the maximum number of people that can board. The occupancy rates calculated using these two methods are then averaged, or the like, to determine a highly accurate occupancy rate.

[0007] Patent Document 6 describes an in-vehicle monitoring device that can prevent passengers from falling inside a public transportation vehicle intended to transport a large number of passengers. The in-vehicle monitoring device of Patent Document 6 includes a riding state grasping means for grasping the riding state of passengers, a running state grasping means for grasping the running state of the vehicle, and a notification means for issuing a notification regarding passenger safety based on the riding state and running state. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-106729 [Patent Document 2] Japanese Patent Application Publication No. 2019-200715 [Patent Document 3] Patent Publication No. 2021-164008 [Patent Document 4] Japanese Patent Application Publication No. 2018-148402 [Patent Document 5] Japanese Patent Application Laid-Open No. 2013-25523 [Patent Document 6] Japanese Patent Application Laid-Open No. 2016-62414 Summary of the Invention [Problem to be solved by the invention]

[0009] The technology described in the above-mentioned patent documents relates to a technology for detecting and reporting the occurrence of a trouble in a predetermined space such as a vehicle. While many technologies have been proposed, the inventors have considered ways to accurately detect the occurrence of an incident and efficiently report it.

[0010] In view of the above-mentioned problems, one example of the object of the present invention is to provide a monitoring system, a monitoring device, a monitoring method, and a recording medium that solve the problem of improving the accuracy of detecting and reporting the occurrence of an incident within a specified space. [Means for solving the problem]

[0011] According to one aspect of the present invention, a biometric information acquisition means for acquiring biometric information of a subject in a predetermined space; image acquisition means for acquiring an image including the subject and its surroundings; a determination means for determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; and an output processing means for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied.

[0012] According to one aspect of the present invention, An imaging means; a monitoring device; The monitoring device a biometric information acquisition means for acquiring biometric information of a subject in a predetermined space; image acquisition means for acquiring an image including the subject and its surroundings from the imaging means; a determination means for determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; and an output processing means for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied.

[0013] According to one aspect of the present invention, Acquire biometric information of subjects in a specified space, acquiring an image including the subject and their surroundings; determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; There is provided a monitoring method that outputs predetermined information when it is determined that both the first criterion and the second criterion are satisfied.

[0014] According to one aspect of the present invention, On the computer, A procedure for acquiring biometric information of a subject present in a predetermined space; acquiring an image including the subject and their surroundings; a step of determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; A computer-readable recording medium is provided that stores a program for executing a procedure for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied.

[0015] Another aspect of the present invention may be a program that causes at least one computer to execute the method of the above aspect, or a computer-readable recording medium on which such a program is recorded. This recording medium includes a non-transitory tangible medium. The computer program comprises computer program code which, when executed by a computer, causes the computer to perform the monitoring method on a monitoring device.

[0016] Any combination of the above components, and any transformation of the present invention into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present invention.

[0017] Furthermore, the various components of the present invention do not necessarily have to be independent entities, but may be formed as a single member by multiple components, one component may be formed from multiple components, one component may be part of another component, or part of one component may overlap with part of another component, etc.

[0018] Furthermore, although the method and computer program of the present invention describe a number of steps in a sequential order, the order in which the steps are described does not limit the order in which the steps are executed. Therefore, when implementing the method and computer program of the present invention, the order of the steps can be changed as long as it does not cause any problems in terms of the content.

[0019] Furthermore, the multiple steps of the method and computer program of the present invention are not limited to being executed at different times, and therefore, a step may occur while another step is being executed, or the execution timing of a step may partially or completely overlap with the execution timing of another step, etc. [Effects of the Invention]

[0020] According to one aspect of the present invention, it is possible to provide a monitoring system, a monitoring device, a monitoring method, and a recording medium that solve the problem of improving the accuracy of detecting and reporting the occurrence of an incident within a specified space. [Brief explanation of the drawings]

[0021] [Figure 1]1 is a diagram illustrating an overview of a monitoring device according to an embodiment. [Figure 2] 2 is a flowchart showing an example of the operation of the monitoring device of FIG. 1; [Figure 3] 1 is a diagram conceptually illustrating a system configuration of a monitoring system according to an embodiment. [Figure 4] 2 is a block diagram illustrating an example of the hardware configuration of a computer that realizes the monitoring device of FIG. 1. FIG. [Figure 5] 10 is a diagram for explaining the formation of others relative to a target person identified by a determination unit. FIG. [Figure 6] 10 is a flowchart illustrating an example of the operation of the monitoring system according to the embodiment. [Figure 7] FIG. 2 is a functional block diagram illustrating an example of a logical configuration of a monitoring device according to an embodiment. [Figure 8] FIG. 2 is a functional block diagram illustrating an example of a logical configuration of a monitoring device according to an embodiment. [Figure 9] 10 is a flowchart illustrating an example of a location specification process of the monitoring device according to the embodiment. [Figure 10] FIG. 2 is a functional block diagram illustrating an example of a logical configuration of a monitoring device according to an embodiment. [Figure 11] 10 is a flowchart illustrating an example of an output process of the monitoring device according to the embodiment. [Figure 12] 1 is a partial functional block diagram showing a configuration of a main part of a monitoring device according to an embodiment; [Figure 13] 10A and 10B are diagrams illustrating examples of data structures of various information indicating the occurrence status of an incident. [Figure 14] FIG. 10 is a diagram illustrating an example of a data structure of analysis information. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all drawings, similar components are designated by similar reference numerals, and their description will be omitted as appropriate. In addition, in the following drawings, configurations of parts that are not related to the essence of the present invention are omitted and are not shown.

[0023] In the embodiments, "acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and inputting data or information output from another device into the device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, pushed, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0024] <Minimum configuration example> 1 is a diagram showing an overview of a monitoring device 100 according to an embodiment. The monitoring device 100 includes a biometric information acquisition unit 102, an image acquisition unit 104, a determination unit 106, and an output processing unit 108. The biometric information acquisition unit 102 acquires biometric information of a subject who is present in a predetermined space. The image acquisition unit 104 acquires an image including the subject and his / her surroundings. The determination unit 106 determines whether or not the biometric information satisfies a first criterion and the states of other people around the subject satisfy a second criterion. When it is determined that both the first criterion and the second criterion are satisfied, the output processing unit 108 outputs predetermined information.

[0025] When both the first criterion and the second criterion are satisfied in the determination by the determination unit 106, it can be determined that, for example, an incident, which is some kind of emergency involving a subject, has occurred in a predetermined space. Then, the output processing unit 108 can notify the occurrence of an incident by performing a process of outputting predetermined information to a predetermined output means.

[0026] <Example of operation> FIG. 2 is a flowchart showing an example of the operation of the monitoring device 100 of FIG. First, the biometric information acquisition unit 102 acquires biometric information of a subject in a predetermined space (step S101). The image acquisition unit 104 acquires an image including the subject and his / her surroundings (step S103). The determination unit 106 determines whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion (step S105). If the biometric information satisfies the first criterion and the state of other people around the subject satisfies the second criterion (YES in step S105), the output processing unit 108 outputs predetermined information (step S107). If the biometric information does not satisfy the first criterion or if the state of other people around the subject does not satisfy the second criterion (NO in step S105), step S107 is bypassed and this process ends.

[0027] As described above, in the monitoring device 100, the biometric information acquisition unit 102 acquires biometric information of a subject present in a specified space, the image acquisition unit 104 acquires an image including the subject and his or her surroundings, the determination unit 106 determines whether the biometric information satisfies a first criterion and whether the state of other people present around the subject satisfies a second criterion, and the output processing unit 108 outputs specified information when it is determined that both the first criterion and the second criterion are satisfied.

[0028] With this configuration, an incident can be detected based on the subject's biometric information and the status of others around the subject, and the occurrence of the incident can be reported by outputting predetermined information. In other words, by detecting abnormalities in the subject from the subject's biometric information and detecting abnormalities in the status of others around the subject, the occurrence of the incident can be more accurately estimated. This monitoring device 100 can solve the problem of improving the accuracy of detecting and reporting the occurrence of an incident within a specified space.

[0029] A detailed example of the monitoring device 100 will be described below.

[0030] (First embodiment) <System Overview> 3 is a diagram conceptually illustrating the system configuration of a monitoring system 1 according to an embodiment of the present invention. The monitoring system 1 monitors incidents that occur in a predetermined space, for example, inside a moving vehicle 60. Incidents include, for example, nuisances and criminal acts such as molestation, kidnapping, theft, robbery, snatching, intimidation, threats, intimidation, violence, bringing in dangerous substances, the emission of strange odors, and voyeurism.

[0031] The monitoring system 1 detects the subject T's mental instability, fear, or tension caused by the occurrence of these incidents from the subject T's biological information. Then, the monitoring system 1 reports to a monitoring center or a manager according to predetermined conditions.

[0032] The monitoring system 1 includes a monitoring device 100, a camera 5 that captures an image of a predetermined space, and a portable terminal 10 for a subject T.

[0033] The monitoring device 100 includes a storage device 130. The storage device 130 may be provided inside or outside the monitoring device 100. In other words, the storage device 130 may be hardware that is integrated with the monitoring device 100, or may be hardware that is separate from the monitoring device 100.

[0034] The camera 5 includes an imaging element such as a lens and a CCD (Charge Coupled Device) image sensor, and is, for example, a network camera such as an IP (Internet Protocol) camera. The network camera has, for example, a wireless LAN (Local Area Network) communication function, and is connected to the monitoring device 100 via a relay device (not shown) such as a router of the communication network 3.

[0035] In one example, camera 5 is installed on the ceiling of a predetermined space. Camera 5 may be a camera capable of capturing images in all directions of 360 degrees. In one example, camera 5 is a camera equipped with a fisheye lens. Camera 5 may also be equipped with a mechanism for controlling the orientation of the camera body and lens, zoom control, focusing, etc., in accordance with the movement of a person.

[0036] Although only one camera 5 is shown in the drawing, the number of cameras 5 is not limited to this. A plurality of cameras 5 may be used.

[0037] The images generated by the camera 5 are preferably taken in real time and transmitted to the monitoring device 100. However, the images transmitted to the monitoring device 100 do not have to be transmitted directly from the camera 5, but may be images delayed by a predetermined time. The images taken by the camera 5 may be temporarily stored in another storage device 130, and the monitoring device 100 may read them out from the storage device 130 sequentially or at predetermined intervals. Furthermore, the images transmitted to the monitoring device 100 are preferably moving images, but may also be frame images taken at predetermined intervals or still images.

[0038] The subject's portable terminal 10 is a terminal carried by the subject T, and is, for example, a computer such as a portable personal computer, a smartphone, or a tablet terminal. The subject T may wear a wearable terminal 20 that acquires biometric information.

[0039] The wearable terminal 20 is assumed to be set in advance to be able to communicate with the subject's portable terminal 10 via short-range communication, for example, Bluetooth (registered trademark). The biological information acquired by the wearable terminal 20 is transmitted to the monitoring device 100 via the subject's portable terminal 10. However, the biological information may also be transmitted directly from the wearable terminal 20 to the monitoring device 100.

[0040] The biological information includes at least one of the heart rate and the amount of sweating. However, the biological information is not limited to this and may also include body temperature, blood pressure, respiratory rate, blood oxygen concentration, etc. The biological information such as the heart rate and amount of sweating is used, for example, to determine whether the psychological state of the subject T is tense. Therefore, the biological information may be other information as long as it can identify the psychological state of the subject T. The wearable terminal 20 has a sensor that measures at least one of the heart rate and amount of sweating of the subject T, and has a function of transmitting the measured value to the subject's mobile terminal 10 or the monitoring device 100.

[0041] The monitoring system 1 may further include a display device 120. The display device 120 is connected to the monitoring device 100. The display device 120 may be, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like, but is not limited to these. The display device 120 may be provided, for example, on a monitoring panel installed in a monitoring center or the like. An operator U at the monitoring center can monitor the situation in a predetermined space by looking at the display device 120. The operator U may wear headphones 50 and listen to the audio output from the monitoring device 100. The headphones 50 may also be combined with a microphone and may be used, for example, to talk to a station employee C inside a station.

[0042] The monitoring system 1 may further include at least one of an in-vehicle speaker 7 and an in-house speaker 40. The in-vehicle speaker 7 and the in-house speaker 40 are each connected to the monitoring device 100.

[0043] In this embodiment, the predetermined space to be monitored is the interior of a moving vehicle 60. The monitoring system 1 may further include a station staff terminal 30 carried by the station staff C. The station staff terminal 30 is, for example, a computer such as a portable personal computer, a smartphone, or a tablet terminal.

[0044] An emergency call button may be installed on board the vehicle to notify the conductor or driver of a crime or other incident that occurs on board. However, emergency call buttons installed on board the vehicle are often difficult to use when an actual problem occurs. In recent years, crimes and other incidents have occurred frequently on board moving vehicles 60, and crime prevention measures are urgently needed. By targeting incidents that occur on board the vehicle, it becomes possible to accurately detect the occurrence of incidents on board the vehicle and efficiently notify the driver.

[0045] The object to be monitored is not limited to the vehicle 60, but may be a predetermined facility, such as a facility for the elderly, a nursery school, a kindergarten, etc. In the embodiment, since the object to be monitored is the vehicle 60, the description will be given using the station staff C and the station staff terminal 30, but in other facility examples, the object to be monitored may be the staff of the facility and an operation terminal carried by the staff.

[0046] When using the monitoring system 1, the subject T must first register to use the services of the monitoring system 1. The service can be used, for example, by installing and launching a specific application on the subject T's mobile terminal 10, or by accessing a specific website using a browser or the like on the subject T's mobile terminal 10.

[0047] The monitoring system 1 may be a so-called server-client system. In this case, the monitoring device 100 functions as a server connected to the terminals 10 and 30 via the communication network 3, and the terminals 10 and 30 function as client terminals. The functions of the monitoring device 100 may be realized by the terminals 10 and 30 accessing a server on the cloud via the Internet (for example, SaaS (Software as a Service), PaaS (Platform as a Service), HaaS, or IaaS (Hardware / Infrastructure as a Service)).

[0048] Alternatively, the services provided by the monitoring device 100 may be available on each terminal 10 and terminal 30 by accessing a predetermined URL (Uniform Resource Locator) from each terminal 10 and terminal 30 and logging in. Here, the services provided by the monitoring device 100 include monitoring the situation in a predetermined space where the subject T is present, and notifying a station staff member C or an operator U at the monitoring center depending on the situation.

[0049] <Hardware configuration example> Fig. 4 is a block diagram illustrating a hardware configuration of a computer 1000 that realizes the monitoring device 100 of Fig. 1. The subject's mobile terminal 10, the wearable terminal 20, and the station staff terminal 30 of Fig. 3 are also realized by the computer 1000. The functions of the monitoring device 100 may be shared and realized by a computer 1000 that realizes the monitoring device 100 and a computer 1000 that realizes the subject's mobile terminal 10 or the station staff terminal 30. There may be multiple computers 1000 that realize each device and terminal.

[0050] The computer 1000 includes a bus 1010 , a processor 1020 , a memory 1030 , a storage device 1040 , an input / output interface 1050 , and a network interface 1060 .

[0051] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0052] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0053] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0054] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the monitoring device 100 (e.g., the biometric information acquisition unit 102, the image acquisition unit 104, the determination unit 106, the output processing unit 108, and the estimation unit 110 and identification unit 112 described below). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also store each data of the storage device 130 of the monitoring device 100.

[0055] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded may include a non-transitory, tangible medium usable by the computer 1000, and the program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

[0056] The input / output interface 1050 is an interface for connecting the computer 1000 with various input / output devices. The input / output interface 1050 also functions as a communication interface for performing short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication).

[0057] The network interface 1060 is an interface for connecting the computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method for connecting the network interface 1060 to the communication network may be a wireless connection or a wired connection.

[0058] The computer 1000 is then connected to necessary equipment (for example, a display device 120, headphones 50, camera 5, in-car speaker 7 connected to the monitoring device 100, or the display, operation unit (or touch panel), camera, speaker, microphone of the subject's mobile terminal 10 or the station staff terminal 30, the display, operation unit (or touch panel), speaker, microphone of the wearable terminal 20, various sensors for acquiring biometric information, etc.) via the input / output interface 1050 or the network interface 1060.

[0059] The components of the monitoring device 100 in each embodiment shown in Figure 1 and in Figures 7, 8, and 10 described below are realized by any combination of hardware and software of the computer 1000 in Figure 4. Those skilled in the art will understand that there are many variations in the realization methods and devices. The functional block diagrams showing the monitoring device 100 in each embodiment show logical functional blocks rather than a hardware-based configuration.

[0060] <Example of functional configuration> The functions of the monitoring device 100 will be described below with reference to FIG. The biometric information acquisition unit 102 acquires biometric information of a subject T who is in a predetermined space. As described above, in the embodiment, the predetermined space is the interior of a moving vehicle 60. The biometric information includes at least one of the subject T's heart rate and sweat rate. The biometric information acquisition unit 102 acquires the subject T's biometric information from the subject's portable terminal 10 worn by the subject T.

[0061] The timing at which the biometric information acquisition unit 102 acquires the biometric information is, for example, at least one of the following: when the subject T specifies the start of use of the monitoring system 1 service (for example, when the subject T launches an application on the subject's mobile terminal 10, or when the subject specifies the start of use by pressing a start use button on the application); when it is detected that the subject T has entered the mobile vehicle 60; and at any timing while the subject T is inside the mobile vehicle 60.

[0062] There are various possible methods for detecting that the subject T has boarded the moving vehicle 60 and is still in the vehicle, including, but not limited to, the following examples. For example, a general check-in detection technique for a predetermined area can be used. Note that a combination of the following methods may also be used. (a1) Using a short-range wireless communication function such as Bluetooth (registered trademark) of the subject's portable terminal 10 of the subject T, communication with a wireless communication device (not shown) installed in the moving vehicle 60 is detected. (a2) Using a wireless communication function such as a wireless LAN (Local Area Network) of the subject's portable terminal 10 of the subject T, communication with a wireless LAN access point (not shown) installed in the moving vehicle 60 is detected. (a3) A predetermined sound wave output from a sound wave generator (not shown) installed in the moving vehicle 60 is received by a microphone (not shown) of the subject's portable terminal 10.

[0063] An application for detecting check-in to a predetermined area by a method such as (a1) to (a3) ​​is installed and activated on the portable terminal 10 for the subject. The portable terminal 10 for the subject specifies the check-in area of ​​the portable terminal 10 for the subject by notifying the monitoring device 100 from the application of the detection of communication or reception of sound waves.

[0064] Further, other examples include the following: (a4) Using a camera 5 that takes pictures inside the moving vehicle 60, the boarding of the subject T is detected by comparing the subject T with the pre-registered biometric authentication information of the subject T. Alternatively, the boarding position of the subject T is identified by tracking the boarding position of the subject T using an image captured by a camera (not shown) installed in the station. (a5) The location registration information of the target person's portable terminal 10 with the base station of the mobile phone communication network is used. (a6) The current location of the portable terminal 10 for the subject, which is identified by the GPS (Global Positioning System) receiving function of the portable terminal 10 for the subject, is used.

[0065] The image acquisition unit 104 acquires an image including the subject and his / her surroundings from the camera 5 installed in the moving vehicle 60. The timing for the image acquisition unit 104 to acquire an image is at least when the determination unit 106 obtains a determination result that the biometric information satisfies the first criterion.

[0066] The determination unit 106 determines whether the biometric information of the subject T acquired by the biometric information acquisition unit 102 satisfies a first criterion and whether the state of other people around the subject T satisfies a second criterion.

[0067] The first criterion is, for example, that at least one of the subject T's biometric information, that is, the heart rate and the amount of sweating, exceeds a threshold. In other words, the first criterion is a value of the biometric information that indicates that the subject T is in a state of tension or fear. Alternatively, the first criterion may include the biometric information changing from a constant state (steady state) to a value that indicates a sudden change. For example, the first criterion may include the fluctuation range of the biometric information within a predetermined time period exceeding a threshold. Furthermore, the first criterion may further include a condition that the state that satisfies the first criterion continues for a predetermined period of time or longer.

[0068] In this way, the biometric information acquisition unit 102 acquires at least one of the heart rate and the amount of sweat as biometric information of the subject T, and the determination unit 106 can perform a determination process, thereby estimating and accurately detecting the state of the subject T when he or she becomes a victim. In situations where the subject T is victimized, the subject T is often unable to report the incident himself or herself. Furthermore, in order to acquire the biometric information of the assailant, a separate sensor must be installed in the vehicle, as will be described in the embodiment below. Therefore, for example, women and minors who are likely to be victimized can wear the wearable device 20 and use the services of the monitoring system 1, thereby allowing the biometric information acquisition unit 102 to acquire the biometric information of the subject T, thereby enabling the occurrence of an incident to be detected accurately and efficiently.

[0069] The determination unit 106 identifies the state of other people around the subject T by performing image analysis processing on the image acquired by the image acquisition unit 104.

[0070] Around subject T position The other person may be, for example, a person who exists in the same area as the target person T when a predetermined space is divided into a plurality of areas. There are various methods for acquiring the location information of the target person T. For example, the above-mentioned (a1) to (a3) ​​can be used to detect check-ins in a general predetermined area, or the methods for acquiring current location information (a4) to (a6) can be used. The function for acquiring the location information of the target person T can be realized by the identification unit 112 of the third embodiment, which will be described later.

[0071] An example of the second criterion is that the degree of congestion in a predetermined space is equal to or greater than a reference value. For example, the determination unit 106 counts the number of passengers inside the vehicle using an image of the interior of the moving vehicle 60. As one example, the image acquisition unit 104 acquires an image of the entrance / exit door of the moving vehicle 60 using the camera 5, and the determination unit 106 performs image analysis processing to count the number of people who have boarded or alighted, and calculates the number of people inside the moving vehicle 60. As another example, the image acquisition unit 104 acquires an image of the interior of the moving vehicle 60 using the camera 5, and the determination unit 106 performs image processing to detect and count the heads of passengers, and calculates the number of people inside the moving vehicle 60.

[0072] The determination unit 106 may divide the calculated number of passengers by the area of ​​the specified space to convert it into the number of passengers per unit area, and use this as the degree of congestion. The determination unit 106 may calculate the occupancy rate from the calculated number of passengers and the number of passengers that can be accommodated by the moving vehicle 60, and use this as the degree of congestion.

[0073] In another example, the determination unit 106 may determine whether the degree of congestion in a partial area within a predetermined space around the target person T is equal to or greater than a reference value. For example, the determination unit 106 may divide the moving vehicle 60 into multiple areas, identify the degree of congestion or the number of passengers for each area, and determine that the degree of congestion is equal to or greater than a reference value if the degree of congestion or the number of passengers in the area where the target person T is located is higher than the other areas by a certain percentage or more.

[0074] In particularly crowded situations, the second criterion preferably includes subject T being surrounded by multiple people. The determination unit 106 performs image analysis processing on the image acquired by the image acquisition unit 104 to identify the formation of multiple people present around the target person T. As shown in Fig. 5, for example, the determination unit 106 identifies whether the formation is a circle or a fan-shaped formation in which the target person T (shown as a black circle in the figure) is surrounded by multiple other people P (shown as white circles in the figure).

[0075] The output processing unit 108 outputs predetermined information when the formation of other persons P around the target person T is circular or fan-shaped.

[0076] In this way, the judgment unit 106 performs judgment processing using the degree of congestion in a specified space being equal to or greater than a reference value as a second criterion, and can detect the occurrence of an incident, thereby efficiently detecting crimes and nuisance acts committed by a group of perpetrators surrounding the target T.

[0077] Furthermore, the determination unit 106 may use image processing to identify the directions in which multiple persons P around the subject T are facing, calculate the proportion of persons P facing the direction of the subject T, and determine whether or not the proportion is equal to or greater than a reference value. The output processing unit 108 may output predetermined information when it is determined that the proportion of persons P facing the direction of the subject T is equal to or greater than a reference value.

[0078] In yet another example, the second criterion may include that the distance between the subject T and the other person P is equal to or less than a reference value. The judgment unit 106 may perform image analysis processing on the image acquired by the image acquisition unit 104 to measure the distance between the subject T and another person P from their positional information, and determine that the distance is below a reference value, i.e., that the subject T and person P are close to each other.

[0079] In this way, the judgment unit 106 performs judgment processing using the distance between the subject T and another person P being equal to or less than a reference value as the second criterion, and can detect the occurrence of an incident, thereby accurately detecting situations in which an assailant is directly harming the subject T.

[0080] When it is determined that both the first criterion and the second criterion are satisfied, the output processing unit 108 outputs predetermined information. For example, when it is determined that both the first criterion and the second criterion are satisfied, it is determined that the occurrence of a predetermined incident has been identified, and the output processing unit 108 outputs information notifying the occurrence of the incident as the predetermined information.

[0081] The predetermined information includes, for example, at least one of the type of incident that occurred, the date and time of the incident, and the location of the incident (for example, the route name and destination, the section (such as the station name), the vehicle location (vehicle number, door number), etc.), and the image used to identify the occurrence of the incident. Furthermore, the predetermined information includes a message notifying the occurrence of the incident.

[0082] The output destination of the output processing unit 108 includes at least one of the display device 120, the station staff terminal 30, the target person's mobile terminal 10, the in-car speaker 7, and the station speaker 40. The predetermined information to be output may be changed depending on the output destination. The output form includes at least one of audio output, alarm sound output, image display, message display, lamp lighting, and rotating light drive.

[0083] <Example of operation> FIG. 6 is a flowchart showing an example of the operation of the monitoring system 1 according to the embodiment. This flow is assumed to be started when it is detected that the subject T has entered the vehicle 60. However, the timing of starting the flow is not limited to this, and may be any of the timings for acquiring the biometric information described above.

[0084] First, the biometric information acquiring unit 102 of the monitoring device 100 requests the biometric information of the subject T from the subject's portable terminal 10. The subject's portable terminal 10 of the subject T periodically communicates with the wearable terminal 20 to acquire the biometric information of the subject T (step S201). In response to the request from the monitoring device 100, the subject's portable terminal 10 transmits the biometric information acquired from the wearable terminal 20 to the monitoring device 100, and the biometric information acquiring unit 102 of the monitoring device 100 acquires the biometric information of the subject T from the subject's portable terminal 10 (step S203).

[0085] Then, the determination unit 106 determines whether the acquired biometric information of the subject T satisfies a first criterion (step S205). For example, if the heart rate of the subject T exceeds a threshold, and it is determined that the biometric information satisfies the first criterion (YES in step S205), the image acquisition unit 104 acquires an image of a predetermined space from the camera 5 (step S207).

[0086] Then, the determination unit 106 determines whether the state of other persons P located around the target person T satisfies a second criterion (step S209). For example, the determination unit 106 performs image analysis processing on the image acquired in step S207 to count the number of other persons P present in the same area as the target person T, and determines whether the number of other persons P is equal to or greater than a reference value. If the number of other persons P is equal to or greater than the reference value (YES in step S209), the output processing unit 108 transmits predetermined information to at least one of the display device 120 of the monitoring center and the station staff terminal 30 (step S211) and causes it to be output (step S213).

[0087] As described above, according to this embodiment, in the monitoring device 100, the biometric information acquisition unit 102 acquires biometric information of a subject present in a specified space, the image acquisition unit 104 acquires an image including the subject and his or her surroundings, the determination unit 106 determines whether the biometric information satisfies a first criterion and whether the state of other people present around the subject satisfies a second criterion, and the output processing unit 108 outputs specified information when it is determined that both the first criterion and the second criterion are satisfied.

[0088] With this configuration, an incident can be detected based on the subject's biometric information and the status of others around the subject, and the occurrence of the incident can be reported by outputting predetermined information. In other words, by detecting abnormalities in the subject from the subject's biometric information and detecting abnormalities in the status of others around the subject, the occurrence of the incident can be more accurately estimated. This monitoring device 100 can solve the problem of improving the accuracy of detecting and reporting the occurrence of an incident within a specified space.

[0089] (Second embodiment) <Example of functional configuration> Fig. 7 is a functional block diagram showing an example of the logical configuration of a monitoring device 100 according to an embodiment. This embodiment is similar to the above-described embodiment except that it has a configuration in which attributes of a subject T and another person P are estimated by image processing. The monitoring device 100 shown in this figure further includes an estimation unit 110 in addition to the configuration of the monitoring device 100 in Fig. 1. However, the configuration of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that no contradiction occurs.

[0090] The estimation unit 110 uses image processing to estimate the attributes of the target person T and the other person P. The second criterion includes a condition regarding the attributes. The determining unit 106 determines, based on the estimation result of the estimating unit 110, whether or not the second criterion is satisfied.

[0091] The attributes include conditions regarding at least one of the gender and age group of the subject T and at least one of the other persons P.

[0092] Examples of conditions related to attributes include, but are not limited to, the following. In addition, a combination of the following may be used. (b1) The gender of the subject T is different from that of another person P. (b2) The rate of identity between subject T and other person P is equal to or greater than the reference value. (b3) The age of the subject T is equal to or less than the first standard value, and the age of the other person P is equal to or greater than the first standard value or a second standard value that is older than the first standard value. Here, the first reference value may be, for example, 6 years old, with the target person T being a toddler, and the second reference value may be 18 years old, with the other person P being a juvenile or older.

[0093] Furthermore, even if the subject T and the other person P are of the same sex, if another second criterion is met, for example, if the distance between the subject T and the other person P is less than a reference value, the output processing unit 108 can output the specified information by satisfying the second criterion.

[0094] The estimation unit 110 may further estimate the relationship between the subject T and another person P by image processing. The estimation unit 110 may estimate whether the subject T and another person P are family members based on, for example, the movements of the subject T and the other person P and whether or not they are talking. For example, the estimation unit 110 may estimate that the subject T and the other person P are family members when it is detected that the subject T and the other person P are having a calm conversation or making a movement such as patting each other's heads.

[0095] Furthermore, the actions used to estimate that a person is not a family member may include the subject T raising both hands (holding up), the other person P holding a weapon, brandishing it, or holding it ready, the other person P using violence against the subject T, the other person P holding down the subject T, embracing the subject T, pulling, or dragging the subject T, etc. When such an action is estimated, the estimation unit 110 can estimate that the person is not a family member.

[0096] Furthermore, the estimation unit 110 may estimate the emotion of at least one of the subject T and the other person P through image processing. The estimation unit 110 may detect the movement, posture, facial expression, sweating, complexion, physical condition (trembling), etc. through image processing to estimate the emotion of each person.

[0097] For example, when it is estimated that at least one of the emotions of the subject T indicates fear, anger, sadness, discomfort, etc., and the emotions of the other person P indicates excitement, anger, rage, intimidation, etc., the judgment unit 106 judges that the second criterion is met.

[0098] For example, a score may be set for each emotion, and the determination unit 106 may determine whether or not the emotion score exceeds a reference value.

[0099] As described above, the monitoring device 100 of this embodiment includes the estimation unit 110 that estimates the attributes of the target person T and the other person P through image processing. The second criterion includes a condition related to the attributes, and the determination unit 106 determines whether the second criterion is satisfied based on the estimation result of the estimation unit 110.

[0100] In this way, the monitoring device 100 of this embodiment not only achieves the same effects as the above embodiment, but also estimates the attributes of the subject T and other person P, and adds attribute-related conditions to the process of determining whether an incident has occurred, making it possible to detect the occurrence of an incident more accurately.

[0101] (Third embodiment) <Example of functional configuration> Fig. 8 is a functional block diagram showing an example of the logical configuration of a monitoring device 100 according to an embodiment. This embodiment is similar to the second embodiment, except that it has a configuration for identifying the position of the subject T by image processing. The monitoring device 100 shown in this figure further includes an identification unit 112 in addition to the configuration of the monitoring device 100 in Fig. 7. However, the configuration of this embodiment may be combined with at least one of the configurations of other embodiments other than the second embodiment, as long as no contradiction occurs.

[0102] When detecting a subject T whose biometric information is determined to satisfy a first criterion, the identification unit 112 identifies the position of the subject. The estimation unit 110 acquires an image including the identified position and performs image processing.

[0103] The identification unit 112 can identify the location of the subject T by using the above-mentioned (a1) to (a3), general check-in detection techniques for a specified area, or by using the methods of acquiring current location information (a4) to (a6).

[0104] <Example of operation> FIG. 9 is a flowchart illustrating an example of a location specification process performed by the monitoring device 100 according to the embodiment. This process is started when a subject T whose biometric information satisfies the first criterion is detected in step S205 of the flowchart in Fig. 6 (YES in step S205). The identification unit 112 identifies the position of the subject determined to satisfy the first criterion (step S301).

[0105] The estimation unit 110 requests and acquires an image including the identified position from the camera 5 (step S303). Control of the camera 5 will be described later in the fourth embodiment. The estimation unit 110 processes the image acquired from the camera 5 to estimate the attributes of the subject T and the other person P (step S305).

[0106] As described above, the monitoring device 100 of this embodiment includes an identification unit 112 that, when detecting a subject T whose biological information is determined to satisfy the first criterion, identifies the position of the subject T. Then, the estimation unit 110 acquires an image including the identified position and performs image processing.

[0107] In this way, the monitoring device 100 of this embodiment not only achieves the same effects as the above-described embodiment, but also makes it possible to identify the position of the subject T, and then acquire an image focused on other persons P present around the subject T, which can be used for attribute estimation processing and incident occurrence determination processing, thereby improving the accuracy of incident occurrence detection.

[0108] (Fourth embodiment) <Example of functional configuration> Fig. 10 is a functional block diagram showing an example of the logical configuration of a monitoring device 100 according to an embodiment. This embodiment is similar to the third embodiment, except that it has a configuration for controlling a camera. The monitoring device 100 shown in this figure further includes a camera control unit 114 in addition to the configuration of the monitoring device 100 shown in Fig. 8. However, the configuration of this embodiment may be combined with at least one of the configurations of other embodiments other than the third embodiment, as long as no contradiction occurs.

[0109] The camera control unit 114 directs the camera 5 toward the position identified by the identification unit 112 and causes the camera 5 to generate an image. The estimation unit 110 acquires the generated image and performs image processing.

[0110] <Switching image processing methods based on congestion level> In the above embodiment, the determination unit 106 calculates the degree of congestion in a predetermined space. The camera control unit 114 may be configured to change the image generated by controlling the camera 5 using the degree of congestion.

[0111] For example, the accuracy of image processing of images captured by camera 5 changes depending on whether the area is crowded or not. When the area is not crowded, it is possible to zoom in on a specific person and analyze their facial expression to estimate their emotion. Emotion estimation is as described in the second embodiment. On the other hand, when the area is crowded, it becomes difficult to zoom in on a specific person and analyze their facial expression. This is because there is a large amount of overlap between people, which reduces the image area that can be acquired for a specific person, thereby reducing the amount of information that can be obtained, and the accuracy of the analysis results obtained by image processing decreases.

[0112] Therefore, when the degree of congestion in the predetermined space is equal to or greater than a reference value, the determination unit 106 performs a determination process using, as a second criterion, whether the subject T is surrounded by multiple people through image processing. At this time, the camera control unit 114 controls the camera 5 to generate an image with a bird's-eye view angle of view so that the formation of the multiple people in the predetermined space can be identified.

[0113] On the other hand, if the degree of congestion in the predetermined space is less than the reference value, the estimation unit 110 estimates the attributes of the subject T and the other persons P through image processing, and the determination unit 106 performs determination processing using conditions related to the attributes as a second criterion. At this time, the camera control unit 114 controls the camera 5 to point and focus the camera 5 in the direction of the subject T and the other persons P in the predetermined space, and to zoom in as necessary. This allows the image acquisition unit 104 to acquire high-resolution images, thereby improving the estimation accuracy achieved by the image processing of the estimation unit 110.

[0114] As described above, the monitoring device 100 of this embodiment includes a camera control unit 114 that directs the camera 5 toward the position identified by the identification unit 112 and causes the camera 5 to generate an image. The estimation unit 110 acquires the generated image and performs image processing. In this way, the monitoring device 100 of this embodiment achieves the same effects as the above-mentioned embodiment, and furthermore, by controlling the camera 5 using the camera control unit 114, it is possible to switch the images generated depending on the situation, and therefore it is possible to perform image processing using images suitable for judgment processing and estimation processing, thereby improving the accuracy of detecting incident occurrence.

[0115] (Fifth embodiment) <Example of functional configuration> This embodiment is the same as any of the above-described embodiments, except that it has a configuration in which warning information is output before the output processing unit 108 outputs predetermined information. The following description will be given using the configuration of the fourth embodiment in Fig. 10, but the configuration of this embodiment may be combined with at least one of the configurations of other embodiments other than the fourth embodiment, as long as no contradiction occurs.

[0116] Before outputting the specified information, the output processing unit 108 outputs warning information toward a specified space, and after outputting the warning information, if the state satisfying the first criterion and the second criterion continues, outputs the specified information.

[0117] Whether or not the state that satisfies the first and second criteria continues can be determined, for example, by repeating the determination at predetermined intervals from the time when it is determined that at least one of the first and second criteria is satisfied, and then determining that the first and second criteria are satisfied after a predetermined time has elapsed.

[0118] The warning information includes an audio message such as an announcement output from the interior speaker 7 installed inside the moving vehicle 60. The warning information is a message that calls attention to people in a specified space. The message preferably includes at least one of content that causes the perpetrator to hesitate or give up on the nuisance or criminal act, and content that prompts people around the victim to recognize the occurrence of the incident.

[0119] Examples of messages include, "Nuisance acts such as molestation and violent acts are crimes," "If you witness harassment or violent acts such as molestation, please press the report button on the train to report to the conductor, or notify a nearby security guard or railway police officer," "Knives, dangerous objects, items that give off an odor, items that may harm other passengers, or items that may damage the interior of the train are not permitted on board," and "If you see a foreign or suspicious object on the train, please notify a nearby station staff member."

[0120] Another example of the message may be a message informing people in a specified space of a situation where an incident is suspected or has already occurred. The message may include, for example, "There appears to be trouble on board the train. Please remain calm and follow the conductor's instructions," or "An emergency has occurred. Please wait where you are until the conductor gives instructions."

[0121] In other words, the predetermined information is information intended to notify the outside world of the occurrence of an incident and request rescue, whereas the warning information is information intended to make the assailant stop or give up their behavior or to notify people around them in a predetermined space where an incident is predicted or occurring. In this way, the output processing unit 108 can switch the information to be output in stages depending on the situation.

[0122] The predetermined space is the interior of the vehicle, and the output processing unit 108 outputs the predetermined information to at least one of the driver and manager of the vehicle 60. The administrator includes, for example, the police, security companies, staff of management facilities, etc., and operators of monitoring centers. The driver includes station staff C and conductors.

[0123] The output processing unit 108 can select and output warning information and predetermined information in stages according to the situation. The output patterns are exemplified below, but are not limited to these. In addition, a plurality of the following output patterns may be combined. (c1) Announcement of warning information inside the moving vehicle 60. For example, at least one of audio output from the in-vehicle speaker 7 installed in the vehicle and message display on a display (not shown) inside the vehicle is performed. (c2) Predetermined information is output to the driver or conductor of the moving vehicle 60 and the station staff C of the nearest station. For example, at least one of the following is performed: outputting a voice or an alarm sound to the in-car speaker 7 installed in the driver's seat, displaying a message on the display installed in the driver's seat, and turning on a lamp. (c3) Predetermined information is output to the operator at the monitoring center. For example, at least one of the following is performed: displaying an image or message on the display device 120 at the monitoring center, lighting a lamp on the monitoring panel, outputting voice or alarm sound from a speaker installed at the monitoring center, driving a rotating light installed at the monitoring center, and outputting voice or alarm sound to the headphones 50 worn by the operator U. (c4) Predetermined information or warning information is output to the station staff terminal 30 carried by station staff C or the conductor. (c5) Output warning information to the mobile terminals of other users other than the victim. The other users are people who have registered to use the services of the monitoring system 1 and who are in the same moving vehicle 60. (c6) Report to a manager, including the police, a security company, a staff member of a management facility, etc. For example, at least one of sending information about the occurrence of the incident to a mobile terminal carried by the manager and activating a rotating light installed in the facility is performed.

[0124] <Example of operation> FIG. 11 is a flowchart illustrating an example of the output process of the monitoring apparatus 100 according to the embodiment. The process in this figure is started when the determination unit 106 determines in step S205 of Fig. 6 that the biometric information of the subject T satisfies the first criterion, and when the determination unit 106 determines in step S209 that the state of another person P around the subject T satisfies the second criterion. Furthermore, this process may be repeated (return from step S409 to step S401) at a predetermined cycle (for example, at one-minute intervals).

[0125] When this process starts, the output processing unit 108 outputs warning information toward a predetermined space (step S401). For example, the output processing unit 108 outputs a voice message from the in-vehicle speaker 7 to alert passengers, or displays the warning information on a display in the vehicle.

[0126] 6 (step S403), and if the result of the determination is that the first criterion and the second criterion are satisfied (YES in step S405), the determination unit 106 increments the counter i (step S407). Here, in the first case, the counter i is set to 1.

[0127] Then, the determination unit 106 determines whether the counter i exceeds a predetermined value N (step S409). The predetermined value N is a value for determining whether the determination processes of steps S205 and S209 in FIG. 6 have been repeated a predetermined number of times, N times. If the counter i is equal to or less than the predetermined value N (NO in step S409), the process returns to step S401, and the output processing unit 108 outputs the warning information toward a predetermined space. Alternatively, the process may return to step S403.

[0128] Then, the determination process is repeated, and if the counter i exceeds the predetermined value N in step S409 (YES in step S409), the process proceeds to step S213 in FIG. 6, where the output processing unit 108 outputs predetermined information.

[0129] On the other hand, if it is determined in step S405 that at least one of the first criterion and the second criterion is not satisfied (NO in step S405), the counter i is reset and the process ends. At this time, if the output of warning information by the output processing unit 108 is continuing (for example, displaying on a display), the output of the warning information is stopped.

[0130] As described above, in the monitoring device 100 of this embodiment, the output processing unit 108 outputs warning information toward a predetermined space before outputting predetermined information, and after outputting the warning information, if the state satisfying the first criterion and the second criterion continues, outputs the predetermined information. In this way, the monitoring device 100 of this embodiment not only achieves the same effects as the above-described embodiment, but also can switch the output information in stages according to the conditions of a given space, thereby improving the effectiveness of the output information.

[0131] First, warning information sent inside the vehicle will be used to deter criminal acts, and if incidents continue to be detected, an alert will be sent to the police or security companies, requesting their response, thereby reducing false alerts such as requests for response to the police or security companies.

[0132] (Sixth embodiment) <Example of functional configuration> This embodiment is the same as any of the above-described embodiments, except that the biometric information acquisition unit 102 further acquires biometric information of a person P other than the subject T and performs determination processing using the biometric information of the other person P. The following description will be given using the configuration of the fourth embodiment in Fig. 10, but the configuration of this embodiment may be combined with at least one of the configurations of other embodiments other than the fourth embodiment as long as no contradiction occurs.

[0133] The biometric information acquisition unit 102 further acquires biometric information of other people. The determination unit 106 determines whether or not the biometric information of the other person satisfies a third criterion. The output processing unit 108 outputs predetermined information when the biometric information of the other person satisfies the third criterion.

[0134] Examples of other persons P include the perpetrator who is harming the subject T, and third parties who are people around the subject T other than the perpetrator.

[0135] For example, sensors for detecting biometric information may be built into straps, handrails, etc. of the moving vehicle 60, and the biometric information acquisition unit 102 may acquire biometric information of other people from these sensors.

[0136] For example, by acquiring the assailant's biometric information, it is possible to detect the assailant's state of tension or excitement. If it is possible to detect the assailant's state of tension before committing a crime, it is possible to output warning information such as a message to discourage the assailant from committing a crime in advance.

[0137] Furthermore, by acquiring biometric information of a third party, it is possible to detect the state of tension or excitement of people in the vicinity. The determination unit 106 may count the number of people in the vicinity whose biometric information meets a first criterion, and determine that an incident has occurred when the number of people exceeds a threshold. This makes it possible to more accurately determine the occurrence of an incident.

[0138] As described above, in the monitoring device 100 of this embodiment, the biometric information acquisition unit 102 further acquires biometric information of other people, the determination unit 106 determines whether the biometric information of other people satisfies the third criterion, and the output processing unit 108 outputs predetermined information when the biometric information of other people satisfies the third criterion. In other words, since the determination process can be performed based on the biometric information of person P around the target person T, it is possible to detect a state of tension or excitement before committing a crime.

[0139] In this way, the monitoring device 100 of this embodiment not only achieves the same effects as the above-mentioned embodiments, but also can perform judgment processing based on the biometric information of the person P around the subject T, making it possible to detect the state of tension or excitement that the perpetrator may be in before committing a crime, predict the occurrence of an incident such as a crime in advance, and output warning information or specified information.

[0140] Furthermore, since the state of people around the perpetrator can be detected, the accuracy of determining whether an incident has occurred can be further improved.

[0141] Seventh embodiment <Example of functional configuration> FIG. 12 is a partial functional block diagram showing the configuration of the main part of the monitoring device 100 according to the embodiment. This embodiment is the same as any of the above-described embodiments, except that it has a configuration in which information about incidents detected by the determination unit 106 is accumulated and various reference values ​​and thresholds used in the determination process are adjusted. Hereinafter, the configuration of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that no contradiction occurs.

[0142] The monitoring device 100 further includes a storage processing unit 116 and an adjustment unit 118 . The determination unit 106 detects that an incident has occurred when it is determined that the first criterion and the second criterion are satisfied. The storage processing unit 116 records incident information 200 in the storage device 130 when the determination unit 106 detects the occurrence of an incident.

[0143] 13 and 14 are diagrams showing examples of data structures of various information indicating the occurrence status of an incident. The incident information 200 in FIG. 13(a) includes, for example, identification information (subject ID) of the subject T in whom the occurrence of the incident was detected, the date and time of the incident, information on the location of the incident, biometric information of the subject T that was the basis for detecting the occurrence of the incident, and information such as the congestion level indicating the state of the other person P and the distance between the subject T and the other person P that was the basis for detecting the occurrence of the incident. However, FIG. 13(a) is an example and is not limited to this. For example, the incident information 200 may be stored in association with image data at the time the occurrence of the incident was detected.

[0144] The storage processing unit 116 may store the incident information 200 as accumulated information 210 so that the adjustment unit 118 can adjust various reference values ​​and thresholds. (b) of FIG. 13 is a diagram showing an example of the data structure of the accumulated information 210. The accumulated information 210 includes the date and time of the incident, identification information of the subject T, the name of the route on which the incident occurred and location information, the type of the incident (groping, theft, violence, kidnapping, etc.), and the level of the incident. (b) of FIG. 13 is an example and is not limiting.

[0145] The level of an incident is information indicating, for example, the urgency or danger of the incident. The levels may be classified into low urgency levels such as a scuffle between passengers, high danger levels such as violent acts, and high urgency and danger levels such as the occurrence of a crime such as kidnapping, robbery, or murder.

[0146] The storage processing unit 116 may analyze the incident information 200 and store it as stored information 210 in the storage device 130 .

[0147] The adjustment unit 118 analyzes the occurrence of the incident based on the accumulated information 210. The storage processing unit 116 stores the results of the analysis by the adjustment unit 118 as analysis information 220 in the storage device 130.

[0148] Fig. 14 is a diagram showing an example of the data structure of the analysis information 220. The analysis information 220 includes, for example, the incident type, incident level, line name, time period when the incident occurred, day of the week, congestion level, frequency of the most recent incident, temperature at the time of occurrence, and location of occurrence (position of the vehicle (which car), position within the vehicle, for example, near the door, corner of the vehicle, etc.). Fig. 14 is an example, and is not limited to this.

[0149] The adjustment unit 118 may obtain the temperature at the time of the incident, for example, the temperature of the vehicle from a temperature sensor installed in the vehicle, or may obtain the temperature of the area where the incident occurred from weather information at the time of the incident.

[0150] <Adjustment of reference values> The adjustment unit 118 adjusts various reference values ​​and thresholds based on the analysis information 220. Optimal reference values ​​and thresholds may be identified and adjusted for each season or region. Optimal reference values ​​and thresholds may also be identified and adjusted according to the time of day or day of the week. Furthermore, for each type of incident, conditions under which an incident is likely to occur may be identified, and optimal reference values ​​and thresholds may also be identified and adjusted.

[0151] The adjustment process for various reference values ​​and threshold values ​​may be configured to display information regarding the analysis information 220 on the screen of the display device 120 and present it to the operator U, and also to display a screen on the display device 120 that accepts adjustments from the operator U and accepts the setting of adjustment values.

[0152] As described above, in the monitoring device 100 of this embodiment, the adjustment unit 118 analyzes the occurrence status of an incident based on the accumulated information 210. The storage processing unit 116 stores the results of the analysis by the adjustment unit 118 in the storage device 130 as analysis information 220. Then, the adjustment unit 118 adjusts various reference values ​​and thresholds based on the analysis information 220.

[0153] As described above, the monitoring device 100 of this embodiment can identify and adjust optimal reference values ​​and thresholds depending on the conditions, thereby enabling more accurate detection of the occurrence of an incident.

[0154] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted. For example, the subject of the monitoring method of the above embodiment is not limited to the computer 1000 of the monitoring device 100. At least one of the camera 5, the subject's mobile terminal 10, and the computer 1000 of the monitoring device 100 may realize each function of the monitoring device 100. At least two of the camera 5, the subject's mobile terminal 10, and the computer 1000 of the monitoring device 100 may be combined to share and realize each function of the monitoring device 100.

[0155] Alternatively, the computer 1000 of the camera 5 may have the function of communicating directly with the subject T's mobile terminal 10, and may realize the biometric information acquisition unit 102, image acquisition unit 104, judgment unit 106, output processing unit 108, estimation unit 110, and identification unit 112. Alternatively, the computer 1000 of the subject's mobile terminal 10 may have the function of communicating with the camera 5 and may realize the biometric information acquisition unit 102, the image acquisition unit 104, the judgment unit 106, the output processing unit 108, the estimation unit 110, and the identification unit 112.

[0156] <Monitoring examples for nurseries and elderly care facilities> In addition, if the specified space is a nursery school, a facility for the elderly, etc., the monitoring device 100 may detect incidents such as fights between children or residents, or abuse of children or residents by staff, and alert staff, parents, or family members.

[0157] A child or resident wears a wearable device 20 or a device for acquiring biometric information including a sensor for acquiring biometric information, and the biometric information acquisition unit 102 acquires the biometric information from the device via a tablet device, personal computer, or smartphone, or directly from the device. The camera 5 is installed so that it can capture images of the area to be monitored within the facility.

[0158] The judgment unit 106 may determine whether the state in which the biometric information satisfies a first criterion (at least one of the heart rate and the amount of sweat exceeds a standard value) and the state between the subject T and the other person P satisfies a second criterion (the distance between the subject T and the other person P is closer than the standard value) has continued for a certain period of time.

[0159] If the above state continues for a certain period of time, the output processing unit 108 may output predetermined information. In this manner, the determination unit 106 may determine whether the state that satisfies the first criterion and the second criterion has continued for a certain period of time.

[0160] The output processing unit 108 outputs predetermined information to a mobile device (for example, a smartphone or tablet device) carried by a staff member, guardian, or family member. The output processing unit 108 may drive a rotating light installed in the facility, output voice or warning sound to a speaker, and display images and messages on a display.

[0161] Destination information (e.g., mobile phone number or email address) of a mobile device carried by a staff member, guardian, or family member may be registered in advance, and the output processing unit 108 may send a message containing predetermined information to that destination. Alternatively, an application for using the services of the monitoring system 1 may be installed in advance on the mobile device, launched, and logged into the service, allowing the mobile device to obtain and output predetermined information from the monitoring device 100.

[0162] Furthermore, it may be possible to notify the nearest police station or a manager in a predetermined department of a relevant government office. The output processing unit 108 may transmit information about the occurrence of the incident to a mobile terminal carried by the manager. The transmission method is the same as that for the mobile terminal of a family member or the like described above. Alternatively, it may be possible to activate a rotating light installed in a police station or a predetermined department of a relevant government office, output a voice or warning sound to a speaker, or display an image or message on a display.

[0163] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0164] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In the present invention, when information about a user is acquired and used, it shall be done lawfully.

[0165] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. A biometric information acquisition means for acquiring biometric information of a subject in a predetermined space; image acquisition means for acquiring an image including the subject and its surroundings; a determination means for determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; an output processing means for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied. 2. In the monitoring device described in 1., A monitoring device, wherein the predetermined space is the interior of a moving vehicle. 3. In the monitoring device according to 1. or 2., The second criterion includes a degree of congestion in the specified space being equal to or greater than a reference value. 4. In the monitoring device according to any one of 1. to 3., The monitoring device, wherein the second criterion includes the subject being surrounded by multiple people. 5. In the monitoring device according to any one of 1. to 4., A monitoring device, wherein the second criterion includes that the distance between the subject and the other person is equal to or less than a reference value. 6. In the monitoring device according to any one of 1. to 5., The method further includes estimating means for estimating attributes of the subject and the other person through image processing, The second criteria include a condition regarding the attribute. 7. In the monitoring device according to 6., The attributes include conditions regarding at least one of the gender and age group of the subject and at least one of the others. 8. In the monitoring device according to 6. or 7., further comprising an identification means for identifying a position of the subject when the subject whose biological information is determined to satisfy the first criterion is detected, The estimation means acquires an image including the identified position and performs the image processing. 9. In the monitoring device according to 8., a camera control means for directing a camera to the specified position and causing the camera to generate the image; The estimation means acquires the generated image and performs the image processing. 10. In the monitoring device according to any one of 1. to 9., The biological information includes at least one of a heart rate and a sweat rate. 11. The monitoring device according to 10., The biometric information acquisition means acquires the biometric information of the subject from a wearable device worn by the subject. 12. In the monitoring device according to any one of 1. to 11., The output processing means outputting warning information toward the predetermined space before outputting the predetermined information; a monitoring device that outputs the predetermined information if the state that satisfies the first criterion and the second criterion continues after the output of the warning information; 13. The monitoring device according to 12., the predetermined space is the interior of a moving vehicle, The output processing means A monitoring device that outputs the predetermined information to at least one of a driver and a manager of the mobile vehicle. 14. In the monitoring device according to any one of 1. to 13., the biometric information acquisition means further acquires biometric information of the other person; the determining means determines whether the biometric information of the other person satisfies a third criterion; The output processing means outputs the predetermined information when the biological information of the other person satisfies the third criterion.

[0166] 15. An imaging means; a monitoring device; The monitoring device a biometric information acquisition means for acquiring biometric information of a subject in a predetermined space; image acquisition means for acquiring an image including the subject and its surroundings from the imaging means; a determination means for determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; an output processing means for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied. 16. In the monitoring system according to 15., A surveillance system, wherein the predetermined space is the interior of a moving vehicle. 17. In the monitoring system according to 15. or 16., A monitoring system, wherein the second criterion includes a degree of congestion in the specified space being equal to or greater than a reference value. 18. In the monitoring system according to any one of 15. to 17., The second criterion includes the subject being surrounded by multiple people. 19. In the monitoring system according to any one of 15. to 18., A surveillance system, wherein the second criterion includes a distance between the subject and the other person being equal to or less than a reference value. 20. In the monitoring system according to any one of 15. to 19., The monitoring device The method further includes estimating means for estimating attributes of the subject and the other person through image processing, The second criteria includes a condition regarding the attribute. 21. In the monitoring system according to 20., The attributes include conditions regarding at least one of the gender and age group of the subject and at least one of the others. 22. In the monitoring system according to 20. or 21., The monitoring device further comprising an identification means for identifying a position of the subject when the subject whose biological information is determined to satisfy the first criterion is detected, A surveillance system in which the estimation means of the surveillance device acquires an image including the identified position and performs the image processing. 23. In the monitoring system according to 22., The monitoring device a camera control unit that directs the image capturing unit to the specified position and causes the image capturing unit to generate the image; The estimation means of the monitoring device acquires the generated image and performs the image processing. 24. In the monitoring system according to any one of 15. to 23., A monitoring system in which the biological information includes at least one of heart rate and sweat rate. 25. In the monitoring system according to 24., A monitoring system in which the biometric information acquisition means of the monitoring device acquires the biometric information of the subject from a wearable device worn by the subject. 26. In the monitoring system according to any one of 15. to 25., In the monitoring device, The output processing means outputting warning information toward the predetermined space before outputting the predetermined information; a monitoring system that outputs the predetermined information if the state that satisfies the first criterion and the second criterion continues after the output of the warning information; 27. In the monitoring system according to 26., the predetermined space is the interior of a moving vehicle, In the monitoring device, The output processing means A monitoring system that outputs the predetermined information to at least one of a driver and a manager of the mobile vehicle. 28. In the monitoring system according to any one of 15. to 27., In the monitoring device, the biometric information acquisition means further acquires biometric information of the other person; the determining means determines whether the biometric information of the other person satisfies a third criterion; The output processing means outputs the predetermined information when the biological information of the other person satisfies the third criterion.

[0167] 29. Acquire biometric information of a subject in a specified space; acquiring an image including the subject and their surroundings; determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; a monitoring method that outputs predetermined information when it is determined that both the first criterion and the second criterion are satisfied. 30. In the monitoring method described in 29., A monitoring method, wherein the predetermined space is the interior of a moving vehicle. 31. In the monitoring method according to 29. or 30., A monitoring method, wherein the second criterion includes a degree of congestion in the specified space being equal to or greater than a reference value. 32. In the monitoring method according to any one of 29. to 31., A monitoring method, wherein the second criterion includes the subject being surrounded by multiple people. 33. In the monitoring method according to any one of 29. to 32., A monitoring method, wherein the second criterion includes that the distance between the subject and the other person is equal to or less than a reference value. 34. In the monitoring method according to any one of 29. to 33., Estimating attributes of the subject and the other person through image processing; The second criterion includes a condition regarding the attribute. 35. In the monitoring method described in 34., The monitoring method, wherein the attributes include conditions regarding at least one of the gender and age group of the subject and at least one of the others. 36. In the monitoring method according to 34. or 35., When detecting the subject whose biometric information is determined to satisfy the first criterion, the location of the subject is identified; A surveillance method comprising acquiring an image including the identified location and processing the image. 37. In the monitoring method according to 36., pointing a camera at the identified location and causing the camera to generate the image; A monitoring method comprising acquiring the generated image and performing the image processing. 38. In the monitoring method according to any one of 29. to 37., A monitoring method, wherein the biological information includes at least one of a heart rate and a sweat rate. 39. In the monitoring method according to 38., A monitoring method for acquiring the subject's biological information from a wearable device worn by the subject. 40. In the monitoring method according to any one of 29. to 39., outputting warning information toward the predetermined space before outputting the predetermined information; a monitoring method in which, after the warning information is output, if the state satisfying the first criterion and the second criterion continues, the predetermined information is output. 41. In the monitoring method according to 40., the predetermined space is the interior of a moving vehicle, The monitoring method includes outputting the predetermined information to at least one of a driver and a manager of the mobile vehicle. 42. In the monitoring method according to any one of 29. to 41., Further acquiring biometric information of the other person; determining whether the biometric information of the other person satisfies a third criterion; When the biological information of the other person satisfies the third criterion, the predetermined information is output.

[0168] 43. To the computer, A procedure for acquiring biometric information of a subject present in a predetermined space; acquiring an image including the subject and their surroundings; a step of determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; a program for executing a procedure for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied. 44. In the program described in 43., The program, wherein the predetermined space is the interior of a moving vehicle. 45. In the program described in 43. or 44., The second criterion includes a degree of congestion in the specified space being equal to or greater than a reference value. 46. ​​In the program according to any one of 43. to 45., The second criterion includes the subject being surrounded by multiple people. 47. In the program according to any one of 43. to 46., The program, wherein the second criterion includes that the distance between the subject and the other person is equal to or less than a reference value. 48. In the program according to any one of 43. to 47., further causing the computer to execute a procedure of estimating attributes of the subject and the other person through image processing; The second criterion includes a condition regarding the attribute. 49. In the program described in 48., The program, wherein the attributes include conditions regarding at least one of the gender and age group of the subject and at least one of the others. 50. In the program described in 48. or 49., when the subject whose biometric information is determined to satisfy the first criterion is detected, further causing the computer to execute a procedure of identifying the position of the subject; A program for acquiring an image including the specified position and performing the image processing in the estimation step. 51. In the program described in 50., further causing the computer to perform the steps of pointing a camera at the identified position and causing the camera to generate the image; A program for acquiring the generated image and performing the image processing in the estimation step. 52. In the program according to any one of 43. to 51., The biological information includes at least one of a heart rate and a sweat rate. 53. In the program described in 52., A program for acquiring the biometric information of the subject from a wearable device worn by the subject in the step of acquiring the biometric information. 54. In the program according to any one of 43. to 53., In the outputting step, outputting warning information toward the predetermined space before outputting the predetermined information; a program that outputs the predetermined information if the state satisfying the first criterion and the second criterion continues after the warning information is output; 55. In the program described in 54., the predetermined space is the interior of a moving vehicle, In the outputting step, A program that outputs the predetermined information to at least one of a driver and a manager of the mobile vehicle. 56. In the program according to any one of 43. to 55., In the step of acquiring the biometric information, biometric information of the other person is further acquired; In the step of determining, it is determined whether or not the biometric information of the other person satisfies a third criterion; a program for outputting the predetermined information when the biometric information of the other person satisfies the third criterion in the outputting step;

[0169] 57. To the computer, A procedure for acquiring biometric information of a subject present in a predetermined space; acquiring an image including the subject and their surroundings; a step of determining whether the biometric information satisfies a first criterion and whether the state of other people around the subject satisfies a second criterion; a computer-readable recording medium storing a program for executing a procedure for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied. 58. In the recording medium according to 57., A computer-readable recording medium storing a program, wherein the predetermined space is the interior of a moving vehicle. 59. In the recording medium according to 57. or 58., A computer-readable recording medium storing a program, wherein the second criterion includes a degree of congestion in the specified space being equal to or greater than a reference value. 60. The recording medium according to any one of items 57 to 59, A computer-readable recording medium storing a program, wherein the second criterion includes the subject being surrounded by multiple people. 61. The recording medium according to any one of items 57 to 60, A computer-readable recording medium storing a program, wherein the second criterion includes that the distance between the subject and the other person is equal to or less than a reference value. 62. The recording medium according to any one of items 57 to 61, further causing the computer to execute a procedure of estimating attributes of the subject and the other person through image processing; a computer-readable recording medium storing a program, wherein the second criteria include conditions related to the attribute; 63. In the recording medium according to 62., A computer-readable recording medium storing a program, wherein the attributes include conditions regarding at least one of the gender and age group of at least one of the subject and the others. 64. In the recording medium according to 62. or 63., when the subject whose biometric information is determined to satisfy the first criterion is detected, further causing the computer to execute a procedure of identifying the position of the subject; A computer-readable recording medium storing a program for acquiring an image including the identified position and performing the image processing in the estimation procedure. 65. In the recording medium according to 64., further causing the computer to perform the steps of pointing a camera at the identified position and causing the camera to generate the image; A computer-readable recording medium storing a program for acquiring the generated image and performing the image processing in the estimation procedure. 66. The recording medium according to any one of 57. to 65., A computer-readable recording medium storing a program, wherein the biological information includes at least one of heart rate and sweat rate. 67. In the recording medium according to 66., A computer-readable recording medium storing a program for acquiring the subject's biometric information from a wearable device worn by the subject in a procedure for acquiring the biometric information. 68. In the recording medium according to any one of 57. to 67., In the outputting step, outputting warning information toward the predetermined space before outputting the predetermined information; A computer-readable recording medium storing a program that outputs the predetermined information if the state that satisfies the first criterion and the second criterion continues after the warning information is output. 69. The recording medium according to 68., the predetermined space is the interior of a moving vehicle, In the outputting step, A computer-readable recording medium storing a program for outputting the predetermined information to at least one of a driver and a manager of the mobile vehicle. 70. The recording medium according to any one of items 57 to 69, In the step of acquiring the biometric information, biometric information of the other person is further acquired; In the step of determining, it is determined whether or not the biometric information of the other person satisfies a third criterion; A computer-readable recording medium storing a program for outputting the predetermined information when the biometric information of the other person satisfies the third criterion in the outputting step. [Explanation of symbols]

[0170] 1. Surveillance System 3. Communication Network 5. Camera 7. Car speakers 10 Mobile device for the subject 20 Wearable devices 30 Station staff terminal 40 Indoor speaker 50 headphones 60 Mobile Vehicles 100 Monitoring equipment 102 Biometric information acquisition unit 104 Image acquisition unit 106 Judgment section 108 Output Processing Unit 110 Estimation part 112 Specific section 114 Camera control unit 116 Memory Processing Unit 118 Adjustment section 120 Display device 130 Storage device 200 Incident Information 210 Accumulated Information 220 Analysis information 1000 computers 1010 Bus 1020 processor 1030 memory 1040 Storage Device 1050 Input / Output Interface 1060 Network Interface

Claims

1. a biometric information acquisition means for, when detecting the presence of a subject in a predetermined space using information notified from a communication device carried by the subject, acquiring from the communication device the biometric information of the subject measured by a sensor of a wearable device worn by the subject and transmitted to the communication device; an image acquisition means for acquiring an image including the subject and his / her surroundings within the predetermined space; a determining means for determining whether the biometric information of the subject acquired by the biometric information acquiring means satisfies a first criterion, and for determining whether the state of another person around the subject, identified by performing image analysis processing on the image acquired by the image acquiring means, satisfies a second criterion; an output processing means for outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied; an estimation means for estimating attributes of the subject and the other person by image processing; camera control means for causing a camera to generate said image; Equipped with The determination means calculates a degree of congestion in the predetermined space by performing image analysis processing on the image acquired by the image acquisition means, If the congestion level is equal to or greater than a reference value, the camera control means causes the camera to generate an image with a bird's-eye view angle that allows identification of the formation of a plurality of people; the determination means performs the determination process using the second criterion that the subject is surrounded by the plurality of people; If the congestion level is less than the reference value, The determination means performs a determination process using a condition related to the attribute estimated by the estimation means as the second criterion.

2. 2. The monitoring device according to claim 1, A monitoring device, wherein the predetermined space is the interior of a moving vehicle.

3. 3. The monitoring device according to claim 1, A monitoring device, wherein the second criterion includes that the distance between the subject and the other person is less than or equal to a reference value.

4. The monitoring device according to any one of claims 1 to 3, The attributes include conditions regarding at least one of the gender and age group of the subject and at least one of the others.

5. 5. The monitoring device according to claim 1, further comprising an identification means for identifying a position of the subject when the subject whose biological information is determined to satisfy the first criterion is detected, The estimation means acquires an image including the identified position and performs the image processing.

6. When the presence of a subject in a predetermined space is detected using information notified from a communication device carried by the subject, the biometric information of the subject measured by a sensor of a wearable device worn by the subject and transmitted to the communication device is acquired from the communication device; acquiring an image including the subject and their surroundings within the predetermined space; determining whether the acquired biometric information of the subject satisfies a first criterion, and determining whether the state of other people around the subject, identified by performing image analysis processing on the acquired image, satisfies a second criterion; outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied; Estimating attributes of the subject and the other person through image processing; Calculating the degree of congestion in the predetermined space by performing image analysis processing on the acquired image; If the congestion level is equal to or greater than a reference value, The camera generates an image with a bird's-eye view that can identify the formation of multiple people, performing a determination process using the fact that the subject is surrounded by the plurality of people as the second criterion; If the congestion level is less than the reference value, A monitoring method in which a determination process is performed using a condition related to the estimated attribute as the second criterion.

7. On the computer, a step of detecting the presence of a subject in a predetermined space using information notified from a communication device carried by the subject, and acquiring, from the communication device, biometric information of the subject measured by a sensor of a wearable device worn by the subject and transmitted to the communication device; acquiring an image including the subject and their surroundings within the predetermined space; a step of determining whether the biometric information of the subject acquired in the step of acquiring the biometric information satisfies a first criterion, and determining whether the states of other people around the subject identified by performing image analysis processing on the image acquired in the step of acquiring the image satisfy a second criterion; a step of outputting predetermined information when it is determined that both the first criterion and the second criterion are satisfied; an estimation step of estimating attributes of the subject and the other person by image processing; controlling a camera that generates the image; In the step of determining, the degree of congestion in the predetermined space is calculated by performing image analysis processing on the image acquired in the step of acquiring the image; If the congestion level is equal to or greater than a reference value, In the step of controlling the camera, the camera is caused to generate an image with a bird's-eye view angle capable of identifying a formation of a plurality of people; In the step of determining, a determination process is performed using the second criterion that the subject is surrounded by the plurality of people; If the congestion level is less than the reference value, a program for performing a determination process in the determining step using a condition related to the attribute estimated in the estimating step as the second criterion;

Images