Suspicious person detection device, suspicious person detection method, and program

Abstract

We provide technology that can detect suspicious individuals with greater accuracy. [Solution] The suspicious person detection device comprises: video acquisition means for acquiring video of a person; vital data acquisition means for acquiring vital data of a person; first determination means for determining the state of the person as represented by the vital data; second determination means for determining whether the situation of the person based on the video matches the situation expected from the state of the person as represented by the vital data; and output means for outputting an alert if it is determined that the situation of the person based on the video does not match the situation expected from the state of the person as represented by the vital data.

Description

Technical Field

[0001] The present disclosure relates to a suspicious person detection device, a suspicious person detection method, and a program.

Background Art

[0002] There is a technology for detecting and notifying a suspicious person, who is a person likely to commit an illegal act such as a crime. For example, there is a technology for detecting and notifying a suspicious person by detecting a specific action of a person or a combination of specific actions of a person. Also, for example, there is a technology for detecting and notifying a suspicious person using a vibration pattern emitted by a person or thermal image data of a person.

[0003] Patent Document 1 describes a suspicious person notification system that calculates a stress index from vital data such as electroencephalogram, cerebral blood flow, pulse wave, blood pressure, respiratory rate, body temperature, and sweating amount, estimates a person with a stress index higher than a reference value as a suspicious person, and issues a warning when the suspicious person enters a specific area.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the suspicious person notification system described in Patent Document 1, a person with a high stress index may be estimated as a suspicious person due to reasons other than committing an illegal act.

[0006] An example of an object of the present disclosure is to provide a technology for detecting a suspicious person with higher accuracy.

Means for Solving the Problems

[0007] A suspicious person detection device in one aspect of this disclosure includes: video acquisition means for acquiring video of a person; vital data acquisition means for acquiring vital data of a person; first determination means for determining whether the state of the person represented by the vital data is a specific state that may lead to misconduct; second determination means for determining whether the situation of the person based on the video is a situation that causes the specific state of the vital data and is related to factors other than misconduct; and output means for outputting an alert when it is determined that the situation of the person based on the video does not conform to a situation that can be expected from the state of the person represented by the vital data.

[0008] A suspicious person detection method in one aspect of this disclosure involves a computer acquiring video footage of a person, acquiring vital data of the person, determining that the person's state as represented by the vital data is a specific state that may lead to misconduct, determining whether the person's situation based on the video footage is a situation that causes the specific state in the vital data and is not related to misconduct, and if it is determined that the person's situation based on the video footage does not conform to the situation that can be expected from the person's state as represented by the vital data, an alert is issued.

[0009] A suspicious person detection program in one aspect of this disclosure involves a computer that acquires video footage of a person, acquires vital data of a person, determines that the state of the person represented by the vital data is a specific state that may lead to misconduct, determines whether the situation of the person based on the video footage is a situation that causes the specific state of the vital data and is related to factors other than misconduct, and if it is determined that the situation of the person based on the video footage does not conform to the situation that can be expected from the state of the person represented by the vital data, the program executes a process to output an alert.

[0010] Each program may be stored on a non-temporary storage medium that is readable by the computer. [Effects of the Invention]

[0011] One example of the benefits of this disclosure is that it will be possible to detect suspicious individuals with greater accuracy. [Brief explanation of the drawing]

[0012] [Figure 1] This diagram shows an overview of the system, including the suspicious person detection device. [Figure 2] This is a block diagram showing the functional configuration of a suspicious person detection device. [Figure 3] This figure shows an example of an alert output screen. [Figure 4] This is a flowchart showing the operation of the suspicious person detection device. [Figure 5] This figure shows the hardware configuration for implementing the suspicious person detection device described in this disclosure using a computer and its peripheral devices. [Modes for carrying out the invention]

[0013] [Embodiment] Figure 1 shows an overview of the system, including the suspicious person detection device.

[0014] A suspicious person detection device is a device that detects suspicious individuals using their vital data and video footage.

[0015] The suspicious person detection device is communicatively connected to sensors for acquiring vital data of individuals, cameras for acquiring video footage of individuals, and an output device for issuing alerts. The suspicious person detection device is used, for example, to detect suspicious individuals in facilities such as stores, but is not limited to this.

[0016] Here, vital data may be at least one of the following: physiological information of a person, information representing the person's internal state, and information representing the person's mental state. For example, vital data may be physiological information such as heart rate, respiratory rate, body temperature, and pupil dilation. Alternatively, for example, vital data may be a numerical representation of the person's internal state or mental state, such as stress levels, tension levels, and aggression, estimated using various physiological information.

[0017] A sensor for acquiring vital data of a person is a sensor that measures the vital data of the person. The sensor for acquiring vital data of a person may be, for example, a wearable terminal or one attached to a cart or basket used for carrying goods in a store. Also, the sensor for acquiring vital data of a person may be a camera that photographs the person. In this case, the vital data is extracted by analyzing the color change, facial movement, gaze, blink, pupil diameter, iris movement, expression, etc. included in the face image of the person photographed by the camera.

[0018] A camera for acquiring an image of a person is, for example, a surveillance camera. Also, the camera for acquiring an image of a person may be a camera of a terminal device used for self-scanning shopping in a store where a customer scans a product by himself / herself. Note that when the sensor for acquiring vital data of a person is a camera, it may be the same camera.

[0019] An output device that outputs an alert is a device that outputs an alert by displaying sound or an image. The output device may be, for example, a display or a speaker. Also, the output device may be a terminal device.

[0020] Next, the configuration of the suspicious person detection device 100 in the embodiment will be described.

[0021] FIG. 2 is a block diagram showing the configuration of the suspicious person detection device 100 in the first embodiment. Referring to FIG. 2, the suspicious person detection device 100 includes a vital data acquisition unit 101, a first determination unit 102, an image acquisition unit 103, a second determination unit 104, and an output unit 105.

[0022] Next, the configuration of the suspicious person detection device 100 in the first embodiment will be described in detail.

[0023] In Figure 2, the vital data acquisition unit 101 is an example of a vital data acquisition means for acquiring a person's vital data. The vital data acquisition unit 101 may acquire vital data continuously or at regular time intervals.

[0024] The vital data acquisition unit 101 acquires vital data, for example, from a sensor that measures vital data. The sensor that measures vital data may be provided in a wearable device or a communication-enabled terminal device such as a smart cart used by a customer in a store. The vital data acquisition unit 101 may associate the identification information of the sensor or terminal device that acquires vital data with the acquired vital data. In addition, the vital data acquisition unit 101 may associate the identification information, such as membership information of a person associated with the sensor or terminal device that acquires vital data, with the acquired vital data.

[0025] Furthermore, the vital data acquisition unit 101 may acquire vital data based on video, for example. In this case, the vital data acquisition unit 101 may acquire video from a camera and extract vital data. Alternatively, the vital data acquisition unit 101 may acquire vital data from another device that extracts vital data from video. The vital data acquisition unit 101 may associate the identification information of the terminal device equipped with a camera that captures video from which vital data is extracted with the extracted vital data. The vital data acquisition unit 101 may also associate the acquired vital data with identification information such as member information of a person associated with the terminal device equipped with a camera that captures video from which vital data is extracted. Furthermore, if the camera that captures video from which vital data is extracted is not a surveillance camera or otherwise not corresponding to each individual, the vital data acquisition unit 101 may associate the acquired vital data with an identification number or the like for identifying the person.

[0026] The vital data acquisition unit 101 may acquire both vital data measured by sensors and vital data based on video. For example, the vital data acquisition unit 101 may acquire vital data measured by sensors from people using measurement sensors, and vital data based on video from people not using measurement sensors.

[0027] The first determination unit 102 is an example of a first determination means for determining a person's state as represented by vital data. For example, the first determination unit 102 determines that the person's state as represented by vital data is a state that may lead to misconduct. To determine this state, for example, the first determination unit 102 can determine whether the vital data meets the conditions for vital data in the event of misconduct. The conditions for vital data in the event of misconduct may be the values ​​of vital data or changes in vital data values ​​when a person is committing misconduct or attempting to commit misconduct.

[0028] Here, "misconduct" refers to criminal acts, violence, intimidation, etc. For example, misconduct in a store includes shoplifting and robbery. Generally, when a person commits or attempts to commit misconduct, they experience emotional changes such as increased tension and stress. Such emotional changes are reflected in vital data. For example, changes in vital data when committing or attempting misconduct include increased heart rate, increased skin temperature, increased respiratory rate, and dilated pupils. In other words, the conditions for vital data when committing misconduct may be that the values ​​shown in the vital data exceed a predetermined threshold, or that the amount of change in the values ​​shown in the vital data exceeds a predetermined threshold. Alternatively, for example, the conditions for vital data when committing misconduct may be that the changes in vital data are similar to the changes in vital data when committing misconduct. Here, similarity may mean that the difference is below a certain level or the degree of similarity is greater than a threshold. The conditions for vital data when committing misconduct are not limited to these examples and may be determined appropriately depending on the vital data used. The first determination unit 102 only needs to determine whether the person's vital data meets the conditions for vital data in the case of fraudulent activity.

[0029] The first determination unit 102 may perform its determination using multiple types of vital data. For example, the first determination unit 102 may determine whether the heart rate and pupil dilation meet the conditions for vital data in the event of fraud, but the combination of vital data that the first determination unit 102 determines is not limited to this example.

[0030] The video acquisition unit 103 is an example of a video acquisition means for acquiring video of a person. The video of the person is used by the second determination unit 104, described later, to make a determination regarding the situation in which the person is located. The video acquisition unit 103 acquires video from a camera that photographs the person. As mentioned above, the camera may be a surveillance camera or a camera on a terminal device used for scan shopping. The timing at which the video acquisition unit 103 acquires video of a person is not particularly limited. For example, the video acquisition unit 103 may acquire video of a person when the first determination unit 102 determines that the vital data meets the conditions for fraudulent activity.

[0031] For example, the video acquisition unit 103 acquires video footage of a person who is determined to meet the conditions for vital data fraud. For example, if vital data measured by a measurement sensor installed on a terminal device used by a person is acquired, the video acquisition unit 103 may identify a camera whose shooting range includes the position indicated in the terminal device's location information, based on the location information of the terminal device equipped with the measurement sensor, and acquire video footage captured by the identified camera. Also, if the vital data acquisition unit 101 acquires vital data based on video footage, the video acquisition unit 103 may acquire the video footage used to acquire the vital data. For example, if the vital data acquisition unit 101 extracts vital data from video footage, the video acquisition unit 103 may acquire the video footage used to extract the vital data. Also, for example, if the vital data acquisition unit 101 acquires vital data extracted from video footage from another device, the video acquisition unit 103 may acquire the video footage used to extract the vital data from the camera or other device.

[0032] Furthermore, the video acquisition unit 103 acquires video footage at the time when vital data is acquired from a person. For example, the video acquisition unit 103 acquires video footage at the time when vital data that meets the conditions for fraudulent activity is acquired from a person. The time when vital data is acquired from a person is the time of measurement if the vital data acquisition unit 101 acquires vital data from a measurement sensor, or the time the video is captured if the vital data acquisition unit 101 acquires vital data based on video footage.

[0033] Furthermore, for example, the video acquisition unit 103 may acquire video footage from a predetermined time period prior to the time when vital data is acquired from a person. This is because, at the time when vital data that meets the conditions for fraudulent activity is acquired from a person, the person's actions and other circumstances in the period prior to that time may affect the person's vital data. For example, a person with a high heart rate at a certain point in time may have had a high heart rate because they were running before that point in time, and their heart rate may also be high if they were walking at that point in time. The predetermined time period prior to the time when vital data is acquired from a person, during which the video acquisition unit 103 acquires video footage, should be any time period that affects the vital data at the time of acquisition, and can be set as appropriate.

[0034] The second determination unit 104 is an example of a second determination means that determines whether the situation of a person based on the video footage matches the situation that can be expected from the state of the person represented by the vital data. For example, if the first determination unit 102 determines that the vital data meets the conditions for when someone is committing fraud, the second determination unit 104 determines whether the situation of the person based on the video footage acquired by the video acquisition unit 103 matches the state of the person represented by the vital data.

[0035] The person's situation refers to at least one of the following: the person's actions, the person's movements, and the person's surrounding environment. For example, the second determination unit 104 may detect the person's actions and movements from the person's video using a known method. The person's surrounding environment may be, for example, the actions and movements of other people present in the vicinity of the person, as detected from the video. In this case, for example, the second determination unit 104 may detect the actions and movements of other people present within a predetermined distance range from the person using a known method. Alternatively, for example, the person's surrounding environment may be detected from sounds detected in the vicinity of the person. In this case, for example, the second determination unit 104 may detect sounds included in the person's video using a known method.

[0036] Even if vital data meets the conditions for vital data in the event of fraud, other factors unrelated to the fraud may cause the vital data to meet those conditions. Therefore, the second determination unit 104 uses video footage to determine the situation of a person from whom vital data meeting the conditions for vital data in the event of fraud has been obtained. Accordingly, the second determination unit 104 determines whether the situation of the person based on the video footage is consistent with the state of the person represented by the vital data. In other words, if there are circumstances that should be factors such as the person's tension or emotional state as represented by the vital data, the second determination unit 104 determines that the situation of the person based on the video footage from whom vital data meeting the conditions for vital data in the event of fraud has been obtained is consistent with the situation that can be expected from the state of the person represented by the vital data. When this determination is made, the person is not treated as a suspicious person. In contrast, if the vital data meets the conditions for vital data in the event of fraud, and no contributing factors are found, the second determination unit 104 determines that the person's situation based on the video footage of the person from whom the vital data meeting the conditions for vital data in the event of fraud was obtained does not match the situation that would be expected from the person's state as represented by the vital data. In such a determination, the person may be treated as a suspicious person.

[0037] In this context, other factors besides the act of fraud that would qualify a person's vital data as fraudulent include, for example, situations that place a physical burden on the person or situations that place a mental burden on the person due to external factors.

[0038] Situations that place a physical burden on a person include, for example, situations involving exercise. For example, a situation that places a physical burden on a person is when a person is running. In this case, the second determination unit 104 determines that the person is running based on the person's image. For example, the second determination unit 104 may determine that the person is running based on the speed of movement of the person identified from the image, or based on the posture of the person identified from the image. Another example of a situation that places a physical burden on a person is when a person is carrying something heavy. In this case, the second determination unit 104 may determine that the person is carrying something heavy based on the size of the person's luggage identified from the image.

[0039] Situations in which an external factor causes mental stress to a person include, for example, situations in which a person is surprised or feels uncomfortable due to an external factor. For example, one situation in which an external factor causes mental stress to a person is when a person is startled by a loud noise. In this case, the second determination unit 104 determines, based on the person's video including sound, that a loud sound, whose loudness value is above a threshold, occurred around the person. In this case, the second determination unit 104 may also determine that a loud noise occurred suddenly around the person. This is because, in situations where a loud noise continues to play, the person is accustomed to the loud noise and is less likely to be surprised. Another example in which an external factor causes mental stress to a person is when a person bumps into another person or another person's luggage. In this case, the second determination unit 104 may determine, based on the person's video, that the person came into contact with another person or another person's luggage.

[0040] Furthermore, if the second determination unit 104 determines that the person's condition based on the video of the person at the time of vital data acquisition does not match the condition expected from the person's state as represented by the vital data, the second determination unit 104 may make a similar determination to the person's condition based on the video of the person at a predetermined time prior to the time of vital data acquisition. In this case, the video acquisition unit 103 may acquire video from a predetermined time prior to the time of vital data acquisition, in response to the determination that the person's condition based on the video at the time of vital data acquisition does not match the condition expected from the person's state as represented by the vital data.

[0041] As mentioned above, the state of a person as represented by vital data can be influenced by circumstances prior to the time the vital data was acquired. Therefore, the second determination unit 104 makes a determination based on the person's circumstances at a predetermined time prior to the time the vital data was acquired. However, if the second determination unit 104 makes a determination based on the circumstances prior to the time the vital data was acquired each time, the processing load will increase. In contrast, as mentioned above, if the second determination unit 104 determines that the person's circumstances based on the person's circumstances at the time the vital data was acquired do not match the circumstances expected from the state of the person as represented by the vital data, the processing load can be reduced by making a determination based on the person's circumstances based on the person's circumstances at a predetermined time prior to the time the vital data was acquired.

[0042] The output unit 105 is an example of an output means that outputs an alert when the second determination unit 104 determines that the person's condition does not match the condition expected from the person's state as indicated by the vital data. The output unit 105 may output the alert to at least one of a display, speaker, or terminal device.

[0043] First, let's explain the output destinations to which the output unit 105 outputs alerts.

[0044] The output unit 105 may output an alert to a person (hereinafter referred to as a suspicious person) who has been determined by the first determination unit 102 to meet the conditions for vital data fraud, and who has also been determined by the second determination unit 104 to not conform to the situation expected from the person's state as represented by the vital data. In this case, the output unit 105 may output an alert to an output device near the location of the person identified from the video or the information of the camera that captured the video. The camera information may be the camera's location information, information indicating the camera's shooting range, or identification information of the camera associated with this information. Alternatively, the output unit 105 may output an alert to the terminal device used to acquire the vital data video.

[0045] Furthermore, the output unit 105 may output an alert to an employee or other person other than the suspicious person. In this case, the output unit 105 should output the alert to a predetermined terminal device or the like. For example, the output unit 105 may output the alert to a terminal device used by an employee of the facility where the person is staying.

[0046] Furthermore, the output unit 105 may output an alert to a terminal device used by the facility manager if the suspicious person is an employee. The fact that the suspicious person is an employee may be identified by the fact that the person's clothing in the video is an employee uniform, that the person is holding an employee ID card in the video, or by information that has been pre-associated between the terminal device that acquired the vital data and the employee's identification information.

[0047] Next, we will explain the content of the alerts output by the output unit 105.

[0048] Alerts directed at the suspicious person are issued, for example, to prevent them from committing fraudulent acts. For example, an alert directed at the suspicious person may be an audio or video warning against fraudulent activity, or an audio message designed to attract the attention of those around them. The idea is that attracting the attention of those around them will make it more difficult for the suspicious person to commit fraud. The content of the audio message designed to attract the attention of those around them is not particularly limited; it may be a warning against fraudulent activity, an advertisement related to a store, or a message asking if they need assistance, such as "Are you in trouble?".

[0049] Alerts to individuals other than the suspicious person are issued, for example, to prompt employees or other individuals to take action to prevent the suspicious person from committing fraudulent acts. Furthermore, alerts to individuals other than the suspicious person may be issued to identify the suspicious person if fraudulent activity is discovered. For example, an alert to individuals other than the suspicious person may include information that can identify the suspicious person. This information may include at least one of the following: the suspicious person's current location, the location at the time vital data was acquired, or video footage of the suspicious person.

[0050] Figure 3 shows an example of an alert output screen for employees. In Figure 3, the alert output screen includes information that a suspicious person has been detected, the location of the suspicious person within the store, and video footage of the suspicious person. Also in Figure 3, the alert output screen includes an "Alert" button. For example, if the "Alert" button is selected on the alert output screen for employees, the output unit 105 may further output an alert to other employees or security guards informing them that a suspicious person has been detected. Alternatively, if the above button is selected, the output unit 105 may also output an alert to the person detected as a suspicious person. In this way, by allowing employees to decide whether or not to issue an alert, it is possible to prevent accidentally alerting customers or other individuals, or provoking individuals who are becoming increasingly aggressive.

[0051] Next, we will explain the case where the person's condition based on the video is determined to match the condition expected from the person's state as indicated by the vital data.

[0052] For example, the output unit 105 does not output an alert if it determines that the situation of a person based on the video matches the situation expected from the person's state as indicated by the vital data. Alternatively, for example, if the output unit 105 determines that the situation of a person based on the video matches the situation expected from the person's state as indicated by the vital data, it may output that the person is not a suspicious person. For example, the output unit 105 may superimpose on each person in the video whether or not the situation of a person based on the video matches the situation expected from the person's state as indicated by the vital data in a way that is easily identifiable.

[0053] Furthermore, for example, the output unit 105 may output information about a person who meets the conditions for vital data when vital data is being misused. For example, if the output unit 105 determines that the situation of a person based on the video matches the situation expected from the state of the person represented by the vital data, it may output that the situation of a person who meets the conditions for vital data when vital data is being misused matches the situation expected from the state of the person represented by the vital data. For example, the output unit 105 may output the situation of a person based on the video, such as that the person who meets the conditions for vital data when vital data is being misused is running.

[0054] Furthermore, the output unit 105 may output an alert if the vital data acquisition unit 101 is unable to acquire vital data from an employee wearing a measurement sensor for a predetermined period of time or longer. This prevents fraud even if an employee who knows that vital data is used for suspicious person detection tries to commit fraud without being detected as a suspicious person and intentionally prevents the acquisition of vital data.

[0055] The operation of the suspicious person detection device 100, configured as described above, will be explained with reference to the flowchart in Figure 4.

[0056] As shown in Figure 4, first, the vital data acquisition unit 101 acquires the vital data of the person (step S101). Vital data may be acquired from each person included in the facility at any time.

[0057] Next, the first determination unit 102 determines whether the vital data of the person acquired in step S101 meets the conditions for vital data in the case of fraudulent activity (step S102). If, in step S102, it is determined that the vital data of the person does not meet the conditions for vital data in the case of fraudulent activity (step S102: No), the suspicious person detection device 100 terminates processing.

[0058] In the determination in step S102, if it is determined that the person's vital data meets the conditions for vital data in the case of fraudulent activity (step S102: Yes), the video acquisition unit 103 acquires video of the person (step S103).

[0059] Next, the second determination unit 104 determines whether the person's situation based on the video acquired in step S103 matches the situation expected from the person's state as represented by the vital data acquired in step S101 (step S104). If, in step S104, it is determined that the person's situation based on the video matches the situation expected from the person's state as represented by the vital data (step S104: Yes), the suspicious person detection device 100 may terminate processing.

[0060] If the video footage determines that the person's condition does not match the condition expected from the person's vital data (Step S104: No), the output unit 105 outputs an alert.

[0061] With this, the suspicious person detection device 100 completes its series of operations. The series of operations may be repeated. The order of each operation may also be changed as appropriate. For example, the step of acquiring the person's image in step S103 may be performed before step S101 or at the same time as step S101.

[0062] In the above-described embodiment of the suspicious person detection device 100, the first determination unit 102 determines the state of the person as represented by the vital data, and the second determination unit 104 determines whether the situation of the person based on the video of the person matches the situation expected from the state of the person as represented by the vital data. If the output unit 105 determines that the situation of the person does not match the situation expected from the state of the person as represented by the vital data, it outputs an alert.

[0063] As a result, the suspicious person detection device 100 in this embodiment can detect suspicious persons with greater accuracy compared to a system that detects suspicious persons based solely on vital data. This is because it does not issue an alert when the conditions for fraudulent vital data are met due to factors other than the fraudulent vital data itself.

[0064] [Hardware configuration] Some or all of the components of each apparatus or system in the embodiments of this disclosure described above can be realized by any combination of an information processing device 1000 and a program, for example, as shown in Figure 5. The information processing device 1000 includes, as an example, the following configuration.

[0065] ·CPU(Central Processing Unit)1001 • ROM (Read Only Memory) 1002 ·RAM(Random Access Memory)1003 • Program 1004 loaded into RAM1003 • Storage device 1005 for storing program 1004 • Drive device 1007 for reading and writing recording medium 1006 • Communication I / F 1008 connected to communication network 1009 • Input / output interface 1010 for data input and output. • Bus 1011 connecting each component Note that I / F is an abbreviation for Interface.

[0066] Each component of each device or system in each embodiment is realized by the CPU 1001 acquiring and executing a program that realizes its function. The program that realizes the function of each component of each device is, for example, stored in advance in the storage device 1005 or RAM 1003 and read by the CPU 1001 as needed. The program 1004 may be supplied to the CPU 1001 via a communication network, or it may be stored in advance in the recording medium 1006 and read by the drive device 1007 and supplied to the CPU 1001.

[0067] There are various variations in how each device is implemented. For example, each device or system may be implemented by any combination of a separate information processing device 1000 and a program for each component. Alternatively, the multiple components of each device may be implemented by any combination of a single information processing device 1000 and a program.

[0068] Furthermore, some or all of the components of each device or system are realized by general-purpose or dedicated circuits, including processors, or combinations thereof. Examples of such circuits include CPUs, GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), and AI (Artificial Intelligence) processing LSIs (Large Scale Integration). These may be comprised of a single chip or multiple chips connected via a bus. Some or all of the components of each device may be realized by combinations of the aforementioned circuits and programs.

[0069] When some or all of the components of each device or system are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-server system or a cloud computing system.

[0070] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the configuration and details of the present invention can be understood by those skilled in the art within the scope of the present invention.

[0071] Furthermore, although multiple operations are described sequentially in flowchart format, the order in which they are described does not limit the order in which the operations must be performed. Therefore, when implementing each embodiment, the order of the operations may be changed to the extent that it does not impair the content.

[0072] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0073] [Note 1] A means of acquiring video footage of a person, A vital data acquisition means for acquiring the vital data of the aforementioned person, A first determination means for determining the state of the person as represented by the vital data, A second determination means for determining whether the situation of the person based on the video corresponds to the situation expected from the state of the person as represented by the vital data, An output means that outputs an alert if it is determined that the person's condition based on the video does not match the condition expected from the person's state as represented by the vital data, A suspicious person detection device equipped with the following features.

[0074] [Note 2] The first determination means determines that the state of the person as indicated by the vital data is a state in which there is a possibility that the person may engage in misconduct. The suspicious person detection device described in Appendix 1.

[0075] [Note 3] The second determination means determines that if the situation is one in which the person is under physical strain, the situation is consistent with the situation that can be inferred from the state of the person as represented by the vital data. An intruder detection device as described in Appendix 1 or 2.

[0076] [Note 4] The situation in which the person experiences physical strain is when the person is running. The suspicious person detection device described in Appendix 3.

[0077] [Note 5] A situation in which the person is physically burdened is when the person is carrying a large load. An intruder detection device as described in Appendix 3 or 4.

[0078] [Note 6] The second determination means determines that if the situation involves external factors that place a mental burden on the person, it is consistent with the situation that can be inferred from the state of the person as represented by the vital data. A suspicious person detection device as described in any of the appendices 1 to 5.

[0079] [Note 7] An external factor that causes mental distress to the aforementioned person is a situation where a loud noise occurs near the person. The suspicious person detection device described in Appendix 6.

[0080] [Note 8] A situation in which external factors cause mental stress to the aforementioned person is a situation in which the aforementioned person clashes with another person. An intruder detection device as described in Appendix 6 or 7.

[0081] [Note 9] The aforementioned video is the video taken at the time when the vital data was acquired from the person. A suspicious person detection device as described in any of the appendices 1 to 8.

[0082] [Note 10] If the video acquisition means determines that the person's condition based on the video at the time of acquisition does not match the person's condition as expected from the vital data, it acquires video from a predetermined time prior to the time of acquisition. The second determination means determines whether the person's condition based on video footage from a predetermined time prior to the acquisition time matches the person's condition as expected from the vital data. The suspicious person detection device described in Appendix 9.

[0083] [Note 11] The output means outputs an alert to an output device near the person. A suspicious person detection device as described in any of the appendices 1 to 10.

[0084] [Note 12] The output means outputs an alert to a terminal device used by an employee of the facility where the person is staying. A suspicious person detection device as described in any of the appendices 1 to 11.

[0085] [Note 13] The output means outputs an alert to a terminal device used by the facility manager if the person is an employee of the facility. The suspicious person detection device described in Appendix 12.

[0086] [Note 14] The output means outputs an alert including the vital data and the video. An intruder detection device as described in Appendix 12 or 13.

[0087] [Note 15] The output means outputs an alert if the person is an employee of the facility and if non-video vital data cannot be obtained for a predetermined period of time or longer. A suspicious person detection device as described in any of the appendices 1 to 14.

[0088] [Note 16] Computers Capture the image of the person, We obtained the vital data of the aforementioned person, The vital data indicates the state of the person, The system determines whether the situation of the person based on the video matches the situation that can be inferred from the state of the person as represented by the vital data. If the person's condition based on the video footage is determined not to match the condition expected from the person's state as indicated by the vital data, an alert is issued. Methods for detecting suspicious individuals.

[0089] [Note 17] Capture the image of the person, We obtained the vital data of the aforementioned person, The vital data indicates the state of the person, The system determines whether the situation of the person based on the video matches the situation that can be inferred from the state of the person as represented by the vital data. If the person's condition based on the video footage is determined not to match the condition expected from the person's state as indicated by the vital data, an alert is issued. A recording medium that stores a program that causes a computer to execute a specific process. [Explanation of Symbols]

[0090] 100 Suspicious Person Detection Device 101 Vital Data Acquisition Unit 102 First Judgment Department 103 Video Acquisition Unit 104 Second judgment section 105 Output section 1000 Information Processing Devices 1001 CPU 1002 ROM 1003 RAM 1004 Program 1005 Storage device 1006 Recording media 1007 Drive unit 1008 Communication I / F 1009 Communication Network 1010 Input / Output Interface 1011 Bus

Claims

1. A means of acquiring video footage of a person, A vital data acquisition means for acquiring the vital data of the aforementioned person, A first determination means for determining whether the state of the person indicated by the vital data is a specific state that may lead to misconduct, A second determination means for determining whether the situation of the person based on the video is a situation that causes the specific state of the vital data and is related to factors other than committing fraud, An output means that outputs an alert if it is determined that the person's condition based on the video does not match the condition expected from the person's state as represented by the vital data, A suspicious person detection device equipped with the following features.

2. The aforementioned specific condition is at least one of the following: a state of high tension, a state of high stress, an increased heart rate, an increased skin temperature, an increased respiratory rate, or dilated pupils. The suspicious person detection device according to claim 1.

3. The second determination means determines that if the situation is one in which the person is under physical strain, the situation is consistent with the situation that can be inferred from the state of the person as represented by the vital data. The suspicious person detection device according to claim 1.

4. The second determination means determines that if the situation involves external factors that impose a mental burden on the person, the situation is consistent with the situation that can be inferred from the state of the person as represented by the vital data. The suspicious person detection device according to claim 1.

5. A situation in which an external factor causes mental distress to the person is at least one of the following: a loud noise occurring near the person, and the person colliding with another person. The suspicious person detection device according to claim 4.

6. If the video acquisition means determines that the person's condition based on the video at the time of acquisition, which is the time when the vital data was acquired from the person, does not match the person's condition as expected from the state of the person represented by the vital data, it acquires video from a predetermined time prior to the acquisition time. The second determination means determines whether the person's condition based on video footage from a predetermined time prior to the acquisition time matches the person's condition as expected from the vital data. The suspicious person detection device according to claim 1.

7. The output means outputs an alert to a terminal device used by an employee of the facility where the person is staying. The suspicious person detection device according to claim 1.

8. The output means outputs an alert if the person is an employee of the facility and if non-video vital data cannot be obtained for a predetermined period of time or longer. The suspicious person detection device according to claim 1.

9. Computers Capture the image of the person, We obtained the vital data of the aforementioned person, The vital data indicates that the person's condition is in a specific state that may lead to misconduct, The system determines whether the situation of the person based on the video footage is consistent with a situation that causes the specific state of the vital data and is not due to any other factor. If the situation of the person based on the aforementioned video is determined not to match the situation expected from the person's state as indicated by the aforementioned vital data, an alert will be issued. Methods for detecting suspicious individuals.

10. Capture the image of the person, We obtained the vital data of the aforementioned person, The vital data indicates that the person's condition is in a specific state that may lead to misconduct, The system determines whether the situation of the person based on the video footage is consistent with a situation that causes the specific state of the vital data and is not due to any other factor. If the situation of the person based on the aforementioned video is determined not to match the situation expected from the person's state as indicated by the aforementioned vital data, an alert will be issued. A program that instructs a computer to perform a process.

Images