Suspicious person detection device, suspicious person detection method, and program

Abstract

Provided is a technology for more accurately detecting a suspicious person. This suspicious person detection device comprises: a video acquisition means for acquiring a video of a person; a vital data acquisition means for acquiring vital data about the person; a first determination means for determining the state of the person represented by the vital data; a second determination means for determining whether the situation of the person based on the video matches the situation anticipated from the state of the person represented by the vital data; and an output means for outputting an alert if it is determined that the situation of the person based on the video does not match the situation anticipated from the state of the person represented by the vital data.

Description

Suspicious person detection device, suspicious person detection method, and recording medium

[0001] The present disclosure relates to a suspicious person detection device, a suspicious person detection method, and a recording medium.

[0002] There are technologies that detect and report suspicious individuals who may be committing crimes or other illegal acts. For example, there are technologies that detect and report suspicious individuals by detecting specific behaviors or combinations of specific behaviors by individuals. There are also technologies that detect and report suspicious individuals by using vibration patterns emitted by individuals or thermal imaging data of individuals.

[0003] Patent Document 1 describes a suspicious person notification system that calculates a stress index from vital data such as brain waves, cerebral blood flow, pulse waves, blood pressure, respiratory rate, body temperature, and sweat rate, presumes that a person with a stress index higher than a reference value is a suspicious person, and issues a warning if the suspicious person enters a specific area.

[0004] Japanese Patent Application Laid-Open No. 2018-037075

[0005] However, the suspicious person reporting system described in Patent Document 1 may in some cases assume that a person with a high stress index for reasons other than committing fraud is a suspicious person.

[0006] An example of an objective of the present disclosure is to provide a technology for detecting suspicious individuals with higher accuracy.

[0007] A suspicious person detection device in one aspect of the present disclosure includes an image acquisition means for acquiring an image of a person, a vital data acquisition means for acquiring vital data of the person, a first determination means for determining the state of the person represented by the vital data, a second determination means for determining whether the state of the person based on the image matches the state expected from the state of the person represented by the vital data, and an output means for outputting an alert when it is determined that the state of the person based on the image does not match the state expected from the state of the person represented by the vital data.

[0008] In one aspect of the present disclosure, a suspicious person detection method includes a computer acquiring video of a person, acquiring vital data of the person, determining the person's condition represented by the vital data, determining whether the person's condition based on the video matches a condition expected from the person's condition represented by the vital data, and outputting an alert if it is determined that the person's condition based on the video does not match a condition expected from the person's condition represented by the vital data.

[0009] A suspicious person detection program in one aspect of the present disclosure causes a computer to acquire video of a person, acquire vital data of the person, determine the person's condition represented by the vital data, determine whether the person's condition based on the video matches the condition expected from the person's condition represented by the vital data, and output an alert if it is determined that the person's condition based on the video does not match the condition expected from the person's condition represented by the vital data.

[0010] Each program may be stored in a non-transitory computer-readable recording medium.

[0011] One example of an effect of the present disclosure is that it is possible to detect suspicious individuals with higher accuracy.

[0012] It is a diagram showing an overview of a system including a suspicious person detection device. It is a block diagram showing the functional configuration of the suspicious person detection device. It is a diagram showing an example of an alert output screen. It is a flowchart showing the operation of the suspicious person detection device. It is a diagram showing a hardware configuration for realizing the suspicious person detection device of the present disclosure by a computer and its peripheral devices.

[0013] [Embodiment] FIG. 1 is a diagram showing an outline of a system including a suspicious person detection device.

[0014] A suspicious person detection device is a device that detects a suspicious person using vital data and video of the person.

[0015] The suspicious person detection device is communicably connected to a sensor for acquiring vital data of a person, a camera for acquiring video of the person, and an output device for outputting an alert. The suspicious person detection device detects suspicious people in facilities such as, but not limited to, stores.

[0016] Here, the vital data may be at least one of physiological information of a person, information representing the person's internal state, and information representing the person's mental state. The vital data may be, for example, physiological information such as heart rate, respiratory rate, body temperature, pupil dilation, etc. Furthermore, for example, the vital data may be a numerical representation of the person's internal or mental state, such as stress level, tension level, or aggression, estimated using various physiological information.

[0017] The sensor for acquiring a person's vital data is a sensor that measures the person's vital data. The sensor for acquiring a person's vital data may be, for example, a wearable device or a sensor attached to a cart or basket used to carry merchandise in a store. The sensor for acquiring a person's vital data may also be a camera that captures an image of the person. In this case, the vital data is extracted by analyzing changes in facial color, facial movement, gaze, blinking, pupil diameter, iris movement, facial expression, and the like contained in the image of the person's face captured by the camera.

[0018] The camera for capturing the image of the person may be, for example, a surveillance camera. Alternatively, the camera for capturing the image of the person may be a camera of a terminal device used for self-scan shopping, where customers scan products themselves in a store. If the sensor for capturing the vital data of the person is a camera, the same camera may be used.

[0019] The output device that outputs the alert is a device that outputs the alert by displaying audio or video. The output device may be, for example, a display or a speaker. The output device may also be a terminal device.

[0020] Next, the configuration of the suspicious person detection device 100 according to the embodiment will be described.

[0021] 2 is a block diagram showing the configuration of the suspicious person detection device 100 according to the first embodiment. Referring to FIG. 2, the suspicious person detection device 100 includes a vital data acquisition unit 101, a first determination unit 102, a video acquisition unit 103, a second determination unit 104, and an output unit 105.

[0022] Next, the configuration of the suspicious person detection device 100 in the first embodiment will be described in detail.

[0023] 2, vital data acquisition unit 101 is an example of a vital data acquisition unit that acquires vital data of a person. Vital data acquisition unit 101 may acquire the vital data continuously or at regular time intervals.

[0024] The vital data acquisition unit 101 acquires vital data from, for example, a sensor that measures the vital data. The sensor that measures the vital data may be provided in, for example, a wearable terminal or a communication-enabled terminal device such as a smart cart used by customers in a store. The vital data acquisition unit 101 may associate identification information of the sensor or terminal device that acquires the vital data with the acquired vital data. Furthermore, the vital data acquisition unit 101 may associate identification information, such as membership information of a person associated with the sensor or terminal device that acquires the vital data with the acquired vital data.

[0025] The vital data acquisition unit 101 may also acquire vital data based on, for example, video. In this case, the vital data acquisition unit 101 may acquire video from a camera and extract vital data. For example, the vital data acquisition unit 101 may also acquire vital data from another device that extracts vital data from video. The vital data acquisition unit 101 may associate the extracted vital data with identification information of a terminal device equipped with a camera that captures the video from which the vital data is extracted. The vital data acquisition unit 101 may also associate the acquired vital data with identification information, such as membership information, of a person associated with the terminal device equipped with a camera that captures the video from which the vital data is extracted. Furthermore, if the camera that captures the video from which the vital data is extracted is a surveillance camera or the like that does not correspond to each individual person, the vital data acquisition unit 101 may associate the acquired vital data with an identification number or the like for identifying the person.

[0026] The vital data acquiring unit 101 may acquire both vital data measured by a sensor and vital data based on video. For example, the vital data acquiring unit 101 may acquire vital data measured by a measurement sensor from a person who uses a measurement sensor, and may acquire vital data based on video from a person who does not use a measurement sensor.

[0027] The first determination unit 102 is an example of a first determination means for determining the state of a person represented by vital data. For example, the first determination unit 102 determines that the state of a person represented by vital data is a state that may lead to fraud. To determine this state, for example, the first determination unit 102 may determine whether the vital data meets vital data conditions for fraud. The vital data conditions for fraud may be the value of the vital data or a change in the value of the vital data when a person is committing fraud or when a person is about to commit fraud.

[0028] Here, fraud refers to criminal acts, violence, intimidation, and the like. For example, fraud in a store includes shoplifting and robbery. Generally, when a person commits fraud or is about to commit fraud, the person experiences emotional changes, such as increased tension and stress. These emotional changes are reflected in vital data. For example, changes in vital data when a person commits fraud or is about to commit fraud include an increase in heart rate, an increase in skin temperature, an increase in respiratory rate, and dilation of pupils. That is, a condition for vital data indicating fraud may be that a value indicated by the vital data exceeds a predetermined threshold or that a change in a value indicated by the vital data exceeds a predetermined threshold. Furthermore, for example, a condition for vital data indicating fraud may be that a change in vital data is similar to a change in vital data indicating fraud. Here, similarity may mean that the difference is equal to or less than a certain value or that the similarity is greater than a threshold. The condition for vital data indicating fraud is not limited to these examples and may be determined appropriately depending on the vital data used. The first determination unit 102 may determine whether the vital data of the person satisfies the conditions for vital data in the case of committing fraud.

[0029] The first determination unit 102 may make a determination using multiple types of vital data. For example, the first determination unit 102 may determine whether the heart rate and the degree of pupil dilation meet the vital data conditions for committing fraud, but the combination of vital data determined by the first determination unit 102 is not limited to this example.

[0030] The video acquisition unit 103 is an example of a video acquisition means for acquiring a video of a person. The video of the person is used by the second determination unit 104, which will be described later, to make a determination regarding the situation in which the person is placed. The video acquisition unit 103 acquires the video from a camera that captures the person. As described above, the camera may be a surveillance camera or a camera on a terminal device used for scan shopping. The timing at which the video acquisition unit 103 acquires the video of the person is not particularly limited. For example, the video acquisition unit 103 may acquire the video of the person when the first determination unit 102 determines that the vital data meets the conditions for fraudulent activity.

[0031] For example, the video acquisition unit 103 acquires video of a person whose vital data is determined to meet the vital data conditions for fraudulent use. For example, when acquiring vital data measured by a measurement sensor provided in a terminal device used by the person, the video acquisition unit 103 may identify a camera whose shooting range includes the position indicated in the location information of the terminal device based on the location information of the terminal device equipped with the measurement sensor, and acquire video captured by the identified camera. Furthermore, when the vital data acquisition unit 101 acquires vital data based on video, the video acquisition unit 103 may acquire the video used to acquire the vital data. For example, when the vital data acquisition unit 101 extracts vital data from the acquired video, the video acquisition unit 103 may acquire the video used to extract the vital data. Furthermore, for example, when the vital data acquisition unit 101 acquires vital data extracted from the video from another device, the video acquisition unit 103 may acquire the video used to extract the vital data from the camera or another device.

[0032] Furthermore, the video acquisition unit 103 acquires video at the time of vital data acquisition, which is the time when vital data is acquired from the person. For example, the video acquisition unit 103 acquires video at the time when vital data that meets the conditions for vital data in the case of fraud is acquired from the person. The time when vital data is acquired from the person is the time when the vital data acquisition unit 101 measures the vital data if it acquires it from a measurement sensor, or is the time when the video is captured if the vital data acquisition unit 101 acquires vital data based on video.

[0033] Furthermore, for example, the video acquisition unit 103 may acquire video from a predetermined time before the time when vital data is acquired from the person. This is because, at the time when vital data that meets the vital data conditions for fraudulent activity is acquired from the person, circumstances such as the person's behavior prior to that time may affect the person's vital data. For example, a person with a high heart rate at a certain time may have had a high heart rate because they ran prior to that time, and their heart rate may also be high if they were walking at that time. The predetermined time before the time when vital data is acquired from the person from which the video acquisition unit 103 acquires video may be a time that affects the vital data at the time when the vital data is acquired, and may be set appropriately.

[0034] The second determination unit 104 is an example of a second determination means that determines whether the person's condition based on the video matches the condition expected from the person's state represented by the vital data. For example, when the first determination unit 102 determines that the vital data matches the vital data conditions for fraud, the second determination unit 104 determines whether the person's condition based on the video of the person acquired by the video acquisition unit 103 matches the person's state represented by the vital data.

[0035] The situation of a person is at least one of the person's behavior, the person's movements, and the person's surrounding environment. For example, the second determination unit 104 may detect the person's movements and the person's actions from a video of the person using a known method. The surrounding environment of the person may be, for example, the movements and actions of other people present around the person detected from the video. In this case, for example, the second determination unit 104 may detect the movements and actions of other people present within a predetermined distance range from the person using a known method from the video of the person. The surrounding environment of the person may be detected from sounds detected around the person. In this case, for example, the second determination unit 104 may detect sounds included in the video of the person using a known method.

[0036] Even if the vital data meets the conditions for vital data that constitute fraud, the vital data may also meet the conditions for vital data that constitute fraud due to factors other than fraud. Therefore, the second determination unit 104 uses video to determine the status of a person from whom vital data that meets the conditions for vital data that constitute fraud is acquired. Accordingly, the second determination unit 104 determines whether the status of a person based on the video of the person matches the state of the person represented by the vital data. In other words, if there is a situation that could be a contributing factor to the person's state of tension or emotion represented by the vital data, the second determination unit 104 determines that the status of a person based on the video of the person from whom vital data that meets the conditions for vital data that constitute fraud matches the situation expected from the state of the person represented by the vital data. If this determination is made, the person is not treated as a suspicious person. On the other hand, if the vital data meets the conditions for vital data in the case of committing fraud and it is not determined that there is a contributing situation, the second determination unit 104 determines that the situation of the person based on the video of the person from whom vital data that meets the conditions for vital data in the case of committing fraud is a situation that does not match the situation expected from the state of the person represented by the vital data.If it is determined in this way, the person can be treated as a suspicious person.

[0037] Here, factors other than fraud that meet the conditions of vital data when a person's vital data is used to commit fraud include, for example, situations in which the person is under physical stress or situations in which the person is under mental stress due to external factors.

[0038] An example of a situation that places a physical strain on a person is when the person is exercising. For example, an example of a situation that places a physical strain on a person is when the person is running. In this case, the second determination unit 104 determines that the person is running based on video of the person. For example, the second determination unit 104 may determine that the person is running based on the moving speed of the person identified from the video, or may determine that the person is running based on the posture of the person identified from the video. Furthermore, for example, an example of a situation that places a physical strain on a person is when the person is carrying a heavy object. In this case, the second determination unit 104 may determine that the person is carrying a heavy object based on the size of the person's luggage identified from the video.

[0039] A situation in which a person is under mental stress due to an external factor is, for example, a situation in which the person is surprised or feels uncomfortable due to an external factor. For example, a situation in which a person is under mental stress due to an external factor is when the person is surprised by a loud noise. In this case, the second determination unit 104 determines, based on video of the person including audio, that a loud sound has been heard around the person, with a value representing the volume of the sound equal to or greater than a threshold. In this case, the second determination unit 104 may also determine that a loud sound has suddenly been heard around the person. This is because, in a situation in which a loud sound continues to be heard, the person is accustomed to loud sounds and is unlikely to be surprised. In addition, for example, a situation in which a person is under mental stress due to an external factor is when the person bumps into another person or another person's luggage. In this case, the second determination unit 104 may determine, based on video of the person, that the person has come into contact with another person or another person's luggage.

[0040] Furthermore, if the second determination unit 104 determines that the person's condition based on the video of the person at the time the above-mentioned vital data was acquired does not match the condition expected from the person's state represented by the vital data, the second determination unit 104 may make a similar determination on the condition based on the video of the person at a predetermined time before the time the above-mentioned vital data was acquired from the person. In this case, the video acquisition unit 103 may acquire video from a predetermined time before the time the vital data was acquired in response to the determination that the person's condition based on the video at the time the vital data was acquired does not match the condition expected from the person's state represented by the vital data.

[0041] As described above, the condition of a person represented by vital data may be affected by the situation at a time before the vital data was acquired, and therefore the second determination unit 104 makes a determination based on the situation based on video of the person at a predetermined time before the vital data was acquired from the person. However, if the second determination unit 104 makes a determination based on the situation at a time before the vital data was acquired every time, the processing load will be high. In contrast, as described above, when it is determined that the situation of a person based on the video of the person at the time the vital data was acquired does not match the situation expected from the person's condition represented by the vital data, the processing load can be reduced by having the second determination unit 104 make a determination based on the situation of the person based on video of the person at a predetermined time before the vital data was acquired.

[0042] The output unit 105 is an example of an output means that outputs an alert when the second determination unit 104 determines that the person's condition does not match the condition expected from the person's state represented by the vital data. The output unit 105 may output the alert to at least one of a display, a speaker, or a terminal device.

[0043] First, the output destination to which the output unit 105 outputs an alert will be described.

[0044] The output unit 105 may output an alert to a person (hereinafter referred to as a suspicious person) whose vital data is determined by the first determination unit 102 to meet the vital data conditions for fraudulent activity and whose vital data is determined by the second determination unit 104 to not conform to the situation expected from the person's condition represented by the vital data. In this case, the output unit 105 may output the alert to, for example, an output device near the position of the person identified from the video or information about the camera that captured the video. The camera information may be camera location information, information indicating the camera's shooting range, or identification information of the camera associated with such information. Furthermore, for example, the output unit 105 may output the alert to a terminal device used to acquire the vital data video.

[0045] The output unit 105 may also output an alert to a person other than the suspicious person, such as an employee. In this case, the output unit 105 may output the alert to a predetermined terminal device, etc. For example, the output unit 105 may output the alert to a terminal device used by an employee of the facility where the suspicious person is staying.

[0046] Furthermore, if the suspicious person is an employee, the output unit 105 may output an alert to a terminal device used by a facility manager. Whether the suspicious person is an employee may be determined based on the fact that the person's clothing in the video is an employee uniform, that the person in the video has an employee ID card, or information that previously associates the terminal device that acquired the vital data with the employee's identification information.

[0047] Next, the contents of the alert output by the output unit 105 will be described.

[0048] An alert to the suspicious person is issued, for example, to prevent the suspicious person from committing fraudulent acts. For example, the alert to the suspicious person may be audio or video that warns the suspicious person against fraudulent acts, or audio that attracts the attention of people around. Audio that attracts the attention of people around is thought to make it more difficult for the suspicious person to commit fraud. The content of the audio that attracts the attention of people around is not particularly limited, and may be audio that warns the suspicious person against fraudulent acts, audio that advertises a store or the like, or audio that asks whether action is required, such as "Are you in trouble?"

[0049] An alert to someone other than the suspicious person is issued, for example, to have a person other than the suspicious person, such as an employee, take action to prevent the suspicious person from committing fraud. An alert to someone other than the suspicious person may also be issued to identify the suspicious person when fraud is discovered. For example, an alert to someone other than the suspicious person includes information that can identify the suspicious person. The information that can identify the suspicious person may be at least one of the following: the current location of the suspicious person, the location of the suspicious person at the time of acquiring the suspicious person's vital data, and an image of the suspicious person.

[0050] FIG. 3 is an example of an alert output screen for an employee. In FIG. 3, the alert output screen includes information indicating that a suspicious person has been detected, the suspicious person's location within the store, and video of the suspicious person. Also, in FIG. 3, the alert output screen includes an "Alert" button. For example, when the "Alert" button is selected on the alert output screen for an employee, the output unit 105 may output an alert informing other employees or security guards that a suspicious person has been detected. Furthermore, for example, when the above button is selected, the output unit 105 may output an alert to the person detected as a suspicious person. In this way, allowing an employee to decide whether to issue an alert at their own discretion may prevent the employee from accidentally alerting a customer or other person, or from provoking a person who is becoming increasingly aggressive.

[0051] Next, a case where it is determined that the state of a person based on the video matches the state expected from the state of the person represented by the vital data will be described.

[0052] For example, if the output unit 105 determines that the person's situation based on the video matches the situation expected from the person's condition represented by the vital data, the output unit 105 does not output an alert. Furthermore, if the output unit 105 determines that the person's situation based on the video matches the situation expected from the person's condition represented by the vital data, the output unit 105 may output information indicating that the person is not a suspicious person. For example, the output unit 105 may superimpose on each person appearing in the video a message indicating whether the person's situation based on the video matches the situation expected from the person's condition represented by the vital data in an identifiable manner.

[0053] Furthermore, for example, the output unit 105 may output information about a person whose vital data meets the conditions for when the vital data commits fraud. For example, when the output unit 105 determines that the situation of a person based on the video matches the situation expected from the state of the person represented by the vital data, the output unit 105 may output that the situation of a person whose vital data meets the conditions for when the vital data commits fraud matches the situation expected from the state of the person represented by the vital data. For example, the output unit 105 may output the situation of a person based on the video, such as that a person whose vital data meets the conditions for when the vital data commits fraud is running.

[0054] Furthermore, the output unit 105 may output an alert if the vital data acquisition unit 101 is unable to acquire the vital data of an employee wearing a measurement sensor for a predetermined period of time or longer. This makes it possible to prevent fraud even when an employee who knows that the vital data is used to detect suspicious persons intentionally prevents the acquisition of vital data in an attempt to commit fraud without being detected as a suspicious person.

[0055] The operation of the suspicious person detection device 100 configured as above will be described with reference to the flowchart of FIG.

[0056] 4, first, the vital data acquisition unit 101 acquires vital data of a person (step S101). Vital data may be constantly acquired from each person included in the facility.

[0057] Next, the first determination unit 102 determines whether the vital data of the person acquired in step S101 meets the conditions for vital data in the case of committing fraud (step S102). If it is determined in step S102 that the vital data of the person does not meet the conditions for vital data in the case of committing fraud (step S102: No), the suspicious person detection device 100 ends the process.

[0058] If it is determined in step S102 that the vital data of the person meets the conditions for vital data in the case of committing fraud (step S102: Yes), the video acquisition unit 103 acquires video of the person (step S103).

[0059] Next, the second determination unit 104 determines whether the person's condition based on the video of the person acquired in step S103 matches the condition expected from the person's state represented by the vital data acquired in step S101 (step S104). If it is determined in step S104 that the person's condition based on the video matches the condition expected from the person's state represented by the vital data (step S104: Yes), the suspicious person detection device 100 may end the process.

[0060] If it is determined that the person's condition based on the video does not match the condition expected from the person's state represented by the vital data (step S104: No), the output unit 105 outputs an alert.

[0061] This completes the series of operations of the suspicious person detection device 100. The series of processes may be repeated. The order of the processes may be changed as appropriate. For example, the step of acquiring the image of the person in step S103 may be performed before step S101 or simultaneously with step S101.

[0062] In the suspicious person detection device 100 of the present embodiment described above, the first determination unit 102 determines the state of the person represented by the vital data, the second determination unit 104 determines whether the state of the person based on the video of the person matches the state expected from the state of the person represented by the vital data, and the output unit 105 outputs an alert if it is determined that the state of the person does not match the state expected from the state of the person represented by the vital data.

[0063] As a result, the suspicious person detection device 100 in this embodiment can detect suspicious persons more accurately than when detecting suspicious persons based solely on vital data, because it does not issue an alert when the vital data meets the conditions for fraud due to factors other than the fact that the vital data indicates fraud.

[0064] [Hardware Configuration] Some or all of the components of each device or system in the embodiment of the present disclosure described above are realized by any combination of an information processing device 1000 and a program, for example, as shown in Fig. 5. The information processing device 1000 includes, as an example, the following configuration.

[0065] - CPU (Central Processing Unit) 1001 - ROM (Read Only Memory) 1002 - RAM (Random Access Memory) 1003 - Program 1004 loaded into RAM 1003 - Storage device 1005 for storing program 1004 - Drive device 1007 for reading and writing from and to recording medium 1006 - Communication I / F 1008 for connecting to communication network 1009 - Input / output I / F 1010 for inputting and outputting data - Bus 1011 for connecting each component. Note that I / F is an abbreviation for Interface.

[0066] Each component of each device or system in each embodiment is realized by the CPU 1001 acquiring and executing a program that realizes the function of that component. The program that realizes the function of each component of each device is stored in the storage device 1005 or RAM 1003 in advance, for example, and is read by the CPU 1001 as needed. The program 1004 may be supplied to the CPU 1001 via a communication network, or may be stored in the recording medium 1006 in advance, and the drive device 1007 may read the program and supply it to the CPU 1001.

[0067] There are various variations in the method of realizing each device. For example, each device or system may be realized by any combination of a separate information processing device 1000 and a program for each component. Furthermore, multiple components included in each device may be realized by any combination of a single information processing device 1000 and a program.

[0068] Furthermore, some or all of the components of each device or system may be realized by general-purpose or dedicated circuits (circuitry) including a processor or the like, or a combination of these. Examples of circuits include a CPU, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and an AI (Artificial Intelligence) processing LSI (Large Scale Integration). These may be configured by a single chip or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits and a program.

[0069] When some or all of the components of each device or system are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in which they are connected via a communication network.

[0070] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0071] Furthermore, although the operations are described in a sequential order in the form of a flowchart, the order of description does not limit the order in which the operations are performed. Therefore, when implementing each embodiment, the order of the operations may be changed as long as it does not affect the content.

[0072] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0073] [Supplementary Note 1] A suspicious person detection device comprising: an image acquisition means for acquiring an image of a person; a vital data acquisition means for acquiring vital data of the person; a first determination means for determining the state of the person represented by the vital data; a second determination means for determining whether the state of the person based on the image matches a state expected from the state of the person represented by the vital data; and an output means for outputting an alert when it is determined that the state of the person based on the image does not match a state expected from the state of the person represented by the vital data.

[0074] [Supplementary Note 2] The suspicious person detection device according to Supplementary Note 1, wherein the first determination means determines that the state of the person represented by the vital data is a state that may lead to a case where the person commits fraud.

[0075] [Appendix 3] The suspicious person detection device described in Appendix 1 or 2, wherein the second determination means determines that the situation is consistent with a situation expected from the state of the person represented by the vital data if the situation is one in which the person is under physical stress.

[0076] [Supplementary Note 4] The suspicious person detection device according to Supplementary Note 3, wherein the situation in which the person is under physical strain is a situation in which the person is running.

[0077] [Supplementary Note 5] The suspicious person detection device according to Supplementary Note 3 or 4, wherein the situation in which the person is under physical strain is a situation in which the person is carrying a large amount of baggage.

[0078] [Appendix 6] The suspicious person detection device described in any of Appendices 1 to 5, wherein the second determination means determines that the situation conforms to a situation expected from the state of the person represented by the vital data if the situation is one in which there is an external factor that places mental stress on the person.

[0079] [Supplementary Note 7] The suspicious person detection device according to Supplementary Note 6, wherein the situation in which there is an external factor that places a mental burden on the person is a situation in which a loud noise is heard near the person.

[0080] [Supplementary Note 8] The suspicious person detection device according to Supplementary Note 6 or 7, wherein the situation in which there is an external factor that puts a mental stress on the person is a situation in which the person bumps into another person.

[0081] [Supplementary Note 9] The suspicious person detection device according to any one of Supplementary Notes 1 to 8, wherein the video is video at a time point when the vital data is acquired from the person.

[0082] [Supplementary Note 10] The suspicious person detection device described in Supplementary Note 9, wherein the video acquisition means, when determining that the person's condition based on the video at the time of acquisition does not match the person's condition assumed from the person's state represented by the vital data, acquires video from a predetermined time before the time of acquisition, and the second determination means determines whether the person's condition based on the video from a predetermined time before the time of acquisition matches the person's condition assumed from the person's state represented by the vital data.

[0083] [Supplementary Note 11] The suspicious person detection device according to any one of Supplementary Notes 1 to 10, wherein the output means outputs an alert to an output device in the vicinity of the person.

[0084] [Supplementary Note 12] The suspicious person detection device according to any one of Supplementary Notes 1 to 11, wherein the output means outputs an alert to a terminal device used by an employee of a facility where the person is staying.

[0085] [Supplementary Note 13] The suspicious person detection device according to Supplementary Note 12, wherein the output means outputs an alert to a terminal device used by a manager of the facility if the person is an employee of the facility.

[0086] [Supplementary Note 14] The suspicious person detection device according to Supplementary Note 12 or 13, wherein the output means outputs an alert including the vital data and the video.

[0087] [Supplementary Note 15] The suspicious person detection device according to any one of Supplementary Notes 1 to 14, wherein the output means outputs an alert if the person is an employee of a facility and non-video vital data cannot be obtained for a predetermined period of time or longer.

[0088] [Supplementary Note 16] A suspicious person detection method, in which a computer acquires video of a person, acquires vital data of the person, determines the state of the person represented by the vital data, determines whether the state of the person based on the video matches a state expected from the state of the person represented by the vital data, and outputs an alert if it is determined that the state of the person based on the video does not match a state expected from the state of the person represented by the vital data.

[0089] [Supplementary Note 17] A recording medium that records a program that causes a computer to execute the following processes: acquiring video of a person; acquiring vital data of the person; determining the state of the person represented by the vital data; determining whether the state of the person based on the video matches a state that can be assumed from the state of the person represented by the vital data; and outputting an alert if it is determined that the state of the person based on the video does not match a state that can be assumed from the state of the person represented by the vital data.

[0090] REFERENCE SIGNS LIST 100 Suspicious person detection device 101 Vital data acquisition unit 102 First determination unit 103 Video acquisition unit 104 Second determination unit 105 Output unit 1000 Information processing device 1001 CPU 1002 ROM 1003 RAM 1004 Program 1005 Storage device 1006 Recording medium 1007 Drive device 1008 Communication I / F 1009 Communication network 1010 Input / output I / F 1011 Bus

Claims

1. an image acquisition means for acquiring an image of a person; a vital data acquisition means for acquiring vital data of the person; a first determination means for determining the state of the person represented by the vital data; a second determination means for determining whether the state of the person based on the video image matches a state assumed from the state of the person represented by the vital data; an output means for outputting an alert when it is determined that the condition of the person based on the video does not match the condition expected from the state of the person represented by the vital data; A suspicious person detection device comprising:

2. The first determination means determines whether the state of the person represented by the vital data is a state that may lead to fraud. The suspicious person detection device according to claim 1 .

3. When the situation is a situation in which the person is under physical stress, the second determination means determines that the situation matches a situation expected from the state of the person represented by the vital data. The suspicious person detection device according to claim 1 .

4. When the situation is a situation in which there is an external factor that places a mental burden on the person, the second determination means determines that the situation matches a situation expected from the state of the person represented by the vital data. The suspicious person detection device according to claim 1 .

5. The situation in which there is an external factor that puts a mental load on the person is at least one of a situation in which a loud noise is heard near the person and a situation in which the person collides with another person. The suspicious person detection device according to claim 4.

6. when it is determined that the state of the person based on the image at the time when the vital data is acquired from the person does not match the state of the person assumed from the state of the person represented by the vital data, the image acquisition means acquires an image from a predetermined time before the acquisition time, The second determination means determines whether the state of the person based on the video image taken at a predetermined time before the acquisition time point matches the state of the person assumed from the state of the person represented by the vital data. The suspicious person detection device according to claim 1 .

7. The output means outputs an alert to a terminal device used by an employee of a facility where the person is staying. The suspicious person detection device according to claim 1 .

8. The output means outputs an alert when the person is an employee of a facility and non-video vital data cannot be obtained for a predetermined period of time or longer. The suspicious person detection device according to claim 1 .

9. The computer Capture a picture of the person acquiring vital data of the person; determining a state of the person represented by the vital data; determining whether the situation of the person based on the video matches a situation expected from the state of the person represented by the vital data; If it is determined that the condition of the person based on the video does not match the condition expected from the state of the person represented by the vital data, an alert is output. Method for detecting suspicious persons.

10. Capture a picture of the person acquiring vital data of the person; determining a state of the person represented by the vital data; determining whether the situation of the person based on the video matches a situation expected from the state of the person represented by the vital data; If it is determined that the condition of the person based on the video does not match the condition expected from the state of the person represented by the vital data, an alert is output. A program that causes a computer to perform a process.