Suspicious person detection device and method

Abstract

The objective of the present disclosure is to detect a suspicious person accurately. A suspicious person detection device (101) according to the present disclosure comprises: a first suspiciousness detection unit (11) that detects, from a captured image from a monitoring camera (201), a person in a first suspicious state that is a mental state predetermined as a state having a high probability to do harm to other persons, the first suspiciousness detection unit (11) creating first suspiciousness information including specific information of the person in the first suspicious state; a second suspiciousness detection unit (12) that detects, from the captured image, a person in a second suspicious state exhibiting a behavior predetermined as a sign of doing harm to other persons, and that creates second suspiciousness information including specific information of the person in the second suspicious state; and a suspicious person determination unit (13) that identifies a suspicious person on the basis of the first suspiciousness information and the second suspiciousness information, and creates suspicious person information including the specific information of the suspicious person.

Description

[Technical Field]

[0001] The present disclosure relates to a technique for detecting a suspicious person. [Background technology]

[0002] Patent document 1 discloses a human intention estimation device that can automatically detect people with specific intentions as suspicious by extracting multiple pieces of information indicating behavioral intentions from gaze direction information of people detected in video. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-006427 Summary of the Invention [Problem to be solved by the invention]

[0004] The human intention estimation device in Patent Document 1 calculates multiple gaze feature amounts that represent the behavioral intention of a person from the gaze direction information of the person, and if any one of the gaze feature amounts is equal to or greater than a threshold, the person is judged to be suspicious. Therefore, there is a risk that a person who does not actually intend to harm others may be mistakenly detected as suspicious if their gaze direction meets the conditions.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to detect suspicious individuals with high accuracy. [Means for solving the problem]

[0006] The suspicious person detection device of the present disclosure includes: Installed in the train A person in a first suspicious state, which is a predetermined psychological state that is likely to cause harm to others, is detected from video footage captured by a surveillance camera, and specific information of the person in the first suspicious state is collected. and first suspicious state attribute information representing an attribute of the first suspicious state.a first suspiciousness detection unit that generates first suspiciousness information including: a first suspiciousness detection unit that detects a person in a second suspicious state who is performing a predetermined action as a sign of harming another person from the captured video; and a second suspiciousness detection unit that generates specific information of the person in the second suspicious state. and second suspicious state attribute information representing an attribute of the second suspicious state. a second suspicious detection unit that generates second suspicious information including the first suspicious information; situation and the second suspicious situation to A certain person Suspicious person and and a suspicious person determination unit that identifies the suspicious person and creates suspicious person information including the identification information of the suspicious person. The suspicious person determination unit determines a harm attribute, which is an attribute of harm that the suspicious person is likely to inflict on others, based on a combination of the first suspicious state attribute information and the second suspicious state attribute information, and includes the harm attribute information representing the harm attribute in the suspicious person information. The suspicious person determination unit determines whether the suspicious person corresponds to the harm attribute of "injury" or "molestation or theft." do. [Effects of the Invention]

[0007] The suspicious person detection device of the present disclosure detects a suspicious person using both a first suspicious state related to the person's psychology and a second suspicious state related to the person's behavior, and therefore can detect a suspicious person with high accuracy. Objects, features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of a suspicious person detection system according to first and third embodiments. [Figure 2] 4 is a flowchart showing the operation of the suspicious person detection device according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing the ID, attributes, and detection details of a first suspicious state. [Figure 4] FIG. 10 is a diagram showing the ID, attribute, behavior classification, and harm attribute of a second suspicious state. [Figure 5] FIG. 10 is a diagram showing a situation in which a suspicious person is detected on a train. [Figure 6] FIG. 10 is a configuration diagram of a suspicious person detection system according to a second embodiment. [Figure 7] 10 is a flowchart showing the operation of the suspicious person detection device according to the second embodiment. [Figure 8] FIG. 10 is a diagram showing how suspicious person information is displayed on the train vision screen. [Figure 9]It is a diagram showing a state where suspicious person information is notified to the station control room. [Figure 10] It is a flowchart showing the operation of the suspicious person detection device according to Embodiment 3. [Figure 11] It is a configuration diagram of the suspicious person detection system according to Embodiment 4. [Figure 12] It is a diagram showing the hardware configuration of the suspicious person detection device. [Figure 13] It is a diagram showing the hardware configuration of the suspicious person detection device.

Embodiments for Carrying Out the Invention

[0009] <A. Embodiment 1> <A-1. Configuration> FIG. 1 is a configuration diagram of a suspicious person detection system 1001 according to Embodiment 1. The suspicious person detection system 1001 includes a suspicious person detection device 101 and a surveillance camera 201.

[0010] The surveillance camera 201 is installed at a position such that an area where a suspicious person is to be detected (hereinafter, the suspicious person detection area) is included in the shooting range. For example, when the suspicious person detection area is inside a train, the surveillance camera 201 is installed inside the train. In this embodiment, the suspicious person detection area is inside the train.

[0011] The suspicious person detection device 101 includes a first suspicious detection unit 11, a second suspicious detection unit 12, and a suspicious person determination unit 13.

[0012] Based on the captured video of the surveillance camera 201, the first suspicious detection unit 11 detects passengers on the train in the first suspicious state. The first suspicious state is a psychological state predetermined as a state with a high possibility of harming others. When the first suspicious detection unit 11 detects a person in the first suspicious state, it creates first suspicious information including specific information, which is information for identifying that person.

[0013] The second suspicious state detection unit 12 detects passengers of a train in a second suspicious state based on the captured video of the surveillance camera 201. The second suspicious state refers to a state in which a person is performing actions predetermined as signs of harming others. When the second suspicious state detection unit 12 detects a person in the second suspicious state, it creates second suspicious information including specific information, which is information for identifying the person.

[0014] The suspicious person determination unit 13 determines a person who meets both the first suspicious state and the second suspicious state as a suspicious person based on the first suspicious information obtained from the first suspicious state detection unit 11 and the second suspicious information obtained from the second suspicious state detection unit 12, and creates suspicious person information.

[0015] <A-2. Operation> Figure 2 is a flowchart showing the operation of the suspicious person detection device 101. Hereinafter, the operation of the suspicious person detection device 101 will be described with reference to the flowchart of Figure 2.

[0016] When the suspicious person detection system 1001 starts operating, in step S101, the first suspicious state detection unit 11 acquires the captured video from the surveillance camera 201 and executes the first suspicious state detection process. In the first suspicious state detection process, the first suspicious state detection unit 11 detects a person in the first suspicious state from the persons shown in the captured video of the surveillance camera 201.

[0017] The first suspicious state refers to a mental state or a physiological state predetermined as a suspicious state with a high possibility of causing harm to a person. Figure 3 shows an example of the first suspicious state. In Figure 3, the first column shows the ID of the first suspicious state, the second column shows the attribute of the first suspicious state, and the third column shows the detection content of the first suspicious state.

[0018] For example, attributes of the first suspicious state include "emotionally unstable," "excited," and "malice." The first suspicious state with the "emotionally unstable" attribute is detected by an unstable gaze, wandering behavior, or a specific facial expression. The first suspiciousness detection unit 11 identifies the gaze direction of a person captured in the captured video, and if the gaze direction of a certain person changes more than a predetermined number of times within a predetermined time, it determines that the person has an "unstable gaze" and is in the first suspicious state with the "emotionally unstable" attribute. Here, the first suspiciousness detection unit 11 may determine "unstable gaze" using the direction of the person's face instead of the gaze direction. Furthermore, the first suspiciousness detection unit 11 recognizes the position of a person captured in the captured video, and if a certain person is wandering unrelated to specific equipment or objects, it determines that the person has "wandering behavior" and is in the first suspicious state with the "emotionally unstable" attribute. In addition, the first suspiciousness detection unit 11 recognizes the facial expressions of people appearing in the captured video, and when it detects that a person has a blank or timid expression, it determines that the person is in a first suspicious state with the attribute "emotionally unstable."

[0019] The first suspicious state of the "excited" attribute is detected by head-shaking behavior, trembling, or a specific facial expression. For example, when the first suspiciousness detection unit 11 detects that a person is continuously shaking their head up and down or from side to side for a predetermined period of time or more, it determines that the person is in the first suspicious state of the "excited" attribute. Furthermore, when the first suspiciousness detection unit 11 detects that a person is trembling their fists, shoulders, or other body parts, it determines that the person is in the first suspicious state of the "excited" attribute. Furthermore, when the first suspiciousness detection unit 11 detects that a person has slanted eyes, a tic, an excited facial expression or complexion, or that a person is muttering something with their mouth open, it determines that the person is in the first suspicious state of the "excited" attribute.

[0020] The first suspicious state of the "malicious" attribute is detected by a specific facial expression, clothing typically worn by people who harm others, or a specific gaze or behavior. For example, when the first suspiciousness detection unit 11 detects an angry facial expression on a person's face or detects that a person is shouting loudly, it determines that the person is in the first suspicious state of the "malicious" attribute. Furthermore, when the first suspiciousness detection unit 11 detects that a person is wearing combat-style clothing such as a military uniform or a mask, it determines that the person is in the first suspicious state of the "malicious" attribute. Furthermore, when the first suspiciousness detection unit 11 detects that a person is gazing at or checking a specific object or other person, it determines that the person is in the first suspicious state of the "malicious" attribute.

[0021] Next, in step S102, the second suspiciousness detection unit 12 acquires the video footage from the surveillance camera 201 and executes a second suspicious state detection process. In the second suspicious state detection process, the second suspiciousness detection unit 12 detects a person in a second suspicious state from among people appearing in the video footage captured by the surveillance camera 201.

[0022] The second suspicious state is a state in which a person is engaging in behavior that is predetermined as a precursor to harming others. Figure 4 shows an example of the second suspicious state. In Figure 4, the first column shows the ID of the second suspicious state, the second column shows the attribute of the second suspicious state, the third column shows the behavior classification of the second suspicious state, and the fourth column shows the harm attribute of the second suspicious state.

[0023] For example, attributes of the second suspicious state include approaching behavior, contacting behavior, possession of a dangerous substance, taking an object from clothing, reporting a suspicious object, searching behavior, and surveillance camera-aware behavior. Behavior classifications of the second suspicious state include direct behavior and indirect behavior. Approaching behavior, contacting behavior, possession of a dangerous substance, taking an object from clothing, reporting a suspicious object, and searching behavior are direct behaviors, and surveillance camera-aware behavior is an indirect behavior.

[0024] Approaching behavior is detected as a behavior of clinging to and approaching a specific person. Harmful attributes of approaching behavior include suspiciousness, theft, and molestation.

[0025] A contact behavior is detected as a movement of bringing a body part or an object close to a specific person. Harm attributes of a contact behavior are suspiciousness, theft, and molestation.

[0026] Possession of dangerous substances is detected as possession of a weapon or a photographic device. The harm attributes of possession of dangerous substances are injury and robbery.

[0027] The act of taking an object out of clothing is detected as an action of taking something out of clothing. The harm attributes of the act of taking an object out of clothing are suspicious and injury.

[0028] Leaving a suspicious object is detected as the behavior of leaving an object such as a bag or briefcase whose contents cannot be seen. The harm attribute of leaving a suspicious object is leaving a dangerous object.

[0029] A searching behavior is detected as a behavior of searching for a specific item. The harm attribute of the searching behavior is theft.

[0030] Surveillance camera-aware behavior is detected as behavior of moving into a blind spot of a surveillance camera, behavior of checking the surveillance camera, or behavior of avoiding a police officer. For example, the second suspiciousness detection unit 12 may determine that a person has engaged in surveillance camera-aware behavior when it detects that the person has turned their gaze toward the surveillance camera 201 a predetermined number of times or more within a predetermined period of time. The harm attribute of surveillance camera-aware behavior is suspicious.

[0031] Next, in step S103, the suspicious person determination unit 13 determines whether or not there is a person who corresponds to both the first suspicious state and the second suspicious state. If there is a person who corresponds to both the first suspicious state and the second suspicious state, the suspicious person determination unit 13 determines that person as a suspicious person in step S104 and creates suspicious person information. If there is no person who corresponds to both the first suspicious state and the second suspicious state, the processing of the suspicious person detection device 101 returns to step S101, and the above processing is repeated.

[0032] FIG. 5 is a diagram showing a situation of detecting a suspicious person in a train. A surveillance camera 201 is installed in the train and is photographing the passengers in the train. Assume that person A has a malicious expression and is approaching person B. Here, the first suspicious detection unit 11 determines that person A is in a first suspicious state (ID = 1-3-1) by detecting the malicious expression of person A. In addition, the second suspicious detection unit 12 detects that person A is approaching person B and determines that person A is in a second suspicious state (ID = 2-1-1). The suspicious person determination unit 13 determines person A as a suspicious person because person A meets both the first suspicious state and the second suspicious state.

[0033] <A-3. Effect> The suspicious person detection device 101 according to Embodiment 1 includes a first suspicious detection unit 11, a second suspicious detection unit 12, and a suspicious person determination unit 13. The first suspicious detection unit 11 detects a person in a first suspicious state, which is a mental state predetermined as a state with a high possibility of harming others, from the photographed video of the surveillance camera 201. In addition, the first suspicious detection unit 11 creates first suspicious information including the specific information of the person in the first suspicious state. The second suspicious detection unit 12 detects a person in a second suspicious state who is performing an action predetermined as a sign of harming others from the photographed video of the surveillance camera 20. In addition, the second suspicious detection unit 12 creates second suspicious information including the specific information of the person in the second suspicious state. The suspicious person determination unit 13 identifies a suspicious person based on the first suspicious information and the second suspicious information, and creates suspicious person information including the specific information of the suspicious person. With the above configuration, the suspicious person detection device 101 can detect a suspicious person with high accuracy using both the mental state and the sign of harming behavior.

[0034] <A-4. Modification Example> In the above description, the first suspicious detection unit 11 detected a person in the first suspicious state from the photographed video of the surveillance camera 201. In order to detect a person in the first suspicious state, the first suspicious detection unit 11 may use other devices such as a millimeter-wave radar, a voice recognition device, an infrared sensor, an ultrasonic sensor, and a Lidar in addition to the surveillance camera 201. Thereby, the detection accuracy of a person in the first suspicious state is improved.

[0035] For example, the first suspiciousness detection unit 11 may use a millimeter-wave radar to obtain the respiration rate of a person, and determine that a person with a respiration rate higher than a threshold value, that is, a person with rough breathing, is a person in a first suspicious state of an excitement attribute.

[0036] Alternatively, the first suspiciousness detection unit 11 may use a voice recognition device to recognize the voice of a person, and determine whether the person is in a first suspicious state from the voice of the person.

[0037] In the above description, the suspicious person detection area of the suspicious person detection device 101 is set inside the train, but the suspicious person detection area may be inside other moving bodies such as buses, ships, and airplanes. Further, the suspicious person detection area is not limited to inside a moving body, and may be inside a station building, a platform, a department store, a store, a commercial facility such as a sports gym, an event venue, a passage, or a road.

[0038] <B. Embodiment 2> <B-1. Configuration> FIG. 6 is a configuration diagram of a suspicious person detection system 1002 according to Embodiment 2. The suspicious person detection system 1002 includes a suspicious person detection device 102, a surveillance camera 201, a train vision 202, and a control room 203. The suspicious person detection device 102 includes a notification unit 14 in addition to the configuration of the suspicious person detection device 101 according to Embodiment 1.

[0039] The notification unit 14 notifies the outside of the suspicious person detection device 102 of the suspicious person information, which is the information of the suspicious person determined by the suspicious person determination unit 13.

[0040] The train vision 202 has a display device. The train vision 202 displays the suspicious person information acquired from the notification unit 14 on the display device included in the train vision 202, and notifies the passengers of the suspicious person information.

[0041] The control room 203 outputs the suspicious person information acquired from the notification unit 14 from a display device or an audio notification device included in the control room 203, and notifies the control room staff of the suspicious person information.

[0042] <B-2. Operation> FIG. 7 is a flowchart showing the operation of the suspicious person detection device 102. The flow of FIG. 7 is obtained by adding step S105 after step S104 in the flow showing the operation of the suspicious person detection device 102 according to the first embodiment shown in FIG. 2.

[0043] In step S105, the notification unit 14 notifies the suspicious person information to the train vision 202 or the control room 203.

[0044] FIG. 8 shows a state in which summary information 52 of the suspicious person situation saying "Suspicious person in car 2!" is displayed on the display device 51 of the train vision 202.

[0045] The control room 203 outputs the suspicious person information acquired from the notification unit 14 from a display device or an audio notification device included in the control room 203, and notifies the control room staff of the suspicious person information. FIG. 9 shows a state in which the suspicious person information is displayed on the display device 55 in the control room 203. On the display device 55, summary information 52 of the suspicious person situation saying "Suspicious person in car 2!", video 53 of the suspicious person and its surroundings, and video 54 showing the positions of the suspicious person and other passengers in the vehicle where the suspicious person exists are displayed.

[0046] The display shown in FIG. 9 is an example of the notification of the suspicious person information. The suspicious person information may be notified by voice. Further, more detailed information about the suspicious person and passengers may be displayed, or information useful for dealing with the suspicious person, such as the vehicle position where the suspicious person exists and the vehicle position where the conductor exists in the entire vehicle of this train, may be displayed.

[0047] <B-3. Effect> The suspicious person detection device 102 according to the second embodiment includes a notification unit 14 that notifies suspicious person information outside the suspicious person detection device 102. Therefore, according to the suspicious person detection device 102, the suspicious person information can be notified outside the suspicious person detection device 102.

[0048] <B-4. Modification Example> When the first suspiciousness detection unit 11 detects a person in a first suspicious state, it creates information about the person as first suspiciousness information. The first suspiciousness information may include first suspicious state attribute information that represents an attribute of the first suspicious state. As shown in FIG. 3, the attribute of the first suspicious state is, for example, at least one of "emotional instability," "excitement," and "malice."

[0049] When the second suspiciousness detection unit 12 detects a person in the second suspicious state, it creates information about the person as second suspiciousness information. The second suspiciousness information may include second suspicious state attribute information that indicates attributes of the second suspicious state. As shown in FIG. 4, the attributes of the second suspicious state are, for example, at least one of "approaching behavior," "contact behavior," "possession of dangerous substance," "removing object from clothing," "leaving suspicious object," "searching behavior," and "surveillance camera-aware behavior."

[0050] When the suspicious person determination unit 13 detects a suspicious person, it creates suspicious person information, which is information about the suspicious person. The suspicious person information may include the first suspicious information or the second suspicious information. In this case, particularly in the second embodiment, there is an effect that detailed attributes of the suspicious person can be notified.

[0051] The suspicious person determination unit 13 may identify the harm attribute of the suspicious person based on the second suspicious state attribute information extracted by the second suspiciousness detection unit 12, and include harm attribute information representing the harm attribute in the suspicious person information. The harm attribute includes at least one of suspiciousness, theft, molestation, injury, robbery, and abandonment of dangerous objects. FIG. 4 shows the correspondence between the second suspicious state attribute information and the harm attribute. In this case, particularly in the second embodiment, there is an effect that detailed attributes of the suspicious person can be notified.

[0052] The suspicious person determination unit 13 may determine a person having a specific combination of a first suspicious state and a second suspicious state as a suspicious person having a specific harm attribute. For example, the suspicious person determination unit 13 may determine a person having a first suspicious state with a malicious attribute and a second suspicious state with a dangerous substance possession attribute or an object removal action attribute as a suspicious person having an injury attribute and create suspicious person information.

[0053] In addition, the suspicious person determination unit 13 may determine a person having a first suspicious state of an emotionally unstable attribute and a second suspicious state of a contact behavior attribute as a suspicious person having a molestation attribute or a theft attribute, and create suspicious person information.

[0054] In addition, the suspicious person determination unit 13 may determine a person having a first suspicious state of an emotionally unstable attribute and a second suspicious state of a suspicious object abandonment attribute as a suspicious person having a dangerous object abandonment attribute, and create suspicious person information.

[0055] <C. Embodiment 3> <C-1. Configuration> The configuration of the suspicious person detection system 1003 according to Embodiment 3 is as shown in FIG. 1, and is the same as that of the suspicious person detection system 1001 according to Embodiment 1. The suspicious person detection system 1003 includes a suspicious person detection device 103 and a surveillance camera 201. The suspicious person detection device 103 has the same configuration as the suspicious person detection device 101 according to Embodiment 1.

[0056] The first suspicious state detection unit 11 detects a passenger of a train in a first suspicious state based on the video of the surveillance camera 201. In the present embodiment, the first suspicious state is a psychological state predetermined as a state with a high possibility of harming others, and refers to a state in which a first suspicious degree, which is a psychological suspicious degree, is equal to or higher than a predetermined first threshold value. The first suspicious state detection unit 11 calculates the first suspicious degree of a person captured by the surveillance camera 201 by a predetermined psychological detection method, and determines a person with a first suspicious degree equal to or higher than the first threshold value as being in a first suspicious state. The first suspicious state detection unit 11 includes the first suspicious degree in the first suspicious information.

[0057] The second suspiciousness detection unit 12 detects passengers of a train in a second suspicious state based on the video of the surveillance camera 201. In the present embodiment, the second suspicious state is a state in which a person is performing an action predetermined as a sign of harming others, and refers to a state in which the second suspiciousness degree, which is a behavioral suspiciousness degree, is equal to or higher than a predetermined second threshold value. The second suspiciousness detection unit 12 calculates the second suspiciousness degree of a person captured by the surveillance camera 201 by a predetermined action detection method, and determines a person whose second suspiciousness degree is equal to or higher than the second threshold value as being in a second suspicious state. The second suspiciousness detection unit 12 includes the second suspiciousness degree in the second suspiciousness information.

[0058] The suspicious person determination unit 13 determines, based on the first suspiciousness information acquired from the first suspiciousness detection unit 11 and the second suspiciousness information acquired from the second suspiciousness detection unit 12, a person who meets both the first suspicious state and the second suspicious state and whose first suspiciousness degree and second suspiciousness degree satisfy a predetermined composite condition as a suspicious person, and creates suspicious person information.

[0059] <C-2. Operation> FIG. 10 is a flowchart showing the operation of the suspicious person detection device 103. Hereinafter, the operation of the suspicious person detection device 103 will be described along the flowchart of FIG. 10.

[0060] When the suspicious person detection system 1003 starts operating, in step S201, the first suspiciousness detection unit 11 acquires a captured video from the surveillance camera 201 and calculates the first suspiciousness degree PSY, which is the psychological suspiciousness degree of a person captured in the captured video. This process is the first suspiciousness state detection logic by the first suspiciousness detection unit 11 described in Embodiment 1, including the first suspiciousness degree PSY.

[0061] For example, in the case of detecting an unstable line of sight of the emotional instability attribute in ID1-1-1 of FIG. 3, the detection logic in Embodiment 1 is "detecting line of sight movement to different areas more than a predetermined number of times at a predetermined time". In contrast, in the present embodiment, the first suspiciousness detection unit 11 calculates such that the higher the number of line of sight movements, the higher the value of the first suspiciousness degree PSY, determines a person whose first suspiciousness degree PSY is higher than the first threshold value as being in a first suspicious state, and creates first suspiciousness information.

[0062] The first suspiciousness level PSY is a normalized value between 0 and 1. A person with a first suspiciousness level PSY of 0 is not likely to be suspicious. A person with a first suspiciousness level PSY of 1 is likely to be suspicious. The first threshold is, for example, 0.5.

[0063] Next, in step S202, the second suspiciousness detection unit 12 acquires the captured video from the surveillance camera 201 and calculates the second suspiciousness degree PHY, which is the behavioral suspiciousness degree of a person appearing in the captured video. This process includes the second suspiciousness degree PHY in addition to the detection logic of the second suspicious state by the second suspiciousness detection unit 12 described in the first embodiment.

[0064] For example, the detection logic for ID2-2-1 in Fig. 4, surveillance camera aware behavior, is "detect behavior of looking at surveillance camera" in the first embodiment. In contrast to this, in the present embodiment, the second suspiciousness detection unit 12 calculates the second suspiciousness level PHY so that the more times a person looks at a surveillance camera within a certain period of time, the higher the value of the second suspiciousness level PHY becomes, and determines that a person whose second suspiciousness level PHY is higher than the second threshold value is in a second suspicious state, and creates second suspiciousness information.

[0065] The second suspiciousness level PHY is a normalized value between 0 and 1. A person with a second suspiciousness level PHY of 0 is not likely to be suspicious. A person with a second suspiciousness level PHY of 1 is likely to be suspicious. The second threshold is, for example, 0.5.

[0066] Next, in step S203, the suspicious person determination unit 13 determines whether there is a person who is in the first suspicious state and the second suspicious state and who satisfies the predetermined combined condition of the first suspiciousness level PSY and the second suspiciousness level PHY. If a corresponding person is found in step S203, the suspicious person determination unit 13 determines that person as a suspicious person in step S204 and creates suspicious person information. This step is the same as step S104 in FIG. 2. Then, the processing of the suspicious person detection device 103 returns to step S201.

[0067] If there is no matching person in step S203, the process of the suspicious person detection device 103 returns to step S201.

[0068] In step S203, the predetermined combined condition is, for example, any one of the following logics 1, 2, 3, and 4.

[0069] Logic 1: PSY + PHY ≥ 1.5 Logic 2: PSY × PHY ≥ 0.5 Logic 3: PSY = 1.0, PHY ≥ 0.5 Logic 4: PSY ≥ 0.5, PHY = 1.0 A logic may be used in which a person is determined to be suspicious even if one of the first suspiciousness PSY and the second suspiciousness PHY is low and the other is high. Alternatively, a logic may be used in which a person is determined to be suspicious when both are high.

[0070] <C-3. Effect> In the suspicious person detection device 103 according to the third embodiment, the first suspiciousness detection unit 11 detects a first suspiciousness indicating the degree of the first suspicious state of a person captured in the captured video, determines that a person whose first suspiciousness is equal to or greater than a predetermined first threshold value is in the first suspicious state, and includes the information of the first suspiciousness in the first suspicious information. Further, the second suspiciousness detection unit 12 detects a second suspiciousness indicating the degree of the second suspicious state of a person captured in the captured video, determines that a person whose second suspiciousness is equal to or greater than a predetermined second threshold value is in the second suspicious state, and includes the information of the second suspiciousness in the second suspicious information. Further, the suspicious person determination unit 13 determines a person who is in the first suspicious state and the second suspicious state and satisfies a predetermined combined condition for the first suspiciousness and the second suspiciousness as a suspicious person. Thus, according to the suspicious person detection device 103, since the determination of the suspicious person is made in consideration of the combined condition of the first suspiciousness and the second suspiciousness, the suspicious person can be detected with high accuracy.

[0071] <C-4. Modification Example> The composite condition of the first suspiciousness degree PSY and the second suspiciousness degree PHY may be variable according to the first suspicious state attribute, the second suspicious state attribute, or the hazard attribute. For example, if there are persons in the first suspicious state and the second suspicious state, and the hazard attribute of the person is injury or robbery, the suspicious person determination unit 13 may use the following composite condition to make it easier to detect as a suspicious person.

[0072] Logic 1': PSY + PHY ≥ 1.2 Logic 2': PSY × PHY ≥ 0.4 Machine learning may be used to calculate the first suspiciousness degree PSY and the second suspiciousness degree PHY. The first suspiciousness detection unit 11 may calculate the first suspiciousness degree using the learning result of a predetermined first machine learning with the captured video of the surveillance camera 201 as input. Also, the second suspiciousness detection unit 12 may calculate the second suspiciousness degree using the learning result of a predetermined second machine learning with the captured video of the surveillance camera 201 as input. By utilizing machine learning, a suspicious person can be detected with high accuracy.

[0073] <D. Embodiment 4> <D-1. Configuration> FIG. 11 is a configuration diagram of a suspicious person detection system 1004 according to Embodiment 4. The suspicious person detection system 1004 includes a plurality of suspicious person detection devices 104a, 104b and a plurality of surveillance cameras 201a, 201b provided corresponding to the plurality of suspicious person detection devices 104a, 104b. In FIG. 11, two suspicious person detection devices 104a, 104b and two surveillance cameras 201a, 201b are shown, but as long as a plurality of suspicious person detection devices and surveillance cameras are provided respectively, the number thereof is not limited to two.

[0074] The suspicious person detection device 104a includes a notification unit 14a and a reference information input unit 15a in addition to the configuration of the suspicious person detection device 103 according to embodiment 3. The suspicious person detection device 104b has the same configuration as the suspicious person detection device 104a. However, to distinguish between the two, the components of the suspicious person detection device 104a are designated with an appropriate suffix "a" to the reference numerals, and the components of the suspicious person detection device 104b are designated with an appropriate suffix "b" to the reference numerals.

[0075] In the suspicious person detection system 1004, the suspicious person detection devices 104a and 104b cooperate with each other to detect suspicious persons. The following describes how the suspicious person detection device 104a detects a suspicious person using information acquired from the suspicious person detection device 104b.

[0076] In the suspicious person detection device 104b, the first suspiciousness detection unit 11b calculates the first suspiciousness level PSYb of a person appearing in the image captured by the surveillance camera 201b, and determines that a person whose first suspiciousness level PSYb is higher than a first threshold value is in a first suspicious state, and creates first suspiciousness information.

[0077] The second suspiciousness detection unit 12b calculates the second suspiciousness level PHYb of a person appearing in the video of the surveillance camera 201b, and determines that a person whose second suspiciousness level PHYb is higher than a second threshold is in a second suspicious state, and creates second suspiciousness information.

[0078] The suspicious person determination unit 13b determines, based on the first suspicious information obtained from the first suspicious information detection unit 11b and the second suspicious information obtained from the second suspicious information detection unit 12b, that a person who falls into both the first suspicious state and the second suspicious state and whose first suspicious degree and second suspicious degree satisfy a predetermined composite condition is a suspicious person, and creates suspicious person information.

[0079] Furthermore, the suspicious person determination unit 13b may determine as a suspicious person candidate a person whose first suspiciousness level and second suspiciousness level do not satisfy a predetermined composite condition but do satisfy certain conditions, and create suspicious person candidate information including the first suspiciousness information and second suspiciousness information of the suspicious person candidate. For example, the suspicious person determination unit 13b may determine as a suspicious person candidate a person whose first suspiciousness level PSYb≧0.3 or more and whose second suspiciousness level PHYb≧0.3 or more.

[0080] The notification unit 14b notifies the suspicious person detection device 104a of the suspicious person information or suspicious person candidate information created by the suspicious person determination unit 13b. The reference information input unit 15a of the suspicious person detection device 104a acquires the suspicious person information or suspicious person candidate information notified by the notification unit 14b of the suspicious person detection device 104b as reference information.

[0081] The suspicious person judgment unit 13a creates suspicious person information based on the first suspicious information obtained from the first suspicious information detection unit 11a, the second suspicious information obtained from the second suspicious information detection unit 12a, and the reference information obtained from the reference information input unit 15a.

[0082] The suspicious person determination unit 13a compares the first and second suspicious states detected by the first and second suspicious state detection units 11a and 12a to determine whether they match the suspicious person or suspected suspicious person identified in the reference information. If they match, this means that the person detected as a suspicious person or suspected suspicious person by the suspicious person detection device 104b has also been identified as a suspicious person by the suspicious person detection device 104a. Therefore, the suspicious person determination unit 13a relaxes the combined conditions for the first and second suspiciousness levels required to determine a person as suspicious. For example, the suspicious person determination unit 13a employs the following logic: Note that PSYa represents the first suspiciousness level detected by the first suspiciousness detection unit 11a, and PHYa represents the second suspiciousness level detected by the second suspiciousness detection unit 12a.

[0083] Logic 1a: PSYa + PHYa + (PSYb + PHYb) / 4 ≥ 1.5 Logic 2a: PSYa x PHYa ≥ 0.4 AND PSYb x PHYb ≥ 0.2 In the above description, it has been explained that the suspicious person detection device 104a uses the suspicious person information detected by the suspicious person detection device 104b as reference information to detect suspicious persons. Similarly, in the suspicious person detection device 104b, the suspicious person information detected by the suspicious person detection device 104a can also be used as reference information to detect suspicious persons.

[0084] <D-2. Effect> The suspicious person detection device 104a according to Embodiment 4 acquires the suspicious person information created by another suspicious person detection device 104b as reference information from the other suspicious person detection device 104b, and the suspicious person determination unit 13a identifies a suspicious person based on the first suspicious information, the second suspicious information, and the reference information. In this way, by sharing suspicious person information between the suspicious person detection devices 104a and 104b, it becomes possible to accurately detect suspicious persons.

[0085] <E. Hardware Configuration> The first suspicious detection unit 11, the second suspicious detection unit 12, the suspicious person determination unit 13, the notification unit 14, and the reference information input unit 15 in the above-mentioned suspicious person detection devices 101, 102, 103, 104a, and 104b are realized by the processing circuit 81 shown in FIG. 12. That is, the processing circuit 81 includes the first suspicious detection unit 11, the second suspicious detection unit 12, the suspicious person determination unit 13, the notification unit 14, and the reference information input unit 15 (hereinafter referred to as "the first suspicious detection unit 11, etc."). Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in a memory may be applied. The processor is, for example, a central processing unit, a processing device, an arithmetic device, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

[0086] When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. The functions of each unit such as the first suspicious activity detection unit 11 may be realized by multiple processing circuits 81, or the functions of each unit may be realized together by a single processing circuit.

[0087] When the processing circuit 81 is a processor, the functions of the first suspicious detection unit 11 and the like are realized by a combination of software, etc. (software, firmware, or software and firmware). The software, etc. is written as a program and stored in memory. As shown in FIG. 13, a processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in a memory 83. That is, suspicious person detection devices 101, 102, 103, 104a, and 104b each include memory 83 for storing a program that, when executed by processing circuit 81, results in the execution of the following steps: detecting a person in a first suspicious state, which is a predetermined psychological state indicating a high likelihood of harming others, from video footage captured by surveillance camera 201; creating first suspicious information including identifying information of the person in the first suspicious state; detecting a person in a second suspicious state who is engaging in predetermined behavior as a sign of harming others, from the video footage; creating second suspicious information including identifying information of the person in the second suspicious state; and identifying the suspicious person based on the first suspicious information and the second suspicious information and creating suspicious person information including identifying information of the suspicious person. In other words, this program can be said to cause a computer to execute the procedure or method of first suspicious person detection unit 11, etc. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, or any storage medium that will be used in the future.

[0088] The above describes a configuration in which each function of the first suspiciousness detection unit 11, etc. is realized either by hardware or software, etc. However, this is not limited to this, and a configuration in which part of the first suspiciousness detection unit 11, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the first suspiciousness detection unit 11 can be realized by a processing circuit as dedicated hardware, and the other functions can be realized by the processing circuit 81 as the processor 82 reading and executing a program stored in the memory 83.

[0089] As described above, the processing circuit can realize each of the above functions by hardware, software, or a combination of these.

[0090] When the suspicious person detection devices 101, 102, 103, 104a, and 104b detect suspicious persons on a moving object such as a train, the suspicious person detection devices 101, 102, 103, 104a, and 104b do not necessarily need to be mounted on the moving object. For example, the suspicious person detection devices 101, 102, 103, 104a, and 104b can be applied to a system constructed by appropriately combining a PND (Portable Navigation Device), a communication terminal (e.g., a mobile terminal such as a mobile phone, a smartphone, or a tablet), the functions of applications installed thereon, and a server. In this case, the functions or components of the suspicious person detection devices 101, 102, 103, 104a, and 104b described above may be distributed among the devices constituting the system or may be concentrated in one of the devices.

[0091] It should be noted that the embodiments can be freely combined, and each embodiment can be modified or omitted as appropriate. The above description is an example in all respects. It is understood that countless variations not illustrated can be envisioned. [Explanation of symbols]

[0092] 11 First suspicious person detection unit, 12 Second suspicious person detection unit, 13 Suspicious person judgment unit, 14 Notification unit, 15 Reference information input unit, 81 Processing circuit, 82 Processor, 83 Memory, 101, 102, 103, 104a, 104b Suspicious person detection device, 201 Surveillance camera, 202 Train vision, 203 Control room, 1001, 1002, 1003, 1004 Suspicious person notification system.

Claims

1. A first suspiciousness detection unit that detects a person in a first suspicious state, which is a predetermined psychological state that is likely to cause harm to others, from footage captured by a surveillance camera installed on a train, and creates first suspicious information that includes specific information about the person in the first suspicious state and first suspicious state attribute information that represents the attributes of the first suspicious state; a second suspiciousness detection unit that detects from the captured video a person in a second suspicious state who is engaging in a predetermined behavior as a sign of harming another person, and creates second suspiciousness information that includes specific information of the person in the second suspicious state and second suspicious state attribute information that represents an attribute of the second suspicious state; a suspicious person determination unit that identifies a person in the first suspicious state and the second suspicious state as a suspicious person and creates suspicious person information including identification information of the suspicious person, the suspicious person determination unit determines a harm attribute, which is an attribute of harm that the suspicious person is likely to inflict on others, based on a combination of the first suspicious state attribute information and the second suspicious state attribute information, and includes harm attribute information representing the harm attribute in the suspicious person information; the suspicious person determination unit determines whether the suspicious person corresponds to the harm attribute of "injury" and "molestation or theft"; Suspicious person detection device.

2. The first suspiciousness detection unit detects a first suspiciousness degree that indicates the degree of the first suspicious state of a person appearing in the photographed video, determines that a person whose first suspiciousness degree is equal to or greater than a predetermined first threshold is in the first suspicious state, and includes information about the first suspiciousness degree in the first suspiciousness information, the second suspiciousness detection unit detects a second suspiciousness degree that indicates the degree of the second suspiciousness state of a person appearing in the captured video, determines that a person whose second suspiciousness degree is equal to or greater than a predetermined second threshold is in the second suspiciousness state, and includes information about the second suspiciousness degree in the second suspiciousness information; the suspicious person determination unit determines, as the suspicious person, a person who is in the first suspicious state and the second suspicious state and who satisfies a predetermined composite condition related to an added value or a multiplied value of the first suspiciousness degree and the second suspiciousness degree. The suspicious person detection device according to claim 1 .

3. a notification unit that notifies the suspicious person information to a display device that is provided outside the suspicious person detection device and in a vehicle of the train; The suspicious person information includes information about a vehicle in which the suspicious person is present. The suspicious person detection device according to claim 1 .

4. the first suspiciousness detection unit detects a first suspiciousness degree that indicates the degree of the first suspicious state of a person appearing in the captured video, determines that a person whose first suspiciousness degree is equal to or greater than a predetermined first threshold is in the first suspicious state, and includes information about the first suspiciousness degree in the first suspiciousness information; the second suspiciousness detection unit detects a second suspiciousness degree that indicates the degree of the second suspiciousness state of a person appearing in the captured video, determines that a person whose second suspiciousness degree is equal to or greater than a predetermined second threshold is in the second suspiciousness state, and includes information about the second suspiciousness degree in the second suspiciousness information; the suspicious person determination unit determines, as the suspicious person, a person who is in the first suspicious state and the second suspicious state and who satisfies a predetermined composite condition for the first suspiciousness level and the second suspiciousness level. The suspicious person detection device according to claim 1 .

5. the first suspiciousness detection unit calculates the first suspiciousness degree of a person appearing in the photographed video using a learning result of a first machine learning process that uses the photographed video as an input; and the second suspiciousness detection unit calculates the second suspiciousness degree of a person appearing in the photographed video using a learning result of a second machine learning process that uses the photographed video as an input; The suspicious person detection device according to claim 4.

6. The surveillance cameras are installed in multiple cars within the train. The suspicious person detection device according to claim 1 .

7. The attributes of the first suspicious state include at least one of emotional instability, excitement, and malice, The suspicious person detection device according to claim 1 .

8. The attributes of the second suspicious state include at least one of approach behavior, contact behavior, possession of a dangerous object, taking an object from clothing, leaving a suspicious object behind, searching behavior, and surveillance camera-aware behavior. The suspicious person detection device according to claim 7.

9. the suspicious person determination unit determines whether the suspicious person corresponds to the harm attribute of suspiciousness, theft, robbery, or abandonment of dangerous objects; The suspicious person detection device according to claim 8.

10. The first suspiciousness detection unit detects the line of sight of a person appearing in the captured video, and determines that a person whose line of sight moves a predetermined number of times or more within a predetermined period of time is in the first suspicious state, and determines that an attribute of the first suspicious state is the emotional instability. The suspicious person detection device according to claim 7.

11. The second suspiciousness detection unit detects the line of sight of a person appearing in the captured video, and determines that a person who directs their line of sight toward the surveillance camera a predetermined number of times or more within a predetermined time is in the second suspicious state, and determines that an attribute of the second suspicious state is surveillance camera-aware behavior. The suspicious person detection device according to claim 8.

12. the suspicious person determination unit determines a person who has the first suspicious state of a malicious attribute and the second suspicious state of a dangerous substance possession attribute or an object removal action attribute from clothing as the suspicious person, and determines the harm attribute of the suspicious person as injury; The suspicious person detection device according to claim 9.

13. the suspicious person determination unit determines a person who has the first suspicious state of an emotional instability attribute and the second suspicious state of a contact behavior attribute as the suspicious person, and determines the harm attribute of the suspicious person as molestation or theft; The suspicious person detection device according to claim 9.

14. the suspicious person determination unit determines a person who has the first suspicious state of an emotionally unstable attribute and the second suspicious state of a suspicious object abandonment attribute as the suspicious person, and determines the harm attribute of the suspicious person as abandonment of a dangerous object; The suspicious person detection device according to claim 9.

15. A notification unit that notifies the suspicious person information to an outside of the suspicious person detection device, The notification unit notifies the suspicious person information to other suspicious person detection devices in the train. The suspicious person detection device according to claim 6.

16. The suspicious person information created by the other suspicious person detection device is acquired from the other suspicious person detection device as reference information; the suspicious person determination unit identifies the suspicious person based on the first suspicious information, the second suspicious information, and the reference information, and when a person described in the reference information matches a person who is indicated in the first suspicious information as being in the first suspicious state and who is indicated in the second suspicious information as being in the second suspicious state, the suspicious person determination unit relaxes the conditions for identifying the person described in the reference information as the suspicious person and identifies the suspicious person; The suspicious person detection device according to claim 15.

17. a first suspiciousness detection unit detects a person in a first suspicious state, which is a predetermined psychological state that is likely to cause harm to others, from video footage captured by a surveillance camera installed on the train, and creates first suspiciousness information including specific information of the person in the first suspicious state and first suspicious state attribute information that represents an attribute of the first suspicious state; a second suspiciousness detection unit detects from the captured video a person in a second suspicious state who is engaging in a predetermined behavior as a sign of harming another person, and creates second suspiciousness information including specific information of the person in the second suspicious state and second suspicious state attribute information that represents an attribute of the second suspicious state; a suspicious person determination unit that identifies a person in the first suspicious state and the second suspicious state as a suspicious person and creates suspicious person information including identification information of the suspicious person; determining a harm attribute, which is an attribute of harm that the suspicious individual is likely to inflict on others, based on a combination of the first suspicious state attribute information and the second suspicious state attribute information, and including harm attribute information representing the harm attribute in the suspicious individual information; Determine whether the suspicious person corresponds to the harm attribute of "injury" and "molestation or theft." Method for detecting suspicious persons.

Images