Abnormal behavior notification system

Abstract

To provide an abnormal behavior notification system capable of making an appropriate notification according to the attribute or the like of a behavioral entity when abnormal behavior or the like of the behavioral entity captured on video is detected.SOLUTION: An abnormal behavior notification system 1 includes a determination unit 5 that determines whether abnormal behavior has occurred based on the displacement of stored feature points and the displacement of feature points detected from each time-series image Y, an estimation unit 6 that estimates the attribute or size of a target behavioral entity Z based on the detected feature points or the appearance of the target behavioral entity, and a notification unit 7 that makes a notification when it is determined that abnormal behavior has occurred. Even when it is determined that abnormal behavior has occurred, the notification unit 7 does not make a notification if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an abnormal behavior notification system that can provide appropriate notification according to the attributes of a behavioral entity when abnormal behavior, such as abnormal behavior of a behavioral entity captured on video, is detected. [Background technology]

[0002] Conventionally, there has been known a technique for extracting a behavioral entity that has behaved differently from a predetermined "normal behavior" from a time-series image when the behavior of the behavioral entity captured in the time-series image differs from the "normal behavior" (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] By using the above technology, when behavior that differs from "normal behavior" is observed, it can be determined that "abnormal behavior has occurred," and a notification can be sent to the manager or security guard of the facility where the abnormal behavior occurred.

[0005] However, for example, children often behave in a way that is judged as abnormal, and some of these behaviors would be considered abnormal if the behavior were an adult, but not if the behavior were a child. In such a situation, if notifications were sent uniformly to both adults and children, the frequent notifications would become annoying and could also undermine the reliability of the notifications.

[0006] Therefore, the present invention aims to provide an abnormal behavior notification system that can provide appropriate notifications based on the attributes of an entity when abnormal behavior of the entity captured on video is detected. [Means for solving the problem]

[0007] The present invention provides an abnormal behavior notification system comprising an acquisition unit that acquires time-series images captured by a photographing means, a detection unit that detects feature points of a target behavior entity captured in the time-series images, a memory unit that stores the displacement of the feature points of the behavior entity when the behavior entity performs abnormal behavior, a judgment unit that determines whether the abnormal behavior has been performed based on the displacement of the stored feature points and the displacement of feature points detected from each time-series image, an estimation unit that estimates the attributes or size of the target behavior entity based on the detected feature points or the appearance of the target behavior entity, and a notification unit that issues a notification when it is determined that the abnormal behavior has been performed, wherein the notification unit does not issue the notification even if it is determined that the abnormal behavior has been performed if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0008] With this configuration, when abnormal behavior such as an abnormal behavior of a target behavioral entity is detected in time-series images, it becomes possible to issue an appropriate notification according to the attributes of the target behavioral entity, etc. For example, by setting a first predetermined condition and a second predetermined condition so that a notification is not issued for a predetermined abnormal behavior of a child, frequent notifications are suppressed, and a loss of reliability in notifications is also suppressed.

[0009] In another aspect, the present invention provides an abnormal behavior notification program and an abnormal behavior notification method corresponding to the abnormal behavior notification system. [Effects of the Invention]

[0010] According to the abnormal behavior notification system of the present invention, when abnormal behavior such as abnormal behavior of a behavioral entity captured on video is detected, it is possible to provide an appropriate notification depending on the attributes of the behavioral entity, etc. [Brief explanation of the drawings]

[0011] [Figure 1]1 is an explanatory diagram of a time-series image according to a first embodiment of the present invention; [Figure 2] 1 is a block diagram of an abnormal behavior notification system according to a first embodiment of the present invention; [Figure 3] 1 is a flowchart of an abnormal behavior notification system according to a first embodiment of the present invention. [Figure 4] 1 is a block diagram of an abnormal behavior notification system according to a second embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0012] An abnormal behavior notification system 1 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 to 3. FIG.

[0013] As shown in Fig. 1, the abnormal behavior notification system 1 is for providing notification according to the type of behavior of a target behavior entity Z captured in time-series images Y (frames constituting the video in Fig. 1) captured by a photographing means X. In this embodiment, a human being is used as the target behavior entity Z, and for ease of understanding, the target behavior entity Z is simply displayed using only a skeleton.

[0014] 2, the abnormal behavior notification system 1 includes an acquisition unit 2, a detection unit 3, a storage unit 4, a determination unit 5, an estimation unit 6, and a notification unit 7. In this embodiment, the abnormal behavior notification system 1 is provided integrally with a photographing means X.

[0015] The acquisition unit 2 acquires time-series images Y captured by the imaging means X.

[0016] The detection unit 3 detects feature points of a target moving object Z captured in a time-series image Y.

[0017] There are various possible feature points, but in this embodiment, an example in which joints are detected as feature points will be described.

[0018] When detecting joints as feature points, the following method can be considered, for example.

[0019] First, a "joint identification standard" and a "moving object identification standard" are stored in a storage unit (which may be the storage unit 4 or another storage unit).

[0020] The "joint identification standard" is used to identify multiple joints in a human body, and indicates the shape, direction, size, etc. for identifying each joint.

[0021] The "Behavior Identification Standards" indicate the "basic postures" of various human variations ("walking," "standing upright," etc.), the "range of motion of each joint," and the "distance between each joint" in a single human.

[0022] After detecting multiple joints that meet the above-mentioned "joint identification criteria," it is possible to identify the joints contained in each target action object Z by referring to the "action object identification criteria."

[0023] Note that the feature points may be detected individually for each time-series image Y, or may be detected collectively from a plurality of time-series images Y if the time-series order can be identified.

[0024] The storage unit 4 stores displacements of feature points of the active object when the active object performs abnormal behavior. When joints are detected as feature points, displacements of multiple joints are stored.

[0025] Possible abnormal behaviors include, for example, falling, hitting, kicking, etc., and multiple behaviors may be stored instead of just one. Furthermore, "premonition behaviors" such as preparatory movements for shoplifting may also be stored as abnormal behaviors. Note that the memory unit 4 may also store displacements of feature point information when behaviors other than abnormal behavior (walking, stopping, etc.) occur in order to estimate such behaviors.

[0026] The determination unit 5 determines whether or not abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image Y.

[0027] For example, if the displacement of the detected feature point matches the displacement of the feature point of abnormal behavior by a predetermined amount or more, it may be determined that "the abnormal behavior has been performed."

[0028] The estimation unit 6 estimates the attribute or size of the target action object Z based on the detected feature points or the appearance of the target action object Z.

[0029] The attribute or size of the target action object Z may be estimated by a known method.

[0030] For example, when identifying the target action object Z by its joints as described above, it is possible to estimate the size of the target action object Z by determining the distance between the joint points of the head and the joint points of the ankles.

[0031] In addition to the size described above, attributes can be estimated from features such as posture and bone structure, and external appearance such as clothing, hair length, and hair color.

[0032] The notification unit 7 issues a notification when it is determined that abnormal behavior has occurred.

[0033] For example, it is conceivable that notification to that effect may be sent wirelessly to an information terminal or the like of a manager or security guard of the facility where the image capturing means X is installed.

[0034] Incidentally, for example, even if it is not appropriate for an adult to "run" in a place, a child may run without malicious intent. However, it is not appropriate to issue a uniform notification for the behavior of "running" even if there is little malicious intent or danger.

[0035] Therefore, in this embodiment, even if it is determined that abnormal behavior has occurred, the notification unit 7 will not issue a notification if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0036] For example, by setting the first predetermined condition to "child" and the second predetermined condition to "running (low speed)," it is conceivable that notification will not be made when "a child runs at a low speed." Note that with the above settings, notification will be made when a child runs at a high speed, but if the second predetermined condition is set to "running," notification will not be made even when the child runs at a high speed.

[0037] Also, because children often fall, it is conceivable that by setting the first predetermined condition to "child" and the second predetermined condition to "fall," a notification will not be sent in the case of a "child fall." On the other hand, because elderly people are more likely to be injured in a fall, if "elderly" is not set as the first predetermined condition, a notification will be sent in the case of an elderly person falling.

[0038] Also, a bored man may perform a golf swing slowly, but in this case, there is little malicious intent or danger. Therefore, by setting the first predetermined condition to "male" and the second predetermined condition to "golf swing (slow speed)," it is possible to avoid issuing a notification when "a male is performing a golf swing slowly." However, if the golf swing is performed at high speed, there is a possibility that it is dangerous or malicious, so it is possible to set it to issue a notification.

[0039] Furthermore, since abnormal behavior by children is often not malicious, by setting the first specified condition to "children" and the second specified condition to "all abnormal behaviors," notifications can be prevented from being sent for "abnormal behaviors by children."

[0040] On the other hand, in the case of abnormal behavior by young men, it is relatively common for malicious intent to be present, so it may be possible to issue a notification regardless of the degree of danger. In this case, for example, it is sufficient not to set "young men" as the first predetermined condition.

[0041] However, since it is conceivable that even young men may jog without malicious intent, it is possible to set the first specified condition to "young man" and the second specified condition to "jog," so that no notification will be sent if the person is a "young man jogging."

[0042] As mentioned above, various combinations of the first predetermined condition regarding "attributes or size" and the second predetermined condition regarding "abnormal behavior" are possible, and various combinations may be set.

[0043] Furthermore, the abnormal behavior notification system 1 may be provided with a setting unit so that the first condition and the second condition can be set by the user.

[0044] Furthermore, similar control may be performed simply when the size of the target action object Z satisfies a first predetermined condition (for example, when "the size (height) of the target action object Z is smaller than a predetermined value").

[0045] In this way, in the abnormal behavior notification system 1 according to this embodiment, when abnormal behavior such as abnormal behavior of a target behavior entity Z is detected in a time series image Y, it is possible to provide appropriate notification according to the attributes of the target behavior entity Z, etc.

[0046] Next, the notification flow according to this embodiment will be described with reference to the flowchart of FIG.

[0047] First, when the acquisition unit 2 acquires a time-series image Y (S1), the detection unit 3 detects feature points of the target action object Z captured in the time-series image Y (S2).

[0048] Furthermore, based on the feature points detected in S2 or the appearance of the target action object Z, the attributes or size of the target action object Z are estimated (S3).

[0049] Next, it is determined whether or not abnormal behavior has occurred based on the displacements of the feature points stored in the storage unit 4 and the displacements of the feature points detected in S2 (S4). S3 and S4 may be performed in the reverse order.

[0050] If it is determined that abnormal behavior has occurred (S4: YES), it is determined whether the attribute or size estimated in S3 satisfies a first predetermined condition and whether the abnormal behavior determined in S4 satisfies a second predetermined condition set for the attribute or size (S5).

[0051] If the first predetermined condition and the second predetermined condition are not met (S5: NO), a notification is given that abnormal behavior has occurred (S6).

[0052] On the other hand, if the first predetermined condition and the second predetermined condition are met (S5: YES), no notification is made (S7).

[0053] As described above, in the abnormal behavior notification system 1 according to this embodiment, even if it is determined that abnormal behavior has occurred, notification is not made if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0054] With this configuration, when abnormal behavior such as abnormal behavior of the target behavior entity Z is detected in the time-series image Y, it becomes possible to issue an appropriate notification according to the attributes of the target behavior entity Z. For example, by setting a first predetermined condition and a second predetermined condition so that notifications are not issued for certain abnormal behaviors of children, frequent notifications are suppressed, and the loss of reliability of notifications is also suppressed.

[0055] Next, an abnormal behavior notification system 100 according to a second embodiment of the present invention will be described with reference to Fig. 4. Note that the same members as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted.

[0056] In this embodiment, the abnormal behaviors stored in the storage unit 4 are determined by learning.

[0057] In detail, the abnormal behavior notification system 100 includes a learning-side acquiring unit 11, a learning-side detecting unit 12, and a determining unit 13 in addition to the configuration of the first embodiment.

[0058] The learning-side acquiring unit 11 acquires sample videos (plurality of sample time-series images) captured by an imaging means X installed so as to capture images of a predetermined range.

[0059] The learning-side detection unit 12 detects the behavior of the sample behavior object captured in the sample video.

[0060] The behavior of the sample behavior body can be detected by storing the displacement of feature points (such as the movement of each joint) when the sample behavior body performs a specified behavior in a memory unit (which can be memory unit 4 or another memory unit), detecting the feature points in the same manner as detection unit 3, and then detecting that "the specified behavior has been performed" if the displacement of the detected feature point information matches the displacement of the stored feature points by a specified amount or more.

[0061] The determination unit 13 determines one or more "normal behaviors" within a predetermined range based on the multiple behaviors detected by the learning-side detection unit 12. In this embodiment, the determined "normal behaviors" are stored in the storage unit 4 as abnormal behaviors.

[0062] "Normal behavior" can be determined based on various criteria, but for example, behavior that has a predetermined (threshold) or higher proportion among all detected behaviors can be determined to be "normal behavior."

[0063] Then, the determination unit 5 determines that abnormal behavior has been performed when the displacement of the feature points detected from each time-series image Y does not correspond to "normal behavior."

[0064] In detail, if the displacement of the feature points detected from each time-series image Y does not correspond to "normal behavior," i.e., if it is determined that abnormal behavior has occurred (S4: YES in Figure 3), the operations of S5-S7 in Figure 3 will be performed, as in the first embodiment.

[0065] As described above, in the abnormal behavior notification system 100 according to this embodiment, if the displacement of the feature points detected from each time-series image Y does not correspond to the "normal behavior" determined by learning, it is determined that abnormal behavior has occurred.

[0066] With this configuration, even if a behavior is not clearly (predetermined) abnormal behavior such as violent behavior or falling, it is possible to regard behavior that is inappropriate for the situation (predetermined range) as abnormal behavior and issue a notification. However, with this configuration, for example, the behavior of a child is more likely to be determined as abnormal behavior, but even in such a case, if the attribute or size meets the first predetermined condition, notification is not issued, thereby preventing frequent notifications from being issued due to abnormal behavior such as that of a "child."

[0067] The abnormal behavior notification system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0068] For example, in the above embodiment, all components are provided integrally, but this does not exclude the possibility that some components (memory unit 4, judgment unit 5, estimation unit 6, etc.) are provided in different locations (cloud, etc.).

[0069] The present invention can also be applied to a program and method corresponding to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transitory recording medium. A CD-ROM or the like is conceivable as a non-transitory recording medium, but is not limited to this. [Explanation of symbols]

[0070] 1. Abnormal Behavior Notification System 2 Acquisition part 3. Detection unit 4 Storage section 5 Judgment section 6 Estimation part 7 Notification section 11 Learning side acquisition unit 12 Learning side detection unit 13 Decision Section 100 Abnormal Behavior Notification System X Shooting Method Y time series images Z Target Behavior

Claims

1. an acquisition unit that acquires time-series images captured by the imaging means; a detection unit that detects feature points of a target action object captured in the time-series images; a storage unit that stores displacements of feature points of a behavioral object when the behavioral object performs abnormal behavior; a determination unit that determines whether the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; an estimation unit that estimates an attribute or a size of the target behavior object based on the detected feature points or the appearance of the target behavior object; a notification unit that issues a notification when it is determined that the abnormal behavior has occurred; Equipped with An abnormal behavior notification system characterized in that the notification unit does not issue the notification even if it is determined that the abnormal behavior has occurred, if the attribute or size satisfies a first specified condition and the abnormal behavior satisfies a second specified condition set for the attribute or size.

2. a learning-side acquisition unit that acquires a sample video captured by the imaging means installed to capture a predetermined range; a learning-side detection unit that detects the behavior of a sample behavior object captured in the sample video; a determination unit that determines one or more normal actions within the predetermined range based on the multiple actions detected by the learning-side detection unit; Further provided with The abnormal behavior notification system described in claim 1, characterized in that the judgment unit determines that the abnormal behavior has been committed when the displacement of feature points detected from each time-series image does not correspond to the normal behavior.

3. A program executed by a computer that stores displacements of feature points of a behavioral object when the behavioral object performs abnormal behavior, acquiring time-series images captured by an imaging means; detecting feature points of a target moving object captured in the time-series images; determining whether the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; estimating an attribute or a size of the target activity based on the detected feature points or the appearance of the target activity; a step of notifying when it is determined that the abnormal behavior has occurred; Equipped with An abnormal behavior notification program characterized in that in the notification step, even if it is determined that the abnormal behavior has occurred, the notification is not made if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

4. A method executed by a computer that stores displacements of feature points of a behavioral entity when the behavioral entity performs abnormal behavior, comprising: acquiring time-series images captured by an imaging means; detecting feature points of a target moving object captured in the time-series images; determining whether the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; estimating an attribute or a size of the target activity based on the detected feature points or the appearance of the target activity; a step of notifying when it is determined that the abnormal behavior has occurred; Equipped with An abnormal behavior notification method characterized in that in the notification step, even if it is determined that the abnormal behavior has occurred, the notification is not made if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

Images