Abnormal behavior notification system

Abstract

To provide an abnormal behavior notification system capable of making an appropriate notification according to the attributes of a behavioral entity when abnormal behavior such as an abnormal behavior of a behavioral entity captured on a video is detected. [Solution] The abnormal behavior notification system 1 includes a determination unit 5 that determines whether abnormal behavior has occurred based on the displacement of stored feature points and the displacement of feature points detected from each time-series image Y, an estimation unit 6 that estimates the attribute or size of a target behavior entity Z based on the detected feature points or the appearance of the target behavior entity, and a notification unit 7 that issues a notification when it is determined that abnormal behavior has occurred. Even when it is determined that abnormal behavior has occurred, the notification unit 7 does not issue a notification if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

Description

[Technical field]

[0001] The present invention relates to an abnormal behavior notification system that can provide an appropriate notification according to the attributes of an entity when abnormal behavior, such as abnormal behavior of an entity captured on video, is detected. [Background technology]

[0002] There has been known a technique for extracting a behavioral entity that has behaved differently from a predetermined "normal behavior" from a time-series image when the behavior of the behavioral entity captured in the time-series image differs from the "normal behavior" (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] By using the above technology, it is possible to determine that "abnormal behavior" has occurred when behavior that differs from "normal behavior" has occurred, and to notify the manager or security guard of the facility where the abnormal behavior occurred.

[0005] However, for example, children often behave in a way that is judged to be abnormal, and among these behaviors, there are some that would be abnormal if the behavior was an adult, but would not be abnormal if the behavior was a child. In such a situation, if notifications were sent uniformly to both adults and children, the frequent notifications would be irritating, and the reliability of the notifications would be lost.

[0006] Therefore, the present invention aims to provide an abnormal behavior notification system that can provide appropriate notification based on the attributes, etc. of an entity when abnormal behavior of an entity captured on video is detected. [Means for solving the problem]

[0007] The present invention provides an abnormal behavior notification system comprising an acquisition unit that acquires time-series images captured by an imaging means, a detection unit that detects feature points of a target behavior object captured in the time-series images, a memory unit that stores the displacement of the feature points of the target behavior object when the behavior object performs abnormal behavior, a judgment unit that determines whether the abnormal behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image, an estimation unit that estimates the attributes or size of the target behavior object based on the detected feature points or the appearance of the target behavior object, and a notification unit that issues a notification when it is determined that the abnormal behavior has been performed, wherein the notification unit does not issue the notification even if it is determined that the abnormal behavior has been performed if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0008] With this configuration, when abnormal behavior such as an abnormal behavior of a target behavior object is detected in a time-series image, it becomes possible to give an appropriate notification according to the attributes of the target behavior object, etc. For example, by setting a first predetermined condition and a second predetermined condition so that a notification is not given for a predetermined abnormal behavior of a child, frequent notifications are suppressed, and the loss of reliability of the notifications is also suppressed.

[0009] In another aspect of the present invention, there are provided an abnormal behavior notifying program and an abnormal behavior notifying method corresponding to the abnormal behavior notifying system. Effect of the Invention

[0010] According to the abnormal behavior notification system of the present invention, when abnormal behavior such as abnormal behavior of an entity captured on video is detected, it is possible to provide an appropriate notification according to the attributes of the entity. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is an explanatory diagram of a time-series image according to a first embodiment of the present invention; [Diagram 2] 1 is a block diagram of an abnormal behavior notification system according to a first embodiment of the present invention; [Diagram 3] 1 is a flow chart of an abnormal behavior notification system according to a first embodiment of the present invention. [Figure 4] 1 is a block diagram of an abnormal behavior notification system according to a second embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An abnormal behavior notification system 1 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 to 3. FIG.

[0013] As shown in Fig. 1, the abnormal behavior notification system 1 is for providing notification according to the type of behavior of a target behavior entity Z captured in time-series images Y (frames constituting a video in Fig. 1) captured by a capture means X. In this embodiment, a human being is used as the target behavior entity Z, and for ease of understanding, the target behavior entity Z is simply displayed using only a skeleton.

[0014] 2, the abnormal behavior notification system 1 includes an acquisition unit 2, a detection unit 3, a storage unit 4, a determination unit 5, an estimation unit 6, and a notification unit 7. In this embodiment, the abnormal behavior notification system 1 is provided integrally with an image capture means X.

[0015] The acquisition unit 2 acquires time-series images Y captured by the imaging means X.

[0016] The detection unit 3 detects feature points of a target action object Z captured in a time-series image Y.

[0017] There are various possible feature points, but in this embodiment, an example will be described in which joints are detected as feature points.

[0018] When detecting joints as feature points, for example, the following method can be considered.

[0019] First, a "joint identification standard" and a "behavior object identification standard" are stored in a storage unit (which may be the storage unit 4 or another storage unit).

[0020] The "joint identification standard" is for identifying multiple human joints, and indicates the shape, direction, size, etc. for identifying each joint.

[0021] The "Behavior Identification Standard" indicates the "basic postures" of various human variations ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" in a single human.

[0022] After detecting multiple joints that meet the above-mentioned "joint identification criteria," it is possible to identify the joints contained in each target action object Z by referring to the "action object identification criteria."

[0023] Note that feature points may be detected individually for each time-series image Y, or may be detected collectively from a plurality of time-series images Y if the time-series order can be specified.

[0024] The memory unit 4 stores the displacements of the feature points of the action object when the action object performs an abnormal action. When joints are detected as feature points, the displacements of a plurality of joints are stored.

[0025] Possible abnormal behaviors include, for example, falling, punching, kicking, etc., and multiple behaviors may be stored without being limited to one. Also, as abnormal behaviors, "premonitory behaviors" such as preparatory movements for shoplifting may be stored. Note that in order to estimate behaviors other than abnormal behaviors (walking, stopping, etc.), the memory unit 4 may also store displacements of feature point information when such behaviors occur.

[0026] The determination unit 5 determines whether or not abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image Y.

[0027] For example, when the displacement of a detected feature point matches the displacement of a feature point of an abnormal behavior for a predetermined amount or more, it may be determined that "the abnormal behavior has been performed."

[0028] The estimation unit 6 estimates the attributes or size of the target action object Z based on the detected feature points or the appearance of the target action object Z.

[0029] The attributes or size of the target action object Z may be estimated by a known method.

[0030] For example, when identifying the target action object Z by joints as described above, it is possible to estimate the size of the target action object Z by determining the distance between the head joint point and the ankle joint point.

[0031] In addition to the size described above, attributes can be estimated from characteristic points such as posture and bone structure, and external appearance such as clothing, hair length, and hair color.

[0032] The notification unit 7 issues a notification when it is determined that an abnormal behavior has occurred.

[0033] For example, it is conceivable that a notice to that effect may be sent wirelessly to an information terminal or the like of a manager or security guard of the facility where the image capturing means X is installed.

[0034] Incidentally, for example, even if a place is not appropriate for an adult to "run," a child may run without malicious intent. However, it is not appropriate to uniformly issue a notification for the behavior of "running" even if the behavior is unlikely to be malicious or dangerous.

[0035] Therefore, in this embodiment, even if it is determined that abnormal behavior has occurred, the notification unit 7 does not issue a notification if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0036] For example, by setting the first predetermined condition to "child" and the second predetermined condition to "running (low speed)", it is possible to consider not issuing a notification when "a child runs at a low speed". Note that with the above settings, a notification will be issued if a child runs at high speed, but if the second predetermined condition is set to "running", no notification will be issued even if the child runs at high speed.

[0037] Also, because children often fall, it is conceivable that by setting the first predetermined condition to "child" and the second predetermined condition to "fall," a notification will not be sent in the case of a "child fall." On the other hand, in the case of elderly people, there is a high possibility that they will be injured in a fall, so if "elderly" is not set as the first predetermined condition, a notification will be sent in the case of an elderly person falling.

[0038] Also, a bored man may perform a golf swing slowly, but in this case, the malicious intent and danger are low. Therefore, by setting the first predetermined condition to "man" and the second predetermined condition to "golf swing (slow speed)", it is possible to notify when "a man is performing a golf swing slowly". However, if the golf swing is performed at high speed, it may be dangerous or malicious, so it is possible to set it to notify.

[0039] Furthermore, since abnormal behavior by children is often not malicious, the first specified condition can be set to "children" and the second specified condition to "all abnormal behavior," so that no notifications are sent regarding "abnormal behavior by children."

[0040] On the other hand, in the case of abnormal behavior of young men, since it is relatively common for such behavior to be malicious, it is possible to send a notification regardless of the degree of danger. In this case, for example, the first predetermined condition should not be set to "young men."

[0041] However, since it is conceivable that even young men may jog without malicious intent, it is possible to set the first specified condition to "young man" and the second specified condition to "jog," so that no notification is sent in the case of "young man jogging."

[0042] As described above, since various combinations of the first predetermined condition related to the "attribute or size" and the second predetermined condition related to the "abnormal behavior" are possible, various combinations may be set.

[0043] Furthermore, the abnormal behavior notification system 1 may be provided with a setting unit so that the first condition and the second condition can be set by the user.

[0044] Furthermore, similar control may be performed simply when the size of the target action object Z satisfies a first predetermined condition (for example, when "the size (height) of the target action object Z is smaller than a predetermined value").

[0045] In this way, in the abnormal behavior notification system 1 of this embodiment, when abnormal behavior such as abnormal behavior of a target behavior entity Z is detected in a time-series image Y, it is possible to provide an appropriate notification depending on the attributes, etc. of the target behavior entity Z.

[0046] Next, a notification flow according to this embodiment will be described with reference to the flowchart of FIG.

[0047] First, when the acquisition unit 2 acquires a time-series image Y (S1), the detection unit 3 detects feature points of a target action object Z captured in the time-series image Y (S2).

[0048] In addition, the attributes or size of the target action object Z are estimated based on the feature points detected in S2 or the appearance of the target action object Z (S3).

[0049] Next, based on the displacement of the feature points stored in the memory unit 4 and the displacement of the feature points detected in S2, it is determined whether or not an abnormal behavior has occurred (S4). S3 and S4 may be performed in the reverse order.

[0050] If it is determined that abnormal behavior has occurred (S4: YES), it is determined whether the attribute or size estimated in S3 satisfies a first predetermined condition and whether the abnormal behavior determined in S4 satisfies a second predetermined condition set for the attribute or size (S5).

[0051] If the first and second predetermined conditions are not met (S5: NO), a notification is given that an abnormal behavior has occurred (S6).

[0052] On the other hand, if the first and second predetermined conditions are met (S5: YES), no notification is made (S7).

[0053] As described above, in the abnormal behavior notification system 1 according to this embodiment, even if it is determined that abnormal behavior has occurred, no notification is made if the attribute or size satisfies a first predetermined condition and the abnormal behavior satisfies a second predetermined condition set for the attribute or size.

[0054] With this configuration, when abnormal behavior such as an abnormal action of a target behavior entity Z is detected in the time-series image Y, it becomes possible to give an appropriate notification according to the attributes of the target behavior entity Z. For example, by setting a first predetermined condition and a second predetermined condition so that a notification is not given for a certain abnormal behavior of a child, frequent notifications are suppressed, and the loss of reliability of the notifications is also suppressed.

[0055] Next, an abnormal behavior notification system 100 according to a second embodiment of the present invention will be described with reference to Fig. 4. Note that the same members as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted.

[0056] In this embodiment, the abnormal behaviors stored in the storage unit 4 are determined by learning.

[0057] In detail, the abnormal behavior notification system 100 includes a learning-side acquiring unit 11, a learning-side detecting unit 12, and a deciding unit 13 in addition to the configuration of the first embodiment.

[0058] The learning side acquisition unit 11 acquires sample videos (a plurality of sample time-series images) captured by imaging means X that is installed so as to capture a predetermined range.

[0059] The learning side detection unit 12 detects the behavior of the sample behaving object captured in the sample video.

[0060] The behavior of the sample behavior unit can be detected by storing the displacement of feature points (such as the movement of each joint) when the sample behavior unit performs a specified behavior in a memory unit (which can be memory unit 4 or another memory unit), detecting the feature points in the same manner as detection unit 3, and detecting that "the specified behavior has been performed" if the displacement of the detected feature point information matches the displacement of the stored feature points by a specified amount or more.

[0061] The determination unit 13 determines one or more "normal behaviors" within a predetermined range based on the multiple behaviors detected by the learning-side detection unit 12. In this embodiment, the determined "normal behaviors" are stored in the storage unit 4 as abnormal behaviors.

[0062] "Normal behavior" can be determined based on various criteria, but for example, it is possible to determine that behavior that has a predetermined (threshold) or higher proportion among all detected behaviors is "normal behavior."

[0063] Then, the determination unit 5 determines that an abnormal behavior has been performed when the displacement of the feature points detected from each time-series image Y does not correspond to "normal behavior."

[0064] In detail, if the displacement of the feature points detected from each time-series image Y does not correspond to “normal behavior”, i.e., if it is determined that abnormal behavior has occurred (S4 in FIG. 3: YES), the operations of S5-S7 in FIG. 3 will be performed as in the first embodiment.

[0065] As described above, in the abnormal behavior notification system 100 according to this embodiment, if the displacement of feature points detected from each time-series image Y does not correspond to the “normal behavior” determined by learning, it is determined that abnormal behavior has occurred.

[0066] According to such a configuration, even if the behavior is not obviously (predetermined) abnormal behavior such as violent behavior or falling, it is possible to regard the behavior as being inappropriate for the situation (predetermined range) as abnormal behavior and to issue a notification. However, in the case of such a configuration, for example, the behavior of a child is more likely to be determined as abnormal behavior, but even in such a case, if the attribute or size meets the first predetermined condition, notification is not issued, thereby suppressing frequent notifications due to abnormal behavior such as "children."

[0067] The abnormal behavior notification system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0068] For example, in the above embodiment, all components are provided integrally, but this does not exclude some components (such as the memory unit 4, judgment unit 5, estimation unit 6, etc.) being provided in different locations (such as the cloud).

[0069] The present invention can also be applied to a program and method corresponding to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is conceivable as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0070] 1. Abnormal Behavior Notification System 2 Acquisition part 3. Detection section 4 Storage section 5 Judgment section 6 Estimation part 7 Notification section 11 Learning side acquisition unit 12 Learning side detector 13 Decision Section 100 Abnormal Behavior Notification System X Shooting method Y Time series images Z Target Action

Claims

1. an acquisition unit for acquiring time-series images captured by an imaging means; A detection unit that detects feature points of a target action object captured in the time-series images; A storage unit that stores displacement of a feature point of a behavior object when the behavior object performs an abnormal behavior; a determination unit that determines whether or not the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; an estimation unit that estimates an attribute or a size of the target action object based on the detected feature points or an appearance of the target action object; a notification unit that issues a notification when it is determined that the abnormal behavior has been performed; Equipped with An abnormal behavior notification system characterized in that the notification unit does not issue the notification even if it is determined that the abnormal behavior has occurred if the attribute or size satisfies a first specified condition and the abnormal behavior satisfies a second specified condition set for the attribute or size.

2. A learning side acquisition unit that acquires a sample video captured by the imaging means installed to capture a predetermined range; A learning side detection unit that detects the behavior of a sample behavior object captured in the sample video; A determination unit that determines one or more normal behaviors in the predetermined range based on the multiple behaviors detected by the learning side detection unit; Further comprising: The abnormal behavior notification system according to claim 1, characterized in that the determination unit determines that the abnormal behavior has been performed when the displacement of feature points detected from each time-series image does not correspond to the normal behavior.

3. A program executed by a computer that stores a displacement of a feature point of a behavior object when the behavior object performs an abnormal behavior, A step of acquiring time-series images captured by an imaging means; detecting feature points of a target action object captured in the time-series images; determining whether or not the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; estimating an attribute or a size of the target action based on the detected feature points or an appearance of the target action; a step of notifying when it is determined that the abnormal behavior has occurred; Equipped with An abnormal behavior notification program characterized in that, in the notification step, even if it is determined that the abnormal behavior has occurred, the notification is not made if the attribute or size satisfies a first specified condition and the abnormal behavior satisfies a second specified condition set for the attribute or size.

4. A method executed by a computer that stores displacements of feature points of an action object when the action object performs an abnormal action, comprising: A step of acquiring time-series images captured by an imaging means; detecting feature points of a target action object captured in the time-series images; determining whether or not the abnormal behavior has occurred based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; estimating an attribute or a size of the target action based on the detected feature points or an appearance of the target action; a step of notifying when it is determined that the abnormal behavior has occurred; Equipped with An abnormal behavior notification method characterized in that, in the notification step, even if it is determined that abnormal behavior has occurred, if the attribute or size satisfies a first specified condition and the abnormal behavior satisfies a second specified condition set for the attribute or size, the notification is not made.

Images