Attention-Calling-Action System

Abstract

To provide a system for calling for attention behavior capable of calling for a location appropriate to a behavior when attention behavior such as abnormal behavior of a behavioral body captured in a video is detected. SOLUTION: The attention behavior calling system 1 includes a determination unit 5 that determines whether attention behavior has been performed based on the displacement of stored feature points and the displacement of feature points detected from each time-series image Y, a notification unit 7 that is provided around the shooting range of the shooting means X and that issues a voice call when it is determined that attention behavior has been performed, and a background detection unit 8 that detects the background captured in the time-series image Y. Even when it is determined that attention behavior has been performed, the notification unit 8 does not issue a call or changes the content or manner of the call when the detected background satisfies a first predetermined condition.

Description

[Technical field]

[0001] The present invention relates to a system for calling out attention to an action that is appropriate for a location when attention to an action such as abnormal behavior of an object captured in a video is detected. [Background technology]

[0002] There has been known a technique for extracting a behavioral entity that has behaved differently from a predetermined "normal behavior" from a time-series image when the behavior of the behavioral entity captured in the time-series image differs from the "normal behavior" (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] By using the above technology, it is conceivable that an alarm unit provided near the imaging means could issue a call to an entity that has engaged in abnormal behavior.

[0005] However, depending on the location, there may be situations where it is not appropriate to make a blanket appeal even if abnormal behavior is occurring.

[0006] Therefore, the present invention aims to provide a system for calling out attention to behaviors that can make a call appropriate to a location when attention to behaviors, such as abnormal behavior, of an entity captured in a video are detected. [Means for solving the problem]

[0007] The present invention provides an attention behavior calling system comprising an acquisition unit that acquires time-series images captured by a photographing means, a detection unit that detects feature points of a target behavior object captured in the time-series images, a memory unit that stores the displacement of the feature points of the behavior object when the behavior object performs an attention behavior, a judgment unit that judges whether the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image, an alarm unit that is provided around the photographing range of the photographing means and issues an audio call when it is determined that the attention behavior has been performed, and a background detection unit that detects the background captured in the time-series images, wherein the alarm unit does not issue the call or changes the content or manner of the call if the detected background satisfies a first predetermined condition, even if it is determined that the attention behavior has been performed.

[0008] With this configuration, when a notable behavior, such as an abnormal behavior, of a target behavior entity is detected in a time-series image, it becomes possible to make a call appropriate to the location.

[0009] In addition, in another aspect of the present invention, there are provided a program and a method for calling for attention to an action corresponding to the above-mentioned system for calling for attention to an action. Effect of the Invention

[0010] According to the attention behavior calling system of the present invention, when attention behavior such as abnormal behavior of a behavioral subject captured in a video is detected, it is possible to make a call appropriate to the location. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram of a time-series image according to a first embodiment of the present invention; [Diagram 2] Block diagram of an attention behavior calling system according to a first embodiment of the present invention. [Diagram 3] 1 is a flowchart of a system for calling attention to behavior according to a first embodiment of the present invention; [Figure 4]Block diagram of an attention behavior calling system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an attention behavior calling system 1 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG.

[0013] As shown in Fig. 1, the attention behavior calling system 1 is for calling out when attention behavior (abnormal behavior, etc.) of a target behavior entity Z is detected in time-series images Y (frames constituting a video in Fig. 1) captured by a photographing means X. In this embodiment, a human being is used as the target behavior entity Z, and for ease of understanding, the target behavior entity Z is simply displayed with only a skeleton.

[0014] 2, the attention behavior call system 1 includes an acquisition unit 2, a detection unit 3, a storage unit 4, a determination unit 5, an estimation unit 6, a notification unit 7, and a background detection unit 8. In this embodiment, the attention behavior call system 1 is provided integrally with an image capture means X.

[0015] The acquisition unit 2 acquires time-series images Y captured by the imaging means X.

[0016] The detection unit 3 detects feature points of a target action object Z captured in a time-series image Y.

[0017] There are various possible feature points, but in this embodiment, an example will be described in which joints are detected as feature points.

[0018] When detecting joints as feature points, for example, the following method can be considered.

[0019] First, a "joint identification standard" and a "behavior object identification standard" are stored in a storage unit (which may be the storage unit 4 or another storage unit).

[0020] The "joint identification standard" is for identifying multiple human joints, and indicates the shape, direction, size, etc. for identifying each joint.

[0021] The "Behavior Identification Standard" indicates the "basic postures" of various human variations ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" in a single human.

[0022] After detecting multiple joints that meet the above-mentioned "joint identification criteria," it is possible to identify the joints contained in each target action object Z by referring to the "action object identification criteria."

[0023] Note that feature points may be detected individually for each time-series image Y, or may be detected collectively from a plurality of time-series images Y if the time-series order can be specified.

[0024] The storage unit 4 stores the displacements of the feature points of the action object when the action object performs the action of interest. When joints are detected as feature points, the displacements of a plurality of joints are stored.

[0025] Examples of the behavior of interest include falling, punching, kicking, etc., and a plurality of behaviors may be stored without being limited to one. Also, the behavior of interest may be a "premonition behavior" such as a preparatory movement for shoplifting. Note that the memory unit 4 may store the displacement of the feature point information when behaviors other than the behavior of interest (walking, stopping, etc.) occur in order to estimate such behaviors.

[0026] The determination unit 5 determines whether or not attention behavior has been performed based on the stored displacements of the feature points and the displacements of the feature points detected from each of the time-series images Y.

[0027] For example, when the displacement of the detected feature point matches the displacement of the feature point of the attention behavior for a predetermined period or more, it may be determined that "the attention behavior has been performed."

[0028] The estimation unit 6 estimates the attributes or size of the target action object Z based on the detected feature points or the appearance of the target action object Z.

[0029] The attributes or size of the target action object Z may be estimated by a known method.

[0030] For example, when identifying the target action object Z by joints as described above, it is possible to estimate the size of the target action object Z by determining the distance between the head joint point and the ankle joint point.

[0031] In addition to the size described above, attributes can be estimated from characteristic points such as posture and bone structure, and external appearance such as clothing, hair length, and hair color.

[0032] The notification unit 7 is provided around the photographing range of the photographing means X, and issues a voice call when it is determined that attention behavior has been performed.

[0033] In this embodiment, different calls may be made depending on the type of behavior of interest. For example, if the behavior of interest is a criminal act (shoplifting, violent acts, etc.), calls such as "Shoplifting (violent acts) are a crime" may be made, and if the behavior of interest is an accident (falling, etc.), calls such as "Are you OK?", "Can you get up?", "Are you hurt?", etc. may be made.

[0034] In the case of a criminal act, the above call is expected to stop the criminal act. In addition, in the case of an accident, it is possible to determine whether or not rescue is required based on the action or gesture of the target actor Z in response to the call.

[0035] Incidentally, there may be situations in which it is not appropriate to issue a blanket warning even if abnormal behavior is occurring, depending on the location.

[0036] For example, if the location is stairs, there is a risk that the target object Z will be startled by the call and fall; if the location is an elevator, the distance between the alarm unit 7 and the target object Z is short, so the voice may be heard too loudly.

[0037] Therefore, in this embodiment, the background detection unit 8 detects the background shown in the time-series image Y, and the alarm unit 7 does not issue a call or changes the content or manner of the call even if it is determined that attention behavior has been performed if the detected background satisfies a first predetermined condition.

[0038] Any known method can be used to detect the background. For example, various backgrounds and objects can be stored in a memory unit 4, etc., and if the background shown in the time-series image Y has a degree of similarity with the stored background or objects that is equal to or greater than a predetermined degree, it can be detected as that background.

[0039] A possible background for "not making a call" (first predetermined condition) would be a place where there is a risk of falling if a call is made, such as stairs, escalators, etc. Also, a possible background for "changing the content or manner of the call" (first predetermined condition) would be a narrow place (elevator, etc.) where it is better to turn down the volume, or a wide place (outdoors, etc.) where it is better to turn up the volume.

[0040] Furthermore, in this embodiment, the attribute or size of the target action object Z estimated by the estimation unit 6 is also taken into consideration.

[0041] In detail, even if it is determined that attention behavior has been performed, if the detected background satisfies a first predetermined condition and the attribute or size satisfies a second predetermined condition set for the background, the notification unit 7 does not issue a call or changes the content or manner of the call.

[0042] For example, if an elderly person is paying attention on the stairs, calling out to them may cause them to fall. Therefore, by setting the first predetermined condition to "stairs" and the second predetermined condition to "elderly people," it is possible to avoid calling out to them in the case of "elderly people paying attention on the stairs."

[0043] On the other hand, children are light and therefore the risk of falling is low, so leaving them alone increases the risk. Therefore, by setting the first predetermined condition to "stairs" and the second predetermined condition to "children," it is possible to call out to them when "children's attention-grabbing behavior on stairs" occurs.

[0044] In addition, since it is not desirable for the volume to be too loud in a small elevator, by setting the first specified condition to "elevator" and the second specified condition to "non-elderly people," it is possible to change the manner of the call (lower the volume of the call) in the event of "attention behavior by people other than elderly people in the elevator."

[0045] On the other hand, since elderly people are often hard of hearing, by setting the first specified condition to "elevator" and the second specified condition to "elderly people," it is possible to avoid changing the manner of the call (not lowering the volume) in the case of "elderly people paying attention in an elevator."

[0046] Furthermore, in the case of young men, it is possible that they will not stop their attention behavior even if you simply call out to them, "Please stop doing XX." Therefore, by setting the first predetermined condition to "All locations" and the second predetermined condition to "Attribute: Young men," it is possible to change the content of the call to something like, "Please stop doing XX. Security guards are coming," in the case of "attention behavior by young men (regardless of location)."

[0047] As described above, since various combinations of the first predetermined condition related to the "background" and the second predetermined condition related to the "attribute or size" are possible, various combinations may be set.

[0048] The first condition and the second condition may be set by a setting unit provided in the attention behavior call system 1 so that the user can set them.

[0049] Furthermore, although the above describes the case where attributes are taken into consideration, similar control may also be performed simply when the size of the target action object Z satisfies a first predetermined condition (for example, when "the size (height) of the target action object Z is smaller than a predetermined value").

[0050] In this way, in the attention behavior calling system 1 according to this embodiment, when attention behavior such as abnormal behavior of the target behavior object Z is detected in the time-series image Y, it is possible to make a call appropriate to the location.

[0051] Next, the flow of a call according to this embodiment will be described with reference to the flowchart of FIG.

[0052] First, when the acquisition unit 2 acquires a time-series image Y (S1), the detection unit 3 detects feature points of a target action object Z captured in the time-series image Y (S2).

[0053] Furthermore, the background captured in the time-series image Y is detected (S3).

[0054] Next, based on the displacements of the feature points stored in the storage unit 4 and the displacements of the feature points detected in S2, it is determined whether or not attention behavior has been performed (S4).

[0055] If it is determined that attention behavior has been performed (S4: YES), it is determined whether or not the background detected in S3 satisfies a first predetermined condition (S5).

[0056] If the first predetermined condition is not met (S5: NO), a normal voice call is made (S6).

[0057] On the other hand, if the first predetermined condition is met (S5: YES), the call is not made, or the content or manner of the call is changed (S7).

[0058] It should be noted that the background detection in S3 needs to be performed before S5.

[0059] As described above, in the attention behavior call system 1 of this embodiment, even if it is determined that attention behavior has been performed, if the detected background satisfies a first predetermined condition, no call is made, or the content or manner of the call is changed.

[0060] With this configuration, when a noteworthy behavior such as an abnormal behavior of the target behavior object Z is detected in the time-series images Y, it is possible to make a call appropriate to the location.

[0061] Furthermore, in the attention behavior call system 1 of this embodiment, even if it is determined that attention behavior has been performed, if the detected background satisfies a first specified condition and the attributes or size satisfy a second specified condition set for the detected background, no call is made, or the content or manner of the call is changed.

[0062] According to such a configuration, it is possible to make a call that is appropriate for attributes in addition to the location.

[0063] Next, an attention behavior calling system 100 according to a second embodiment of the present invention will be described with reference to Fig. 4. Note that the same members as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0064] In this embodiment, the attention behavior stored in the storage unit 4 is determined by learning.

[0065] In detail, the attention behavior call system 100 includes a learning-side acquisition unit 11, a learning-side detection unit 12, and a determination unit 13 in addition to the configuration of the first embodiment.

[0066] The learning side acquisition unit 11 acquires sample videos (a plurality of sample time-series images) captured by imaging means X that is installed so as to capture a predetermined range.

[0067] The learning side detection unit 12 detects the behavior of the sample behaving object captured in the sample video.

[0068] The behavior of the sample behavior unit can be detected by storing the displacement of feature points (such as the movement of each joint) when the sample behavior unit performs a specified behavior in a memory unit (which can be memory unit 4 or another memory unit), detecting the feature points in the same manner as detection unit 3, and detecting that "the specified behavior has been performed" if the displacement of the detected feature point information matches the displacement of the stored feature points by a specified amount or more.

[0069] The determination unit 13 determines one or more "normal behaviors" within a predetermined range based on the multiple behaviors detected by the learning-side detection unit 12. In this embodiment, the determined "normal behaviors" are stored in the storage unit 4 as attention behaviors.

[0070] "Normal behavior" can be determined based on various criteria, but for example, it is possible to determine that behavior that has a predetermined (threshold) or higher proportion among all detected behaviors is "normal behavior."

[0071] Then, the determination unit 5 determines that an attention behavior has been performed when the displacement of the feature point detected from each time-series image Y does not correspond to "normal behavior."

[0072] In detail, if the displacement of the feature points detected from each time-series image Y does not correspond to “normal behavior”, i.e., if it is determined that attention behavior has occurred (S4 in FIG. 3: YES), the operations of S5-S7 in FIG. 3 are performed as in the first embodiment.

[0073] As described above, in the attention behavior calling system 100 according to this embodiment, if the displacement of feature points detected from each time-series image Y does not correspond to the “normal behavior” determined by learning, it is determined that attention behavior has been performed.

[0074] According to such a configuration, even if the behavior is not obviously abnormal such as violent behavior or falling, it is possible to call out to the behavior that is inappropriate for the place (predetermined range). However, in the case of such a configuration, for example, the behavior of a child is more likely to be determined as attention-grabbing behavior (abnormal behavior), but even in such a case, if the attribute or size satisfies the first predetermined condition, the call is not made or the content or manner of the call is changed, thereby suppressing frequent calls due to attention-grabbing behavior (abnormal behavior) such as "children."

[0075] The attention behavior call system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0076] For example, in the above embodiment, the imaging means X is provided integrally with the alarm unit 7, but it does not necessarily have to be integral with the imaging means, and may be provided in the vicinity of the imaging range of the imaging means X so that a call can be made to the target action object Z.

[0077] In addition, in the above embodiment, all components are provided integrally, but for example, this does not exclude some components (memory unit 4, judgment unit 5, estimation unit 6, etc.) being provided in different locations (cloud, etc.).

[0078] The present invention can also be applied to a program and method corresponding to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is conceivable as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0079] 1. Attention-Calling System 2 Acquisition part 3. Detection section 4 Storage section 5 Judgment section 6 Estimation part 7. Notification Department 8 Background detection section 9. Setting section 11 Learning side acquisition unit 12 Learning side detector 13 Decision Section 100 Attention Call to Action System X Shooting method Y Time series images Z Target Action

Claims

1. an acquisition unit for acquiring time-series images captured by an imaging means; A detection unit that detects feature points of a target action object captured in the time-series images; A storage unit that stores a displacement of a feature point of a behavior object when the behavior object performs a behavior of interest; a determination unit that determines whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; a notification unit that is provided around the image capturing range of the image capturing means and that issues a voice call when it is determined that the attention behavior has been performed; a background detection unit that detects a background shown in the time-series images; Equipped with The attention behavior calling system is characterized in that the notification unit does not make the call or changes the content or manner of the call when the detected background satisfies a first predetermined condition, even if it is determined that the attention behavior has been performed.

2. An estimation unit is further provided for estimating an attribute or a size of the target action object based on the detected feature points or an appearance of the target action object, The attention behavior calling system described in claim 1, characterized in that the notification unit does not make the call or changes the content or manner of the call when the detected background satisfies a first specified condition and the attribute or size satisfies a second specified condition set for the background, even if it is determined that the attention behavior has been performed.

3. The attention behavior calling system according to claim 1, further comprising a setting unit that enables a user to set whether or not to not make the call or to change the content or manner of the call when the detected background satisfies a first predetermined condition.

4. A learning side acquisition unit that acquires a sample video captured by the imaging means installed to capture a predetermined range; A learning side detection unit that detects the behavior of a sample behavior object captured in the sample video; A determination unit that determines one or more normal behaviors in the predetermined range based on the multiple behaviors detected by the learning side detection unit; Further comprising: The system for calling for attention to an action as described in claim 1, characterized in that the determination unit determines that the attention behavior has been performed when the displacement of feature points detected from each time-series image does not correspond to the normal behavior.

5. A program executed by a computer that stores a displacement of a feature point of a behavior object when the behavior object performs a target behavior, A step of acquiring time-series images captured by an imaging means; detecting feature points of a target action object captured in the time-series images; determining whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; a step of providing a voice call when the attention behavior is determined to have been performed, the voice call being provided around the photographing range of the photographing means; detecting a background in the time-series images; Equipped with In the step of making a call, even if it is determined that the attention behavior has been performed, if the detected background satisfies a first specified condition, the call is not made or the content or manner of the call is changed.

6. A method executed by a computer that stores a displacement of a feature point of a behavior object when the behavior object performs a target behavior, comprising: A step of acquiring time-series images captured by an imaging means; detecting feature points of a target action object captured in the time-series images; determining whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points detected from each time-series image; a step of providing a voice call when the attention behavior is determined to have been performed, the voice call being provided around the photographing range of the photographing means; detecting a background in the time-series images; Equipped with A method for calling out attention behavior, characterized in that in the step of making the call, even if it is determined that the attention behavior has been performed, if the detected background satisfies a first predetermined condition, the call is not made or the content or manner of the call is changed.

Images