Behavior Prediction System

Abstract

To provide a behavior estimation system that suppresses erroneous detection of a non-acting object captured in a video as an active object. [Solution] The behavior estimation system 1 includes an acquisition unit 2 that acquires an image Y captured by an imaging means X installed to capture a specified range, a detection unit 4 that detects a behavioral object Z captured in the image Y based on features in the image Y, an estimation unit 5 that estimates the behavior of the behavioral object Z based on the displacement of the features of the detected behavioral object Z, an output unit 6 that outputs information regarding the estimated behavior, a counting unit 7 that counts the cumulative number of times the behavioral object is detected by the detection unit 4 within a specified time at each of a plurality of positions within a specified range, and a control unit 8 that controls the estimation unit 5 not to estimate behavior for high-frequency positions among the plurality of positions where the cumulative number of times exceeds a specified number, or controls the output unit 6 not to output information regarding the behavior.

Description

[Technical field]

[0001] The present invention relates to a behavior estimation system that suppresses erroneous detection of a non-acting object captured in a video as an active object. [Background technology]

[0002] There is a known technique for identifying a group of joints belonging to an object based on a plurality of joints captured in time-series images, and estimating the behavior of the object based on the displacement of the group of joints (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above technique has a problem in that a non-moving object such as a mannequin or a shadow captured in a time-series image (video) is mistakenly detected as a moving object.

[0005] Therefore, an object of the present invention is to provide a behavior estimation system that suppresses erroneous detection of a non-acting object captured in a video as an active object. [Means for solving the problem]

[0006] The present invention provides a behavior estimation system comprising: an acquisition unit that acquires video captured by an imaging means installed to capture a predetermined range; a detection unit that detects a behavioral object captured in the video based on features in the video; an estimation unit that estimates the behavior of the behavioral object based on displacement of the features of the detected behavioral object; an output unit that outputs information related to the estimated behavior; a counting unit that counts the cumulative number of times a behavioral object is detected by the detection unit within a predetermined time at each of a plurality of positions within the predetermined range; and a control unit that controls the estimation unit not to estimate the behavior or controls the output unit not to output information related to the behavior for high-frequency positions among the plurality of positions where the cumulative number exceeds a predetermined number.

[0007] According to another aspect of the present invention, there is provided a behavior estimation system comprising: an acquisition unit that acquires video captured by an imaging means installed to capture a predetermined range; a detection unit that detects a behavioral object captured in the video based on features in the video; an estimation unit that estimates the behavior of the behavioral object based on displacement of the features of the detected behavioral object; an output unit that outputs information related to the estimated behavior; a counting unit that counts the cumulative number of times that the detection unit detected a behavioral object based on features in the same range within a predetermined time period at each of a plurality of positions within the predetermined range; and a control unit that controls the estimation unit not to estimate behavior based on the features in the same range for high-frequency positions among the plurality of positions where the cumulative number exceeds a predetermined number, or controls the output unit not to output information related to the behavior estimated based on the features in the same range.

[0008] According to another aspect of the present invention, there is provided a behavior estimation program corresponding to the above behavior estimation system, and a behavior estimation method. Effect of the Invention

[0009] According to the behavior estimation system of the present invention, it is possible to prevent a non-acting object captured in a video from being erroneously detected as an active object. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram of a usage state of a behavior inference system according to a first embodiment of the present invention; [Diagram 2] 1 is a block diagram of a behavior estimation system according to a first embodiment of the present invention; [Diagram 3] FIG. 1 is an explanatory diagram of the cumulative number of times according to the first embodiment of the present invention; [Figure 4] 1 is a flowchart of behavior estimation according to a first embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A behavior inference system 1 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 to 4. FIG.

[0012] As shown in Fig. 1, the behavior estimation system 1 is for estimating the behavior of a moving object Z captured in an image Y (in Fig. 1, frames constituting the image) captured by an imaging means X. In this embodiment, a human being is used as the moving object Z, and for ease of understanding, the moving object Z is simply displayed with only a skeleton.

[0013] As shown in FIG. 2, the behavior inference system 1 includes an acquisition unit 2, a storage unit 3, a detection unit 4, an estimation unit 5, an output unit 6, a counting unit 7, and a control unit 8.

[0014] The acquisition unit 2 acquires an image Y captured by an image capturing means X that is installed so as to capture a predetermined range. In this embodiment, an example will be described in which the entire range captured by the image capturing means X is set as the predetermined range.

[0015] The storage unit 3 stores a first feature amount criterion for detecting a moving object and a second feature amount criterion for estimating the behavior of the moving object.

[0016] In this embodiment, a "joint identification criterion" and a "behavior object identification criterion" are stored as the first feature amount criterion, and an "action identification criterion" is stored as the second feature amount criterion.

[0017] The "joint identification standard" is used to identify the multiple joints of an action body (neck, right elbow, left elbow, waist, right knee, left knee, etc.), and indicates the shape, direction, size, etc. for identifying each joint.

[0018] The "action object identification criteria" indicate the "basic postures" of various variations of an action object ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" within a single action object.

[0019] The "behavior identification criterion" indicates the movement of each joint when the action object performs a predetermined action, and in this embodiment, "behavior identification criterion" for various types of actions is stored.

[0020] The detection unit 4 detects a moving object Z captured in the video Y based on a feature amount in the video Y.

[0021] In this embodiment, when a feature corresponding to the "joint identification standard" stored in the storage unit 3 is detected, the "actor identification standard" is referenced to identify multiple joints (feature) included in one acting object. In the example of Fig. 1, joints A1-A6 are identified as joint A included in one acting object Z, and one acting object Z is identified as existing.

[0022] The estimation unit 5 estimates the behavior of the moving object Z based on the detected displacement of the feature amount of the moving object Z.

[0023] In this embodiment, when a movement of each joint (feature amount) corresponding to the "behavior identification criterion" is detected, it is estimated that "the behavior in question has been performed."

[0024] The output unit 6 outputs information relating to the estimated behavior.

[0025] The output may be in the form of displaying actions such as "falling" or "standing", or may simply issue an alert when a predetermined action is estimated.

[0026] For example, when the behavior inference system 1 is used for the purpose of crime prevention or monitoring, the output information may be transmitted to a registered information terminal (such as an information terminal of a security company or an individual's mobile information terminal).

[0027] Here, the detection unit 4 detects as a moving object an object whose feature amount is similar to that of a moving object (human), but when a mannequin is captured in the image Y, it erroneously detects the mannequin as a moving object, which is a problem. Also, when the feature amount of a shadow captured in the image Y is similar to that of a moving object, it erroneously detects the shadow as a moving object, which is a problem.

[0028] Therefore, in this embodiment, in high-frequency locations where detection of moving objects is occurring with high frequency, it is assumed that there is a high possibility that a non-moving object is present, and detections in high-frequency locations are not treated as detection of a moving object.

[0029] In detail, the counting unit 7 counts the cumulative number of times that the detection unit 4 detects a moving object within a predetermined period of time at each of a plurality of positions within a predetermined range (photographing range).

[0030] That is, the counting unit 7 counts the cumulative number of times the object has been detected as a moving object, regardless of whether it is an actual moving object or not.

[0031] The "position" may be a position on the screen (coordinates, etc.) or a position in real space (coordinates, etc.).

[0032] When using a position (coordinates, etc.) in real space, it is possible to estimate the position in real space by taking into consideration the planar coordinates on the screen, the coordinates in the depth direction, the size of the action object, etc. As the coordinates in the depth direction, it is possible to use a known depth estimation method, etc.

[0033] Furthermore, in this embodiment, the "position" may not only be a single coordinate, but may also be a certain range. For example, if the screen is divided into multiple parts and a moving object is present in one of the divided screens, it may be counted that the moving object has been detected at the position of that one of the divided screens. The position of the center of gravity of the moving object or other specific part (such as a specified joint) may also be used as the "position". In this case, it may be considered that the "position" is within a specified range from the center of gravity, etc.

[0034] The control unit 8 controls the estimation unit 5 not to estimate behavior for high frequency locations among the above multiple locations where the cumulative number of times exceeds a predetermined number, or controls the output unit 6 not to output information regarding the behavior.

[0035] For example, in Figure 3, detection of a behaving object is repeated with high frequency at the position where mannequin B is located, and when the cumulative number of times reaches a predetermined number, it is determined to be a high-frequency position.From then on, even if a behaving object is detected at that high-frequency position, no estimation of the behavior or output of information related to the behavior will be performed.

[0036] When "behavior estimation is not performed," the power required for estimation and the load on the computer are reduced, and no output is produced, eliminating the need for the user to respond to erroneous information.

[0037] If "information regarding behavior is not output," power etc. will be required for estimation, but even in this case, if the output unit 6 does not output information regarding behavior, the user will not need to respond to erroneous information.

[0038] Next, the estimation of behavior by the behavior estimation system 1 will be described with reference to the flowchart of Fig. 4. In Fig. 4, the above-mentioned case where "behavior estimation is not performed" will be described.

[0039] First, when an image Y capturing a predetermined range is acquired (S1), a moving object Z captured in the image Y is detected based on features within the image Y (S2), and it is determined whether the cumulative number of times the moving object has been detected at the detection position exceeds a predetermined number (S3).

[0040] If the cumulative number of times does not exceed the predetermined number of times (S3: NO), the number of times the moving object was detected at the detection position is added up (S4), and the behavior of the moving object is estimated based on the displacement of the feature amount of the detected moving object (S5). The order of steps S4 and S5 may be reversed.

[0041] On the other hand, if the cumulative number of times exceeds the predetermined number of times (S3: YES), the behavior of the behavior object is not estimated, and the process returns to S1.

[0042] As described above, in the behavior estimation system 1 according to this embodiment, for high-frequency positions among a plurality of positions within a predetermined range (photographing range) where the cumulative number of times exceeds a predetermined number, the estimation unit 5 is controlled not to estimate the behavior, or the output unit 6 is controlled not to output information related to the behavior.

[0043] With this configuration, it is possible to prevent a non-acting object captured in the image Y from being erroneously detected as an acting object.

[0044] Next, a behavior inference system 1 according to a second embodiment of the present invention will be described. The behavior inference system 1 according to this embodiment has the same configuration as the first embodiment, except that the operations of the counting unit 7 and the control unit 8 are partially different.

[0045] In the first embodiment, for high frequency locations, it is assumed that there is a high possibility that a non-acting object is present, and the estimation unit 5 is controlled not to estimate the behavior. However, if an actual acting object passes through the high frequency location, the behavior of the actual acting object will not be estimated.

[0046] Therefore, in this embodiment, even if the location is a high frequency location, when an actual moving object Z is detected, the movement is estimated.

[0047] In detail, the counting unit 7 counts the cumulative number of times that the detection unit 4 detects a moving object within a predetermined time period at each of a plurality of positions within a predetermined range, based on the feature amount within the same range.

[0048] For example, in the case of a permanently installed mannequin, since it is detected as a moving object based on the same feature every time, the feature of the mannequin corresponds to the feature in the same range. In this embodiment, even if the feature is not exactly the same, as long as it is in the same range, it will be added to the cumulative number of times.

[0049] Then, for a high frequency location among the multiple locations where the cumulative number of times exceeds a predetermined number, the control unit 8 controls the estimation unit 5 not to estimate behavior based on the feature amounts in the same range, or controls the output unit 6 not to output information regarding behavior estimated based on the feature amounts in the same range.

[0050] For example, as shown in Figure 3, at the high-frequency location where mannequin B is present, behavior estimation for mannequin B is no longer performed thereafter, but if an actual moving object passes through the high-frequency location, behavior estimation will be performed.

[0051] The predetermined number of times may be the same as that in the first embodiment, or may be set to a different number of times.

[0052] As described above, in the behavior inference system 1 according to this embodiment, for a high-frequency location among a plurality of locations where the cumulative number of times detection was performed based on the same range of feature quantities exceeds a predetermined number, the estimation unit 5 is controlled so as not to infer behavior based on the same range of feature quantities, or the output unit 6 is controlled so as not to output information related to behavior estimated based on the same range of feature quantities.

[0053] With this configuration, it is possible to prevent failure to estimate the behavior of the actual moving object Z when the actual moving object Z passes through a high frequency position.

[0054] The behavior inference system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0055] For example, in the first embodiment, a configuration different from that of the second embodiment may be adopted in which "if an actual moving object Z is detected even in a high frequency position, behavior estimation, etc. is performed."

[0056] For example, when a moving object Z detected at a position different from the high frequency position moves to the high frequency position, a configuration may be considered in which the behavior of the moving object is estimated even at the high frequency position. In this case, the movement of the moving object Z may be tracked by using a known tracking method.

[0057] Furthermore, in the above embodiment, the behavior of the moving object is detected based on the movements of a plurality of joints, but the behavior of the moving object may be detected by other methods without detecting the joints.

[0058] In addition, in the above embodiment, a human being is used as an example of a moving body, but the present invention can also be used for animals and robots.

[0059] Furthermore, each component of the behavior inference system 1 does not have to be located in the same place. For example, some components may be located locally and other components may be located on the cloud.

[0060] The present invention is also applicable to a program and method corresponding to the processes performed by the acquisition unit 2, detection unit 4, estimation unit 5, output unit 6, counting unit 7, and control unit 8 as a controller, and a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is considered as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0061] 1. Behavior estimation system 2 Acquisition part 3 Storage section 4. Detection section 5 Estimation part 6 Output section 7. Counting Section 8. Control Unit A Joint B Mannequin X Shooting method Y Video Z action body

Claims

1. an acquisition unit that acquires an image captured by an image capturing means installed to capture a predetermined range; a detection unit that detects a moving object captured in the video based on a feature amount in the video; an estimation unit that estimates a behavior of the behavior object based on a change in a feature amount of the behavior object detected; an output unit that outputs information regarding the estimated behavior; a counting unit that counts a cumulative number of times that a moving object is detected by the detection unit within a predetermined time period at each of a plurality of positions within the predetermined range; a control unit that controls the estimation unit not to estimate the behavior or controls the output unit not to output information related to the behavior with respect to a high frequency location among the plurality of locations where the cumulative number of times exceeds a predetermined number of times; Equipped with A behavior estimation system characterized in that, when a behavioral object detected at a position different from the high-frequency position moves to the high-frequency position, the control unit controls the estimation unit to estimate the behavior of the behavioral object even at the high-frequency position, or controls the output unit to output information regarding the behavior.

2. A program executed on a computer, acquiring an image captured by a photographing means installed to photograph a predetermined range; detecting a moving object captured in the video based on a feature amount in the video; A step of estimating a behavior of the action object based on a change in a feature amount of the action object detected; outputting information about the estimated behavior; a step of counting a cumulative number of times that a moving object is detected in the detecting step within a predetermined time period at each of a plurality of positions within the predetermined range; Equipped with A behavior estimation program characterized in that, for high-frequency locations among the multiple locations where the cumulative number exceeds a predetermined number, the behavior is not estimated in the estimation step, or information regarding the behavior is not output in the output step, but when a behavioral object detected at a location other than the high-frequency location moves to the high-frequency location, the behavior of the behavioral object is estimated even at the high-frequency location, or information regarding the behavior is output.

3. 1. A computer-implemented method comprising: acquiring an image captured by a photographing means installed to photograph a predetermined range; detecting a moving object captured in the video based on a feature amount in the video; A step of estimating a behavior of the action object based on a change in a feature amount of the action object detected; outputting information about the estimated behavior; a step of counting a cumulative number of times that a moving object is detected in the detecting step within a predetermined time period at each of a plurality of positions within the predetermined range; Equipped with A behavior estimation method characterized in that, for high-frequency positions among the multiple positions where the cumulative number exceeds a predetermined number, the behavior is not estimated in the estimation step, or information regarding the behavior is not output in the output step, but when a behavioral object detected at a position other than the high-frequency position moves to the high-frequency position, the behavior of the behavioral object is estimated even at the high-frequency position, or information regarding the behavior is output.

Images