Movement direction estimation device

Abstract

To provide a moving object detection device for accurately detecting a movement direction of a moving object, such as an escalator, reflected in video.SOLUTION: A movement direction estimation device 1 comprises: an acquisition section 2 for acquiring video Y photographed by photographing means X installed so as to photograph a predetermined range including a moving object Z1; a storage section 3 for storing a movement range R of the moving object Z1 within a predetermined range, and a detection reference for detecting at least one portion of an active body Z2 reflected in the video Y; an estimation section 4 for estimating a traveling direction of the moving object Z1 in the video Y by using a background difference for the movement range R; and a detection section 5 capable of detecting the active body Z2 reflected in the video Y. When the active body Z2 is detected within the movement range R, the estimation section 4 estimates a traveling direction of a remaining part excluding an active body part including a part in which the active body Z2 is detected in the movement range R as a traveling direction of the moving object Z1.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a moving object detection device for accurately detecting the direction of movement of a moving object such as an escalator captured on an image. [Background technology]

[0002] A technique has been known in the past for identifying a group of joints belonging to an object based on multiple joints captured in time-series images, and estimating the behavior of the object based on the displacement of the group of joints (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] The above technology can also be used to detect abnormal behavior, such as people riding on moving objects such as escalators, going the wrong way. In this case, however, it is necessary to know the direction of movement of the moving object, since the abnormal behavior will be determined based on the direction of movement of the moving object.

[0005] Incidentally, if the direction of movement of a moving object is limited to one direction, it is sufficient to store that direction of movement in advance. However, for example, in the case of an escalator, there are cases where the direction of movement is changed from top to bottom or bottom to top.

[0006] Therefore, in the case of a moving object that can change its direction of movement, it is necessary to grasp its direction of movement from the image using a background subtraction method or the like. However, if a person overlaps the moving object, or if the overlapping person is moving in a different direction from the moving object, there is a problem in that the influence of the person can cause the detection of the moving direction of the moving object to fail.

[0007] Therefore, an object of the present invention is to provide a movement direction estimation device for accurately detecting the movement direction of a moving object such as an escalator captured in an image. [Means for solving the problem]

[0008] The present invention provides a movement direction estimation device comprising: an acquisition unit that acquires video captured by a photographing means installed to photograph a predetermined range including moving objects capable of moving in multiple predetermined directions; a memory unit that stores the movement range of the moving object within the predetermined range and detection criteria for detecting at least a portion of the moving object captured in the video; an estimation unit that estimates the movement direction of the moving object in the video using background difference for the movement range; and a detection unit that is capable of detecting at least a portion of the moving object captured in the video based on the detection criteria, wherein when the moving object is detected within the movement range, the estimation unit estimates the movement direction of the remaining part of the movement range excluding the moving object part including the part where the moving object was detected as the movement direction of the moving object.

[0009] According to another aspect of the present invention, there are provided a movement direction estimation program and a movement direction estimation method corresponding to the above movement direction estimation device. [Effects of the Invention]

[0010] According to the movement direction estimation device of the present invention, it is possible to accurately detect the movement direction of a moving object such as an escalator captured in an image. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram of a detection target in a movement direction estimation device according to an embodiment of the present invention; [Figure 2] 1 is a block diagram of a movement direction estimation device according to an embodiment of the present invention; [Figure 3] Flowchart of an abnormal behavior detection system according to an embodiment of the present invention DETAILED DESCRIPTION OF THE INVENTION

[0012] A movement direction estimating device 1 according to an embodiment of the present invention will be described below with reference to FIGS.

[0013] As shown in FIG. 1, the movement direction estimation device 1 is for detecting the movement direction of a moving object Z1 captured in an image Y (in FIG. 1, frames constituting the image) captured by a capturing means X. In this embodiment, an escalator whose movement direction can be changed from top to bottom or bottom to top is used as the moving object Z1. FIG. 1 also shows an example in which a moving object Z2 is superimposed on the moving object Z1. In this embodiment, a human is used as the moving object Z2, and for ease of understanding, the moving object Z2 is simply displayed using only a skeleton.

[0014] As shown in FIG. 2, the movement direction estimation device 1 includes an acquisition unit 2, a storage unit 3, an estimation unit 4, a detection unit 5, and a determination unit 6.

[0015] The acquisition unit 2 acquires an image Y captured by an image capturing means X installed to capture a predetermined range including a moving object Z1 that can move in a plurality of predetermined directions. In this embodiment, the entire capture range captured by the image capturing means X is set as the predetermined range.

[0016] The storage unit 3 stores the movement range R of the moving object Z1 within the predetermined range and a detection criterion for detecting at least a part of the moving object Z2 captured in the image Y.

[0017] In this embodiment, the user stores as the movement range R the range in which the steps of the escalator are present (visible), as shown in FIG.

[0018] In this embodiment, a "joint identification standard", a "moving object identification standard", and a "behavior identification standard" are stored as detection standards.

[0019] The "joint identification standard" is used to identify the multiple joints of the moving body Z2 (neck, right elbow, left elbow, waist, right knee, left knee, etc.), and indicates the shape, direction, size, etc. for identifying each joint.

[0020] The "actor identification criteria" indicate the "basic posture" of various variations of the acting entity Z2 ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" in a single acting entity Z2.

[0021] The "behavior identification standard" indicates the movement of each joint when the behavioral body Z2 performs a predetermined behavior, and in this embodiment, "behavior identification standard" for various types of behavior is stored.

[0022] The estimation unit 4 estimates the moving direction of the moving object Z1 in the video Y by using background subtraction for the moving range R. In this embodiment, the moving direction is estimated according to changes in the position of the escalator steps. For the above estimation, a known background subtraction method can be used, and in this embodiment, the moving direction can be estimated based on the difference in the position of each step (a collection of feature amounts) in each frame.

[0023] Based on the detection standard, the detection unit 5 is capable of detecting at least a part of the moving object Z2 captured in the image Y. In this embodiment, the entire body of the moving object Z2 is detected.

[0024] To detect the moving object Z2, first, the moving object Z2 captured in the image Y is identified.

[0025] In this embodiment, after detecting multiple joints that match the "joint identification standard" stored in the memory unit 3, the multiple joints included in one moving object Z2 are identified by referring to the "moving object identification standard." In the example of Figure 1, all joints are identified as joints included in one moving object Z2, and it is identified that one moving object Z2 exists.

[0026] The determination unit 6 determines the behavior of the moving object Z2 based on at least a part of the detected moving object Z2.

[0027] In this embodiment, after identifying the moving body Z2, if the movement of each joint that corresponds to the "behavior identification criterion" is detected, it is detected that "the corresponding behavior has been performed."

[0028] For example, when the moving object Z2 is moving in a direction substantially opposite to the moving direction of the moving object Z1 estimated by the estimation unit 4, the determination unit 6 determines that the moving object Z2 is running in the opposite direction.

[0029] Here, in order to determine whether the moving object Z1 is moving in the wrong direction, it is also necessary to know the direction of movement of the moving object Z1. However, since the escalator in this embodiment can change its direction of movement from top to bottom or bottom to top, it is necessary to know its direction of movement.

[0030] Therefore, in this embodiment, the estimation unit 4 estimates the movement direction of the moving object Z1 using background difference for the movement range R. However, if the moving object Z2 overlaps with the moving object Z1 (escalator) in the movement range R, or if the overlapping moving object Z2 is moving in a different direction from the moving object Z1, the movement direction of the moving object Z1 (escalator) and the movement direction of the moving object Z2 will be mixed, resulting in a problem in which the influence of the moving object Z2 will result in failure to estimate the movement direction of the moving object Z1 (escalator).

[0031] Furthermore, if the moving object Z2 is moving in the wrong direction, the direction of movement of the moving object Z1 (escalator) cannot be accurately estimated due to the influence of the moving object Z2 moving in the wrong direction, resulting in a problem in which the system fails to determine whether the moving object Z2 is moving in the wrong direction.

[0032] Therefore, in this embodiment, when a moving object Z2 is detected within the movement range R, the estimation unit 4 estimates the movement direction of the remaining part of the movement range R excluding the moving object part where the moving object Z2 is detected as the movement direction of the moving object Z1.

[0033] In the above estimation, the movement of the moving body part does not need to be used to estimate the moving direction, and the moving body part may be considered to be moving in the same direction as the moving object Z1.

[0034] The moving body part may be not only the actually detected part (the joint in this embodiment), but also, for example, the skeleton part connecting the detected joints, or the silhouette part including the skeleton part (within a predetermined distance from the skeleton part) (both are shown in Figure 1 for ease of understanding).

[0035] In this way, the movement direction estimation device 1 according to this embodiment can estimate the movement direction of the moving object Z1 without being affected by the movement of the moving object Z2, thereby making it possible to accurately estimate the movement direction of the moving object Z1. Then, based on the accurately estimated movement direction of the moving object Z1 and the behavior of the moving object Z2 detected by the detection unit 5, it becomes possible to accurately detect abnormal behavior of the moving object Z2, such as running in the wrong direction.

[0036] Next, estimation of the movement direction by the movement direction estimation device 1 will be described using the flowchart in Fig. 3. Note that Fig. 3 will be described assuming that a moving object Z2 is captured in the video Y.

[0037] First, when an image Y is acquired (S1), a moving object Z2 captured in the image Y is detected (S2).

[0038] Next, the movement direction of the moving object Z1 in the image Y is estimated using background difference for the movement range R, and if the detected moving object Z2 is located within the movement range R (S3: YES), the movement direction of the remaining part of the movement range R excluding the moving object part is estimated as the movement direction of the moving object Z1 (S4).

[0039] On the other hand, if the moving object Z2 is not detected within the movement range R (S3: NO), the movement direction of the moving object Z1 is estimated using the background difference for the movement range R as is (S5).

[0040] If the moving object Z2 is not captured in the image Y, the process of S5 is carried out after S1.

[0041] As described above, in the movement direction estimation device 1 of this embodiment, when a moving object Z2 is detected within the movement range R, the movement direction of the remaining part of the movement range R excluding the moving object part including the part where the moving object Z2 is detected is estimated as the movement direction of the moving object Z1.

[0042] According to this configuration, the moving direction of the moving object Z1 can be estimated without being affected by the movement of the moving object Z2, and therefore the moving direction of the moving object Z1 can be accurately estimated.

[0043] In addition, in the movement direction estimation device 1 according to this embodiment, when the moving object Z2 is moving in a direction substantially opposite to the movement direction of the moving object Z1 estimated by the estimation unit 4, it is determined that the moving object Z2 is running in the opposite direction.

[0044] With this configuration, the moving direction of the moving object Z1 can be accurately estimated, and it is therefore possible to accurately determine whether the moving object Z2 is moving in the opposite direction.

[0045] The movement direction estimation device of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0046] For example, in the above embodiment, the moving direction of the moving object Z1 is estimated to determine whether the moving object Z1 is running in the wrong direction, but the moving direction of the moving object Z1 may be estimated in relation to other actions.

[0047] Furthermore, since the degree of danger of abnormal behavior such as falling or jumping varies depending on the direction of movement of the escalator, the moving direction of the moving object Z1 may be estimated in order to determine abnormal behavior taking into account the degree of danger.

[0048] In this case, the movement direction estimation device 1 further includes a notification unit that issues a notification in accordance with the determined behavior, and stores a risk level corresponding to the movement direction of the moving object Z1 for a predetermined behavior in the storage unit 3. Then, when it is determined that the predetermined behavior has been performed, the notification unit may be configured to issue a different notification in accordance with the risk level corresponding to the estimated movement direction of the moving object Z1.

[0049] For example, if the predetermined action is "falling," the risk is often higher when the escalator is moving downward, so the notification unit may issue an alert with a louder volume or more conspicuous display when the escalator is moving downward than when it is moving upward. Also, even for the same action, there may be cases where the moving object Z1 is dangerous when moving in one direction but not dangerous when moving in another direction, and in this case the notification unit may issue a "different alert" by not issuing an alert when the escalator is moving in the other direction.

[0050] Furthermore, in the above embodiment, an escalator has been described as an example of the moving object Z1, but the present invention may also be applied to other moving objects such as a walking sidewalk, a conveyor in a factory line, or a conveyor belt for sushi.

[0051] In the case of a factory line or a conveyor belt sushi restaurant, if a worker's or customer's hand ("at least part of the moving object" of this invention) is detected within the movement range R, the estimation unit 4 may estimate the movement direction of the remaining part excluding the detected hand as the movement direction of the moving object (conveyor).

[0052] Furthermore, "using background subtraction" in the present invention also includes the use of background subtraction in optical flow.

[0053] Furthermore, in the above embodiment, a human being is used as the moving body Z2, but it is also possible to use an animal or a robot.

[0054] Furthermore, in the above embodiment, the behavior of a moving object is detected based on the movements of a plurality of joints, but the behavior of a moving object may be detected by other methods without detecting the joints.

[0055] The present invention can also be applied to a program and method corresponding to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transitory recording medium. A CD-ROM or the like is conceivable as a non-transitory recording medium, but is not limited to this. [Explanation of symbols]

[0056] 1 Movement direction estimation device 2 Acquisition part 3 Storage section 4 Estimation part 5. Detection unit 6 Judgment section R Moving Range X Shooting Method Y Video Z1 Moving object Z2 Action Body

Claims

1. an acquisition unit that acquires an image captured by an image capturing means installed to capture an image of a predetermined range including a moving object that can move in a plurality of predetermined directions; a storage unit that stores a movement range of the moving object within the predetermined range and a detection criterion for detecting at least a part of the moving object captured in the video; an estimation unit that estimates a moving direction of the moving object in the video by using a background difference with respect to the moving range; a detection unit capable of detecting at least a part of a moving object captured in the video based on the detection criterion; Equipped with A movement direction estimation device characterized in that, when the moving object is detected within the movement range, the estimation unit estimates the movement direction of the remaining part of the movement range excluding the moving object part including the part where the moving object was detected as the movement direction of the moving object.

2. a determination unit that determines the behavior of the behavior object based on at least a part of the detected behavior object; The movement direction estimation device according to claim 1, characterized in that the determination unit determines that the moving object is traveling in the opposite direction when the moving object is moving in a direction substantially opposite to the movement direction of the moving object estimated by the estimation unit.

3. a determination unit that determines the behavior of the behavior object based on at least a part of the detected behavior object; a notification unit that issues a notification in accordance with the determined behavior; Further provided with the storage unit stores a degree of danger corresponding to a moving direction of the moving object with respect to a predetermined action; The movement direction estimation device according to claim 1, characterized in that, when it is determined that the predetermined action has been performed, the notification unit issues a different notification depending on the degree of danger corresponding to the estimated movement direction of the moving object.

4. A program executed by a computer that stores a movement range within a predetermined range of a moving object that can move in a plurality of predetermined directions and a detection criterion for detecting at least a part of the moving object captured in a video, acquiring an image captured by an image capturing means installed to capture the predetermined range including the moving object; estimating a moving direction of the moving object in the video using background subtraction for the moving range; detecting at least a portion of a moving object captured in the video based on the detection criteria; Equipped with A movement direction estimation program characterized in that, in the estimating step, if the moving object is detected within the movement range, the estimation unit estimates the movement direction of the remaining part of the movement range excluding the moving object part including the part where the moving object was detected as the movement direction of the moving object.

5. A method executed by a computer that stores a movement range within a predetermined range of a moving object that can move in a plurality of predetermined directions, and a detection criterion for detecting at least a part of the moving object captured in a video, the method comprising: acquiring an image captured by an image capturing means installed to capture the predetermined range including the moving object; estimating a moving direction of the moving object in the video using background subtraction for the moving range; detecting at least a portion of a moving object captured in the video based on the detection criteria; Equipped with A movement direction estimation method characterized in that, in the estimating step, if the moving object is detected within the movement range, the estimation unit estimates the movement direction of the remaining part of the movement range excluding the moving object part including the part where the moving object was detected as the movement direction of the moving object.

Images