Movement and retention determination device

Abstract

To provide a movement / retention determination device capable of appropriately determining movement and retention of an acting object displayed on time series images.SOLUTION: A movement / retention determination device 1 includes an acquisition unit 4 which acquires, as an observation point, a prescribed part of the body of an acting object detected from a plurality of time series images in which the acting object body being an object is imaged and which is photographed by photographing means fixed to a prescribed position, and a determination unit 5 which determines movement or retention of the acting object on the basis of a ratio of the observation points contained in a prescribed range centered on one observation point being a reference point which is each acquired in each of the time series images.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a movement / stay determination device for determining the movement or stay of a moving object based on an observation point detected at a predetermined part of the body of the moving object captured in time-series images. [Background technology]

[0002] There is a known technique for determining the length of time a target object (person) included in video data stays in a target area (waiting area) by calculating the elapsed time since the person entered the target area (waiting area) (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6534715 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, for example, in nursing care settings, it may be necessary to have information on whether the person being cared for is moving even slightly or not at all, but the above technology only detects whether the target object (person) is staying within a relatively wide range such as a waiting room, and does not disclose a method for detecting even slight movements of the target object (person).

[0005] In addition, the detection device may fail to detect the target object. In such a case, the above technology would result in the dwell time being reset even though the target object (person) is present within the target area (waiting area).

[0006] Therefore, an object of the present invention is to provide a movement / stay determination device that can appropriately determine the movement or stay of a moving object captured in time-series images. [Means for solving the problem]

[0007] The present invention provides a movement / stay determination device, which is characterized by comprising an acquisition unit that acquires as observation points predetermined parts of the body of a target moving object detected from each of a plurality of time-series images in which the moving object is captured by an imaging means fixed at a predetermined position, and a determination unit that determines whether the moving object is moving or staying based on the proportion of observation points among the observation points acquired in each of the time-series images that are included within a predetermined range centered on one observation point that serves as a reference point.

[0008] According to such a configuration, it is possible to judge the "movement" or "stay" of a moving object with high accuracy by a simple configuration of comparing the ratio of observation points included in a predetermined range with a set ratio. In addition, since the "stay (movement)" is judged including those that are likely to be heading for "stay (movement)", it is possible to make an appropriate judgment according to the actual situation. Furthermore, since it is only necessary to detect the observation point to judge movement or stay, and an image that can identify an individual such as a face image is not required, privacy is also protected. Furthermore, by setting the "ratio" to less than 100%, erroneous judgments are suppressed even in cases where the detection of the observation point has failed.

[0009] Furthermore, it is preferable that, when the multiple time-series images are considered as one set, the acquisition unit acquires the observation points detected from multiple sets in which the time captured by the photographing means overlaps with the previous or subsequent set by a predetermined amount, and the judgment unit sequentially judges the movement or stay of the moving object based on the proportion of observation points that fall within the specified range centered on the observation point captured in a predetermined image in each set.

[0010] According to such a configuration, since the determination of movement or stay is performed sequentially with respect to the changing reference point, it becomes possible to perform a more appropriate determination.

[0011] It is also preferable that the determination section determines whether or not to notify the movement or stay of the action object based on the results of the determinations made sequentially for each set.

[0012] According to such a configuration, it is possible to notify "movement" or "retention" in a situation where it is highly necessary, such as when a person has "remained" for a predetermined period of time or more.

[0013] It is also preferable that the device further comprises a setting section for setting the predetermined range.

[0014] According to such a configuration, by setting a predetermined range according to the purpose, it becomes possible to appropriately judge "movement" and "retention."

[0015] It is also preferable that the observation point is a detected part on the center line of the body of the active subject.

[0016] With this configuration, it is possible to prevent the movement of the limbs from being determined as "movement" unless the entire body is moving.

[0017] According to another aspect of the present invention, there is provided a movement / stay determination program, which is a program installed on a computer and comprises the steps of: acquiring, as an observation point, a predetermined part of the body of a target moving object detected from each of a plurality of time series images in which the moving object is captured by an imaging means fixed at a predetermined position; and determining whether the moving object is moving or staying still based on the proportion of observation points among the observation points acquired in each of the time series images that are included within a predetermined range centered on a single observation point that serves as a reference point. Effect of the Invention

[0018] According to the movement / stay determination device of the present invention, it is possible to appropriately determine the movement or stay of a moving object captured in a time-series image. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is an explanatory diagram of a time-series image according to an embodiment of the present invention; [Diagram 2]FIG. 1 is a block diagram of a movement / stay determination device according to an embodiment of the present invention; [Diagram 3] FIG. 2 is an explanatory diagram of a predetermined range according to an embodiment of the present invention. [Figure 4] FIG. 1 illustrates multiple sets of motion detection according to an embodiment of the present invention. [Diagram 5] Flowchart of movement detection according to an embodiment of the present invention DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, a movement / stay determination device 1 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG.

[0021] 1, the movement / stay determination device 1 is for determining the movement or stay of a target moving object Y captured in a plurality of time-series images X (each frame constituting a video, etc.) captured by a photographing means fixed at a predetermined position. In this embodiment, a human being is adopted as the moving object Y, and for ease of understanding, the moving object Y is simply displayed with only a skeleton.

[0022] As shown in FIG. 2, the movement / stay determination device 1 includes a storage unit 2, a detection unit 3, an acquisition unit 4, a determination unit 5, a setting unit 6, and a notification unit 7.

[0023] The storage unit 2 stores a "joint identification standard" and an "action object specification standard."

[0024] The "joint identification standard" is used to identify the multiple joints of the action object Y (in Figure 1, the neck, right elbow, left elbow, waist, right knee, and left knee), and indicates the shape, direction, size, etc. for identifying each joint.

[0025] The "actor specific criteria" indicate the "basic posture" of various variations in the behavior of the actor Y ("walking", "standing upright", etc.), the "range of motion of each joint", the "distance between each joint" for a single actor Y, etc.

[0026] The detection unit 3 detects a predetermined part of the body of the moving object Y from each of a plurality of time-series images X in which the moving object Y is captured, and the acquisition unit 4 acquires the predetermined part detected by the detection unit 3 as an observation point Z. The predetermined part is preferably a part on the center line of the body, and in this embodiment, the neck joint is detected as the observation point Z, as shown in Fig. 3.

[0027] The observation point Z is detected after identifying the moving object Y captured in each time-series image X.

[0028] Although a known method can be used to identify the action object Y, in this embodiment, multiple joints that correspond to the "joint identification criteria" stored in the memory unit 2 are identified, and then the "action object identification criteria" are referenced to identify multiple joints contained in the action object Y. Then, among the identified joints of the action object Y, a joint that corresponds to the "joint identification criteria" of a specific part (in this embodiment, the "neck") is identified. Then, a similar identification is performed for the next time-series image X, and if there is a high possibility that it is a consecutive action or movement, the action object is identified as being the same as the action object Y identified in the previous time-series image X by referring to the "action object identification criteria."

[0029] The determination unit 5 determines the movement or stay of the moving object Y based on the proportion of observation points Z that are included within a predetermined range R centered on one observation point Z that serves as a reference point S, among the observation points Z acquired in each time-series image X (see FIG. 3). In this embodiment, the determination unit 5 also calculates the proportion of observation points Z that are included within the predetermined range R.

[0030] For example, when the acquisition unit 4 is set to acquire five time-series images X, it may be set to determine that there is “movement” when four or more observation points Z are not included within a predetermined range R (FIG. 3(a)), and determine that there is “no movement (stay)” when four or more observation points Z are included within the predetermined range R (FIGS. 3(b)-(e)).

[0031] Then, the judgment unit 5 performs the above judgment sequentially for the multiple sets.

[0032] In detail, when multiple time-series images X are considered as one set, the acquisition unit 4 sequentially acquires observation points Z detected from multiple sets whose captured times overlap with the previous or subsequent set by a predetermined amount, and the judgment unit 5 sequentially judges the movement or stay of the moving object Y based on the proportion of observation points Z that are included within a predetermined range R centered on the observation point Z acquired in the predetermined image in each set.

[0033] For example, if five time-series images X taken at one second intervals are used as one set, first, in the first set, observation point Z1 detected in the first image X1 of the five time-series images X1-X5 is set as a reference point S, and the proportion of observation points Z detected in each of images X1-X5 that are included within a predetermined range R centered on the reference point S is calculated, as shown in Figures 3(a) and 4. In the first set in which observation point Z1 in image X1 is set as the reference point S, only three observation points Z are included within the predetermined range R, so a "MOVE" judgment is made.

[0034] Next, in the second set, observation point Z2 detected in image X2 of the five time-series images X2-X6 is set as the reference point S, and the proportion of observation points Z detected in each of images X2-X6 that are included within a predetermined range R centered on reference point S is calculated. In Figures 3(b) and 4, in the second set in which observation point Z2 in image X2 is set as the reference point S, five observation points Z are included within the predetermined range R, so a "STAY" judgment is made. Similarly, a "STAY" judgment is made in sets 3-5 in which observation point Z3-5 in image X3-5 is set as the reference point S.

[0035] Note that the intervals (time) at which successive time-series images X are captured are uniform, so in the above example, there is a time overlap of four images between the previous and next sets.

[0036] Here, if the movement of object Y were to be determined using a publicly known tracking method, it might be determined that "it moved significantly in images X1-X4 (observation point Z1-Z4) and moved slightly in images X4-X9 (observation point Z4-Z9), and therefore it remained in image X4-X9 (observation point Z4-Z9)."

[0037] On the other hand, in this embodiment, as shown in Fig. 4, only the first set with the observation point Z1 detected in the image X1 as the reference point S is judged as "moving", and the subsequent sets are judged as "staying". This means that the convergence of the action object Y to "moving" or "staying" is predicted in advance. In other words, for example, even if the action object is actually "moving", it is judged as "staying" early on for the action object heading towards "staying".

[0038] In this manner, in this embodiment, the determination of "stay (movement)" is made including those that are highly likely to be heading toward "stay (movement)", making it possible to make an appropriate determination that is in line with the actual situation.

[0039] However, if the results of the "movement" and "stay" judgments for each set are simply listed and output, it may be difficult for the user to understand how to make a decision based on the judgment results.

[0040] For example, when the movement / stay determination device 1 is used to watch over elderly people or infants, or for crime prevention purposes such as detecting suspicious people loitering, the user would likely want to be notified of "stay" if they "stay" for a predetermined period of time or longer.

[0041] Therefore, in this embodiment, the judgment unit 5 judges whether or not to notify the movement or stay of the moving object Y based on the judgment results obtained in each set in this manner. In detail, a notification criterion is set, and when the judgment results obtained in each set satisfy the notification criterion, the movement or stay of the moving object Y is notified.

[0042] For example, it is possible to set a notification criterion such that a notification of "stay" or "move" is issued when a "stay" or "move" determination is made consecutively for a predetermined set or more.

[0043] In addition, by storing the timing of "staying" and "moving" various actions in the memory unit 2, the mode of movement, for example "turning over" or "looking around", can also be notified according to the judgment results obtained from multiple sets.

[0044] The setting unit 6 is for setting the predetermined range R.

[0045] The predetermined range R may be set, for example, as "within the circumference of a circle having a radius of XX cm centered on the reference point S." In this case, the radius may be set according to the purpose. For example, when the movement / stay determination device 1 is used to watch over elderly people, infants, etc., the predetermined range R may be set narrow so that even slight movements are determined as "movement." On the other hand, when the movement / stay determination device 1 is used to detect suspicious individuals loitering, etc., the predetermined range R may be set wide so that even slight movements are determined as "stay."

[0046] The predetermined range R is not limited to a circumference, and may be set to a shape that is point-symmetrical with respect to the reference point S, such as an ellipse or a rectangle. For example, if it is desired to mainly detect movement in the up-down direction, an ellipse with its major axis in the horizontal direction may be used, whereas if it is desired to mainly detect movement in the left-right direction, an ellipse with its major axis in the vertical direction may be used.

[0047] Furthermore, the size on the image X or the actual size may be used as the predetermined range R. When the actual size is used, for example, a "joint identification standard" and a detected joint may be compared to calculate the scale ratio of the action object Y in the image X, and the predetermined range R according to the calculated scale ratio may be used.

[0048] Furthermore, the setting unit 6 also sets the above-mentioned "criteria for determining movement or stagnation (proportion of observation points Z)" and "notification criteria."

[0049] The notification unit 7 issues a notification when the determination unit 5 determines that "announcement should be issued." The notification may be an audio alert, a notification by e-mail or the like, or simply an output indicating "movement" or "retention."

[0050] Next, the movement / stay determination by the movement / stay determination device 1 will be described with reference to the flowchart of FIG.

[0051] First, a predetermined part of the body of a moving object Y detected from each of a plurality of time-series images X (in this embodiment, the joint of the "neck") is acquired as an observation point Z (S1).

[0052] Next, the movement or stagnation of the active object Y is determined based on the proportion of observation points Z that are included within a predetermined range R centered on one observation point Z (reference point S) among the observation points Z acquired in each time series image X (S2).

[0053] Then, while repeating a plurality of sets of the actions of S1 and S2, if the notification criteria are met (S3: YES), a notification of the movement or stay of the active object Y is made (S4).

[0054] The movement / stay determination device 1 having such a configuration can be used for various purposes such as crime prevention and rescue.

[0055] For example, by applying the movement / stay determination device 1 to time-series images X captured under the eaves of a facility, it becomes possible to detect people who have entered the premises without permission and are sleeping.

[0056] Furthermore, by using the movement / stay determination device 1 in the homes of elderly people living alone, in nursing homes, or to watch over infants, it becomes possible to confirm whether the person is alive or not.

[0057] Furthermore, in the movement / stay determination device 1 according to this embodiment, it is possible to determine whether each of the multiple people is staying or moving by detecting the observation point Z of each of the multiple people. For example, even if multiple people are shown on the same screen and each of them is crossing the screen, falling or committing a violent act, or staying still, it is possible to identify only the people who are staying or moving based on the observation point Z.

[0058] As described above, the movement / stay determination device 1 of this embodiment determines whether an active object Y is moving or staying based on the proportion of observation points Z that are included within a predetermined range R centered on one observation point Z (reference point S) among the observation points Z detected in each time-series image X.

[0059] According to such a configuration, it is possible to judge the "movement" or "stay" of the action object Y with a high degree of accuracy by a simple configuration of comparing the ratio of the observation points Z included in the predetermined range R with the set ratio. In addition, since the "stay (movement)" is judged including those that are likely to be heading for "stay (movement)", it is possible to make an appropriate judgment according to the actual situation. Furthermore, since it is only necessary to detect the observation point Z to judge movement or stay, and an image that can identify an individual such as a face image is not required, privacy is also protected. Furthermore, by setting the "ratio" to less than 100%, erroneous judgments are suppressed even in cases where the detection of the observation point Z fails.

[0060] Furthermore, in the movement / stay determination device 1 according to this embodiment, when a set of multiple time-series images X is taken, the observation points Z detected from each of the multiple sets, in which the time taken by the image-taking means overlaps with the previous or subsequent set by a predetermined amount, are obtained, and for each set, the movement or stay of the moving object Y is sequentially determined based on the proportion of observation points Z that are included within a predetermined range R centered on the observation point Z taken in the predetermined image.

[0061] According to such a configuration, since the determination of movement or stay is performed sequentially for the changing reference point S, it becomes possible to perform a more appropriate determination.

[0062] Moreover, the movement / stay determination device 1 according to this embodiment determines whether or not to notify the movement or stay of the active object Y based on the proportion of the plurality of observation points Z determined for each set.

[0063] According to such a configuration, it is possible to notify "movement" or "retention" in a situation where it is highly necessary, such as when a person has "remained" for a predetermined period of time or more.

[0064] Moreover, the movement / stay determination device 1 according to this embodiment includes a setting unit 6 for setting the predetermined range R.

[0065] According to such a configuration, by setting the predetermined range R according to the purpose, it becomes possible to appropriately judge "movement" or "stay."

[0066] Moreover, in the movement / stay determination device 1 according to this embodiment, the observation point Z is a part on the center line of the body of the moving object Y that is detected.

[0067] With this configuration, it is possible to prevent the movement of the limbs from being determined as "movement" unless the entire body is moving.

[0068] The movement / stay determination device of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0069] For example, in the above embodiment, the "proportion of observation points Z within the specified range R" is described using the "number of observation points Z within the specified range R" as an example, but the "time that observation points Z stayed within the specified range R" may also be used. For example, if four observation points Z stay within the specified range R among time-series images X captured at one-second intervals, this means that they "stayed for 4 seconds out of 5 seconds." In this case, the "time that observation points Z stayed within the specified range R" does not need to be continuous, and can be considered as the total time.

[0070] In addition, in the above embodiment, the observation point Z of the first image X in each set is set as the reference point S, but the observation point Z of an image X other than the first may be set as the reference point S. However, in order to fulfill the look-ahead function, it is preferable to set the observation point Z of an image X other than the last one as the reference point S.

[0071] In the above embodiment, the time-series images X included in each set are shifted by one frame, but they may be shifted by multiple frames. The time-series images X used for the determination do not necessarily have to be consecutive frames in the captured video, and images acquired every predetermined number of frames may be used as the time-series images X, and this case is also included in the "time-series images" of the present invention.

[0072] In the above embodiment, only one circumference is used as the predetermined range R, but multiple concentric circumferences may be provided, and the inner circumference may be used for detailed judgment and the outer circumference may be used for simplified judgment. In this case, the shape of the predetermined range R is not limited to the circumference.

[0073] In addition, in the above embodiment, the determination of "movement" or "stay" is performed by focusing on one observation point Z ("neck"), but the determination of "movement" or "stay" may also be performed by focusing on another observation point Z ("waist") simultaneously in parallel. In this case, for example, a configuration may be considered in which, when a predetermined range R based on the "neck" and a predetermined range R based on the "waist" overlap, it is determined that the "determination is correct."

[0074] Furthermore, in the above embodiment, a joint (in the above embodiment, the “neck” joint) is detected as the observation point Z, but a part other than a joint may be detected as the observation point Z.

[0075] In addition, in the above embodiment, a judgment of “movement” or “stay” was made, but it is also possible to determine the “movement trajectory” and “movement pattern” of the moving object Y based on the detected observation point Z.

[0076] The notification may be made in combination with other conditions. For example, it is possible to limit the notification to "standing people" and notify "movement" and "staying". In this case, it is possible to identify "standing people" by referring to the "joint identification criteria" and the "behavior identification criteria" stored in the storage unit 2.

[0077] Furthermore, in the above embodiment, the memory unit 2 and the detection unit 3 are included in the movement / stay determination device 1, but as long as it is possible to acquire the observation point Z, they do not have to be included in the movement / stay determination device 1. For example, a configuration is conceivable in which the detection unit 3 provided separately from the movement / stay determination device 1 detects the observation point Z by referring to the memory unit 2, and the acquisition unit 4 acquires the detected observation point Z through communication.

[0078] Furthermore, the movement / stay determination according to this embodiment may be performed in real time, or may be performed later using recorded time-series images X depending on the purpose.

[0079] In the above embodiment, a human being is used as an example of the moving body Y, but it is also possible to use the present invention for animals or robots. For example, in the case of animals, it is possible to quickly deal with cases where they start fighting.

[0080] The present invention can also be applied to a program corresponding to the processing performed by the movement / stay determination device 1, or a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is conceivable as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0081] 1 Movement / stagnation determination device 2 Storage section 3. Detection section 4 Acquisition part 5 Judgment section 6 Setting section 7. Notification Department R Predetermined range S reference point X Time series images Y Action Body Z observation point

Claims

1. an acquisition unit that acquires, as an observation point, a predetermined part of the body of a target action object detected from a plurality of time-series images in which the target action object is captured by an image capturing means fixed at a predetermined position; a determination unit that determines whether the moving object is moving or stationary based on a ratio of observation points included within a predetermined range centered on one observation point serving as a reference point among observation points acquired in each time-series image; A movement / stay determination device comprising:

2. The acquisition unit acquires the observation points detected from a plurality of sets of the time-series images, each of which is taken as one set, such that the time of the images captured by the image capture means overlaps with a previous or subsequent set by a predetermined time, The movement / stay determination device described in claim 1, characterized in that the determination unit sequentially determines whether the moving object is moving or staying based on the proportion of observation points that fall within the specified range centered on the observation point acquired in a specified image, for each set.

3. 3. The movement / stay determination device according to claim 2, wherein the determination unit determines whether or not to notify the movement or stay of the action object based on the results of the determinations made in sequence for each set.

4. 4. The movement / stay determination device according to claim 1, further comprising a setting unit for setting the predetermined range.

5. 5. The movement / stay determination device according to claim 1, wherein the observation point is a detected part on the center line of the body of the moving object.

6. A computer comprising: A step of acquiring, as an observation point, a predetermined part of the body of a target action object detected from a plurality of time-series images in which the target action object is captured by an image capturing means fixed at a predetermined position; A step of determining whether the moving object is moving or stationary based on a ratio of observation points included within a predetermined range centered on one observation point serving as a reference point among observation points acquired in each time-series image; A program for determining whether movement or retention is required.

7. In the acquiring step, when the plurality of time-series images are regarded as one set, the observation points detected from each of the plurality of sets are acquired such that the time of the images captured by the image capturing means overlaps with the previous or subsequent set by a predetermined time, The movement / stay determination program described in claim 6, characterized in that in the determination step, for each set, the movement or stay of the action object is determined sequentially based on the proportion of observation points that fall within the specified range centered on the observation point acquired in a specified image.

8. 8. The program according to claim 7, wherein in the determining step, it is determined whether or not to notify the movement or stay of the action object based on the results of the determinations made in sequence for each set.

9. 9. The movement / stay determination program according to claim 6, wherein the observation point is a detected part on the center line of the body of the moving object.

Images