Data Transfer System

Abstract

A data transfer system is provided that can appropriately confirm attention behavior detected in surveillance video while reducing data transfer costs and the like. [Solution] A data transfer system 1 detects feature points of a target action object Z captured in an original video Y, and determines whether or not a first attention behavior or the second attention behavior has been performed based on the detected feature points. If it is determined that the first attention behavior has been performed, a portion of the original video Y corresponding to the first attention behavior is transmitted as a short clip video to a user terminal B via a cloud A, and if it is determined that the second attention behavior has been performed, a push notification is transmitted to the user terminal B via the cloud A without transmitting the short clip video.

Description

[Technical field]

[0001] The present invention relates to a data transfer system capable of appropriately confirming attention behavior detected in surveillance video while reducing data transfer costs and the like. [Background technology]

[0002] Conventionally, there has been known a technique for extracting a subject that has performed a behavior different from a "normal behavior" from a time-series image when the behavior of the subject captured in the time-series image is different from a predetermined "normal behavior" (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6525179 Summary of the Invention [Problem to be solved by the invention]

[0004] By using the above technology, a configuration is conceivable in which, only when abnormal behavior is detected in footage captured by a surveillance camera or the like, the footage of the abnormal behavior is sent to a user, such as an administrator, to prompt the user to check it.

[0005] Nowadays, data management is often performed on the cloud due to its convenience, and when the above configuration is performed using the cloud, it is possible to transfer all video footage in which abnormal behavior is detected to the cloud and allow users to view the footage on the cloud. However, transferring footage on the cloud incurs costs according to the amount of data transferred, so transferring all footage in which abnormal behavior is detected would be very costly.

[0006] Therefore, an object of the present invention is to provide a data transfer system that is capable of appropriately checking attention behavior detected in surveillance video while reducing data transfer costs and the like. [Means for solving the problem]

[0007] The present invention provides a data transfer system comprising: an acquisition unit that acquires an original video captured by a shooting means; a video memory unit that stores the original video; an attention behavior memory unit that stores a first attention behavior that requires a high degree of attention and a second attention behavior that requires a low degree of attention; a detection unit that detects characteristic points of a target behavior captured in the original video; a judgment unit that judges whether the first attention behavior or the second attention behavior has been performed based on the detected characteristic points; and a transmission unit that, when it is determined that the first attention behavior has been performed, transmits a portion of the original video corresponding to the first attention behavior as a short clip video to a user terminal via a cloud, and, when it is determined that the second attention behavior has been performed, transmits a push notification to the user terminal via the cloud without transmitting the short clip video.

[0008] According to such a configuration, when it is determined that a first attention behavior has been performed, a portion of the original video corresponding to the first attention behavior is transmitted as a short clip video to the user terminal via the cloud, and when it is determined that a second attention behavior has been performed, a push notification is sent to the user terminal via the cloud without transmitting the short clip video, and the user terminal can access the original video stored in the video storage unit using a P2P communication method.

[0009] In another aspect of the present invention, a data transfer system is provided that includes an acquisition unit that acquires an original video captured by a shooting means, a video memory unit that stores the original video, a focus behavior memory unit that stores a first focus behavior that requires a high degree of confirmation and a second focus behavior that requires a low degree of confirmation, a detection unit that detects feature points of a target behavior body captured in the original video, a pseudo video generation unit that can generate a pseudo behavior video in which a pseudo behavior body is caused to displace the detected feature points, a determination unit that determines whether the first focus behavior or the second focus behavior has been performed based on the detected feature points, and a transmission unit that, when it is determined that the first focus behavior has been performed, transmits the pseudo behavior video of a portion corresponding to the first focus behavior as a short clip video to a user terminal via a cloud, and, when it is determined that the second focus behavior has been performed, transmits a push notification to the user terminal via the cloud without transmitting the short clip video.

[0010] In another aspect of the present invention, there is provided a data transfer program and a data transfer method corresponding to the above data transfer system. Effect of the Invention

[0011] According to the data transfer system of the present invention, it is possible to appropriately confirm attention behavior detected in surveillance video while reducing data transfer costs and the like. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is an explanatory diagram of a data transfer system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is an explanatory diagram of an original video according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a block diagram of a data transfer system according to an embodiment of the present invention; [Figure 4] Flowchart of a data transfer system according to an embodiment of the present invention [Diagram 5] FIG. 1 is a block diagram of a data transfer system according to a modified example of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A data transfer system 1 according to an embodiment of the present invention will now be described with reference to FIGS.

[0014] As shown in Figures 1 and 2, the data transfer system 1 estimates the behavior of a target behavior object Z captured in an original image Y (in Figure 2, frames constituting the image) captured by a capture means X, and if any noteworthy behavior such as abnormal behavior is observed, notifies the user terminal B of this via the cloud A.

[0015] In this embodiment, the data transfer system 1 is provided integrally with the image capturing means X.

[0016] The user terminal B may be a smartphone, a PC, or the like.

[0017] In addition, in this embodiment, a human being is used as the target action object Z, and for ease of understanding, the target action object Z is simply displayed using only a skeleton.

[0018] As shown in FIG. 3, the data transfer system 1 includes an acquisition unit 2, a video storage unit 3, an attention behavior storage unit 4, a detection unit 5, a determination unit 6, a transmission unit 7, and a permission unit 8.

[0019] The acquisition unit 2 acquires an original video Y captured by an image capture means X.

[0020] The video storage unit 3 stores the original video Y.

[0021] The attention behavior storage unit 4 stores a first attention behavior that requires a high degree of confirmation and a second attention behavior that requires a low degree of confirmation.

[0022] In this embodiment, the attention behavior storage unit 4 stores the displacement of the feature point of the action object when the action object performs the attention behavior. For example, when detecting a joint as the feature point, it is considered to store the displacement of a plurality of joints.

[0023] The first type of attention behavior may be, for example, "violence," "theft," or "falling," which requires a quick confirmation of the "degree of attention behavior" or the "appearance of the target behavior Z."

[0024] On the other hand, the second attention behavior may be, for example, "the movement of a person in a wheelchair" or "the movement of a person with a white cane," which does not require a quick confirmation of the "degree of the actual attention behavior" or "the appearance of the target behavior Z."

[0025] In addition, there are cases where the behavior itself is normal (walking), such as "entering a restricted area," but becomes abnormal when combined with a restricted area (whether or not the feature point is located within the restricted area), and such combinations also correspond to the "notable behavior" of the present invention.

[0026] Furthermore, as the first or second attentional behavior, a "premonitory behavior" such as shoplifting or suicide may be stored.

[0027] Furthermore, the attention behavior stored in the attention behavior memory unit 4 may be determined by learning. In this case, a sample video (plurality of sample time-series images) captured by a photographing means X installed to capture a predetermined range is acquired, and one or more "normal behaviors" within a predetermined range are determined based on a large number of behaviors detected in the sample video. If the displacement of a feature point detected from the original video Y does not correspond to a "normal behavior", it is determined that an attention behavior ("abnormal behavior" or "unusual behavior") has been performed. In this case, it can be considered that the attention behavior memory unit 4 stores "behaviors not included in normal behaviors".

[0028] Although "behavior not included in normal behavior" may be set as either the first behavior of interest or the second behavior of interest, if one "wants to confirm what kind of behavior "not included in normal behavior" was," it may be set as the first behavior of interest. However, for example, "behavior not included in normal behavior" may also be classified as either the first behavior of interest or the second behavior of interest depending on the distance from "normal behavior" (such as how far the displacement of the detected feature point is from the displacement of "normal behavior").

[0029] Furthermore, even for noteworthy behaviors of the same classification, whether or not confirmation is necessary differs depending on the conditions of the monitoring location, etc., so it may be set appropriately according to the conditions, etc. For example, even for "entry into a restricted area," if the restricted area is a place where unintentional entry is common, it may be set as a first noteworthy behavior if the intrusion is for a predetermined time or longer, and as a second noteworthy behavior if the intrusion is for less than the predetermined time.

[0030] The detection unit 5 detects feature points of a target action object Z captured in an original video.

[0031] There are various possible feature points, but in this embodiment, an example will be described in which joints are detected as feature points.

[0032] When detecting joints as feature points, for example, the following method can be considered.

[0033] First, a "joint identification criterion" and a "behavior object identification criterion" are stored in a storage unit (which may be the attention behavior storage unit 4 or another storage unit).

[0034] The "joint identification standard" is for identifying multiple human joints, and indicates the shape, direction, size, etc. for identifying each joint.

[0035] The "Behavior Identification Standard" indicates the "basic postures" of various human variations ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" in a single human.

[0036] After detecting multiple joints that meet the above-mentioned "joint identification criteria," it is possible to identify the joints contained in each target action object Z by referring to the "action object identification criteria."

[0037] In this embodiment, feature points are detected collectively from multiple frames constituting the original video Y (after linking the chronological order), but they may also be detected individually for each frame.

[0038] The determination unit 6 determines whether or not the first attention behavior or the second attention behavior has been performed based on the detected feature points.

[0039] For example, when the displacement of the detected feature point matches the displacement of the feature point of the first attention behavior or the second attention behavior stored in the attention behavior memory unit 4 for a predetermined amount or more, it may be determined that "the attention behavior in question has been performed."

[0040] When it is determined that a first attention behavior has been performed, the transmission unit 7 transmits a portion of the original video Y corresponding to the first attention behavior as a short clip video to the user terminal B via the cloud A, and when it is determined that a second attention behavior has been performed, the transmission unit 7 transmits a push notification to the user terminal B via the cloud A without transmitting the short clip video.

[0041] As a result, the original video Y is transmitted to the user terminal B only in the case of the first attention behavior requiring a high degree of confirmation, and only a push notification is transmitted in the case of the second attention behavior requiring a low degree of confirmation, making it possible to significantly reduce the costs involved in data transfer from the data transfer system 1 to the cloud A, data transfer from the cloud A to the user terminal B, and data storage in the cloud A. In addition, the original video Y is not transmitted except in the case of the first attention behavior requiring a high degree of confirmation, reducing the risk of personal information leaks.

[0042] Incidentally, there may be cases in which the user would like to check actual video footage of the second attention behavior for which a push notification was sent or other behaviors.

[0043] Therefore, in this embodiment, the user terminal B is enabled to check the original video Y stored in the video storage unit 3 by using a P2P communication method.

[0044] In order for the user terminal B to be able to check the original video Y stored in the video storage unit 3 using the P2P communication method, the user terminal B needs to acquire the IP address of the data transfer system 1 in advance.

[0045] This allows the user to view the original video Y while eliminating the risk of personal information leaking by going through cloud A.

[0046] However, simply allowing "user terminal B to view the original video Y stored in the video storage unit 3 using a P2P communication method" may lead to a situation in which intrusion into the data transfer system 1 (especially the video storage unit 3) from various networks is permitted, which may result in a security risk.

[0047] Therefore, in this embodiment, when it is determined that the second attention behavior has been performed, the permission unit 8 permits the user terminal B to check the part of the original video Y that corresponds to the second attention behavior by using the P2P communication method. As a method of this permission, for example, the following two patterns are conceivable. (1) Pattern 1

[0048] First, when it is determined that the second attention behavior has been performed, a random key generation unit (not shown) generates a random key and transmits it to both the data transfer system 1 side and the user terminal B. In this embodiment, the random key generation unit is placed on cloud A, and is assumed to have acquired the IP address of the data transfer system 1 side and the IP address of the user terminal B in advance.

[0049] Then, when user terminal B requests access to the part of the original video Y that corresponds to the second attention behavior from the data transfer system 1 (the IP address of the data transfer system 1) along with a random key, if the random keys match, the permission unit 8 permits user terminal B to view the part of the original video Y that corresponds to the second attention behavior using the P2P communication method.

[0050] In detail, a configuration is conceivable in which the video storage unit 3 functions as the permission unit 8 and permits access from the user terminal B to the video storage unit 3 if the random keys match.

[0051] It is also possible to configure the transmission unit 7 to function as an authorization unit 8, and if the random keys match, to transmit a portion of the original video Y stored in the video storage unit 3 that corresponds to the second attention behavior to the user terminal B.

[0052] In addition, when P2P communication is once terminated, the above permission is also terminated, and when P2P communication is to be performed again, it is preferable to generate a random key again and require permission again. In addition, it is preferable to perform the above communication in its entirety in an encrypted form. (2) Pattern 2

[0053] First, when it is determined that the second attention behavior has been performed, a transmission unit (not shown) transmits the IP address of user terminal B to the data transfer system 1. In this embodiment, the transmission unit is arranged on cloud A, and is assumed to have acquired the IP address of the data transfer system 1 and the IP address of user terminal B in advance.

[0054] Then, when user terminal B requests access to the part of the original video Y that corresponds to the second attention behavior from data transfer system 1 (IP address of data transfer system 1), if the IP address requesting access matches the IP address previously transmitted, permission unit 8 permits user terminal B to view the part of the original video Y that corresponds to the second attention behavior using the P2P communication method.

[0055] In detail, a configuration is possible in which the video storage unit 3 functions as the permission unit 8 and permits access from the user terminal B to the video storage unit 3 if the IP addresses match.

[0056] In addition, a configuration is also possible in which the transmission unit 7 functions as the permission unit 8, and if the IP addresses match, the portion of the original video Y stored in the video memory unit 3 that corresponds to the second attention behavior is transmitted to the user terminal B.

[0057] In addition, when the P2P communication is once terminated, the above permission is also terminated, and when the P2P communication is performed again, it is preferable to transmit the IP address again and to require permission again. In addition, it is preferable to perform the above communication in its entirety in an encrypted form.

[0058] As a result, video other than that of the second attention behavior cannot be accessed via the P2P communication method, and access via the P2P communication method is limited, thereby reducing the risks of malware infection and network bandwidth congestion due to the use of the P2P communication method.

[0059] Next, the flow of data transfer by the data transfer system 1 of this embodiment will be described with reference to the flowchart of FIG.

[0060] First, when an original video Y is acquired (S1), feature points of a target action object Z captured in the original video Y are detected (S2).

[0061] Next, based on the feature points detected in S2, it is determined whether or not the first attention behavior or the second attention behavior has been performed (S3).

[0062] When it is determined that the first attention behavior has occurred (S3: first attention behavior), a short clip video is transmitted to the user terminal B via the cloud A (S4).

[0063] On the other hand, if it is determined that the second attention behavior has been performed (S3: second attention behavior), a push notification is transmitted to the user terminal B via the cloud A (S5).

[0064] As described above, in the data transfer system 1 according to this embodiment, when it is determined that a first attention behavior has been performed, a portion of the original video Y corresponding to the first attention behavior is transmitted as a short clip video to the user terminal B via the cloud A, and when it is determined that a second attention behavior has been performed, a push notification is sent to the user terminal B via the cloud A without transmitting the short clip video, and the user terminal B can access the original video Y stored in the video storage unit 3 using a P2P communication method.

[0065] According to such a configuration, the original video Y is transmitted to the user terminal B only in the case of the first attention behavior that requires a high degree of confirmation, and only a push notification is transmitted in the case of the second attention behavior that requires a low degree of confirmation, so that it is possible to significantly reduce the cost of data transfer from the data transfer system 1 to the cloud A, data transfer from the cloud A to the user terminal B, and data storage in the cloud A. In addition, the original video Y is not transmitted except for the first attention behavior that requires a high degree of confirmation, so the risk of personal information leaking is reduced.

[0066] Furthermore, in the data transfer system 1 according to this embodiment, the user terminal B can check the original video Y stored in the video storage unit 3 by using the P2P communication method.

[0067] According to this configuration, the user terminal B can view the original video Y stored in the video storage unit 3 using a P2P communication method, thereby enabling the user to view the original video Y while eliminating the risk of personal information leakage via cloud A.

[0068] Furthermore, in the data transfer system 1 according to this embodiment, when it is determined that the second attention behavior has been performed, the permission unit 8 permits the user terminal B to view the portion of the original video Y that corresponds to the second attention behavior using the P2P communication method.

[0069] According to this configuration, video other than that of the second attention behavior cannot be accessed via the P2P communication method, and access via the P2P communication method is limited, thereby reducing the risks of malware infection and network bandwidth congestion due to the use of the P2P communication method.

[0070] The data transfer system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0071] For example, in the above embodiment, when it is determined that the first attention behavior has occurred, the original video Y (short clip video) is transmitted to the user terminal B, but the configuration may also be such that a pseudo video is transmitted instead of the original video Y.

[0072] In this case, the data transfer system 1 may further include a pseudo image generation unit 9 capable of generating a pseudo behavior image in which the pseudo behavior body is caused to displace the detected feature points, as shown in FIG. 5, and the transmission unit 7 may be configured to, when it is determined that a first attention behavior has been performed, transmit the pseudo image of the portion corresponding to the first attention behavior as a short clip image to user terminal B via cloud A, and, when it is determined that a second attention behavior has been performed, transmit a push notification to user terminal B via cloud A without transmitting the short clip video.

[0073] The pseudo-acting entity may be one from which personal information cannot be identified (one from which identification is difficult), for example, one in which the acting entity Z is represented only by bones, or one in which it is replaced by a doll or the like.

[0074] According to this configuration, a pseudo image rather than the original image Y is transmitted to user terminal B via cloud A, thereby enabling the user to quickly understand "what kind of problem (to what extent) has occurred" through the pseudo image while reducing the risk of personal information (such as an image containing a face) being leaked.

[0075] In the above embodiment, the image capturing means X is provided integrally with the data transfer system 1, but this does not exclude the image capturing means X being a separate entity.

[0076] Furthermore, in the above embodiment, each member is provided integrally, but for example, this does not exclude some members being provided in different locations (such as cloud A).

[0077] For example, a configuration is conceivable in which the attention behavior memory unit 4 and the judgment unit 6 are provided on cloud A and the other components are provided on the edge, the feature points detected by the detection unit 5 are transmitted by the transmission unit 7 to cloud A, and the judgment unit 6 judges on cloud A whether or not attention behavior has been performed.

[0078] Then, if it is determined that the first attention behavior has been performed, a message to that effect is transmitted from cloud A to the edge, and then the short clip video is transmitted from transmission unit 7 to user terminal B via cloud A.

[0079] In addition, in the above embodiment, an example has been described in which data regarding an image captured by one imaging means X is transferred to one user terminal B, but it goes without saying that data regarding images captured by multiple imaging means X may be transferred to multiple user terminals B.

[0080] However, when the data transfer system 1 is used for multiple users, it is necessary to link the image capturing means X and the user terminal B so that the short clip video is not transmitted to another user (a user belonging to another organization) or permission to access the original video using the P2P communication method is not granted.

[0081] The present invention can also be applied to a program and method corresponding to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is conceivable as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0082] 1 Data transfer system 2 Acquisition part 3. Video storage section 4. Attention behavior memory 5. Detection section 6 Judgment section 7. Transmitter 8 Permit Department 9. Pseudo image generation section A Cloud B User terminal X Shooting method Y original video Z Target Action

Claims

1. An acquisition unit that acquires an original video captured by an image capture means; a video storage unit for storing the original video; an attention behavior storage unit that stores a first attention behavior that requires a high degree of confirmation and a second attention behavior that requires a low degree of confirmation; A detection unit that detects feature points of a target action object captured in the original video; a determination unit that determines whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; a transmission unit that, when it is determined that the first attention behavior has been performed, transmits a portion of the original video corresponding to the first attention behavior as a short clip video to a user terminal via a cloud, and, when it is determined that the second attention behavior has been performed, transmits a push notification to the user terminal via the cloud without transmitting the short clip video; Equipped with The image storage unit is provided at an edge, A data transfer system, wherein the user terminal is capable of checking the original video stored in the video storage unit by using a P2P communication method.

2. The data transfer system described in claim 1, further comprising a permission unit that, when it is determined that the second attention behavior has occurred, permits the user terminal to view the portion of the original video corresponding to the second attention behavior using a P2P communication method.

3. An acquisition unit that acquires an original video captured by an image capture means; a video storage unit for storing the original video; an attention behavior storage unit that stores a first attention behavior that requires a high degree of confirmation and a second attention behavior that requires a low degree of confirmation; A detection unit that detects feature points of a target action object captured in the original video; a pseudo image generating unit capable of generating a pseudo behavior image by making a pseudo behavior object perform the displacement of the detected feature points; a determination unit that determines whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; a transmission unit that, when it is determined that the first attention behavior has been performed, transmits the simulated behavior video of a portion corresponding to the first attention behavior as a short clip video to a user terminal via a cloud, and, when it is determined that the second attention behavior has been performed, transmits a push notification to the user terminal via the cloud without transmitting the short clip video; Equipped with The image storage unit is provided at an edge, A data transfer system, wherein the user terminal is capable of checking the original video stored in the video storage unit by using a P2P communication method.

4. A program executed by a computer storing a first attention behavior having a high degree of confirmation required and a second attention behavior having a low degree of confirmation required, A step of acquiring an original image captured by a capturing means; storing the original video; detecting feature points of a target action object captured in the original video; determining whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; a step of transmitting a portion of the original video corresponding to the first attention behavior as a short clip video to a user terminal via a cloud when it is determined that the first attention behavior has been performed, and transmitting a push notification to the user terminal via the cloud without transmitting the short clip video when it is determined that the second attention behavior has been performed; Equipped with The original image is stored in an image storage unit provided in the edge, A data transfer program, characterized in that the user terminal is capable of checking the original video stored in the video storage unit by using a P2P communication method.

5. A method executed on a computer having stored therein a first attention behavior that is more likely to require confirmation and a second attention behavior that is less likely to require confirmation, the method comprising: A step of acquiring an original image captured by a capturing means; storing the original video; detecting feature points of a target action object captured in the original video; determining whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; a step of transmitting a portion of the original video corresponding to the first attention behavior as a short clip video to a user terminal via a cloud when it is determined that the first attention behavior has been performed, and transmitting a push notification to the user terminal via the cloud without transmitting the short clip video when it is determined that the second attention behavior has been performed; Equipped with The original image is stored in an image storage unit provided in the edge, A data transfer method, characterized in that the user terminal can check the original video stored in the video storage unit by using a P2P communication method.

6. A program executed by a computer storing a first attention behavior having a high degree of confirmation required and a second attention behavior having a low degree of confirmation required, A step of acquiring an original image captured by a capturing means; storing the original video; detecting feature points of a target action object captured in the original video; generating a pseudo-behavior video by causing a pseudo-behavior object to perform the displacement of the detected feature points; determining whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; transmitting the simulated behavior video of a portion corresponding to the first attention behavior as a short clip video to a user terminal via a cloud when it is determined that the first attention behavior has been performed, and transmitting a push notification to the user terminal via the cloud without transmitting the short clip video when it is determined that the second attention behavior has been performed; Equipped with The original image is stored in an image storage unit provided in the edge, A data transfer program, characterized in that the user terminal is capable of checking the original video stored in the video storage unit by using a P2P communication method.

7. A method executed on a computer having stored therein a first attention behavior that is more likely to require confirmation and a second attention behavior that is less likely to require confirmation, the method comprising: A step of acquiring an original image captured by a capturing means; storing the original video; detecting feature points of a target action object captured in the original video; generating a pseudo-behavior video by causing a pseudo-behavior object to perform the displacement of the detected feature points; determining whether the first attention behavior or the second attention behavior has been performed based on the detected feature points; transmitting the simulated behavior video of a portion corresponding to the first attention behavior as a short clip video to a user terminal via a cloud when it is determined that the first attention behavior has been performed, and transmitting a push notification to the user terminal via the cloud without transmitting the short clip video when it is determined that the second attention behavior has been performed; Equipped with The original image is stored in an image storage unit provided in the edge, A data transfer method, characterized in that the user terminal can check the original video stored in the video storage unit by using a P2P communication method.

Images