Data transfer system

Abstract

To provide a data transfer system capable of determining an action of interest that appears in a video, on a cloud, while reducing data transfer cost.SOLUTION: In a data transfer system 1, a conversion unit 4 converts original data related to a feature point detected from a target action body Z appearing in a target video Y into low-bit rate data to be transmitted to a cloud A. A determination unit 6 provided on the cloud A performs primary determination as to whether an action of interest has been performed, based on variation of the feature point included in the low-bit rate data. When the primary determination determines that the action of interest has been performed, the conversion unit 4 transmits original data corresponding to the low-bit rate data for which the action of interest has been determined or high-bit rate data to the cloud A. The determination unit 6 performs secondary determination as to whether the action of interest has been performed, based on variation of the feature point included in the original data or the high-bit rate data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a data transfer system capable of determining attention behavior captured in a video on the cloud while reducing data transfer costs and the like. [Background technology]

[0002] Conventionally, an animal behavior estimation system has been known in which an animal behavior estimation device provided on the cloud side estimates the type of behavior of an animal to be estimated based on time-series feature point position information of the animal to be estimated received from an animal behavior estimation support device provided on the edge side (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-144631 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above system, the animal behavior estimation support device installed on the edge side discards unnecessary data for estimating animal behavior, compresses only valid data to a minimum, and transfers it to the animal behavior estimation device installed on the cloud side.

[0005] The present invention aims to provide a data transfer system that can determine attention behavior captured in video on the cloud while reducing data transfer costs, etc., using a method different from that described above. [Means for solving the problem]

[0006] The present invention includes an acquisition unit that acquires a target image captured by a capture means, a detection unit that detects feature points of a target action object captured in the target image, a conversion unit that converts original data relating to the detected feature points into low bit rate data and transmits the data to a cloud, a storage unit provided on the cloud that stores displacements of the feature points of the action object when the action object performs an action of interest, a determination unit provided on the cloud that makes a primary determination as to whether or not the action of interest has been performed based on the displacements of the stored feature points and the displacements of the feature points included in the low bit rate data, and a transmission unit that can transmit the result of the determination to a user terminal that can communicate with the cloud when it is determined that the action of interest has been performed. The present invention provides a data transfer system in which, when it is determined in the primary determination that attention behavior has been performed, the determination unit transmits a detection signal indicating that fact to the conversion unit, and the conversion unit, in response to the detection signal, re-transmits original data corresponding to the low bit rate data for which it has been determined that the attention behavior has been performed, or converts the low bit rate data for which it has been determined that the attention behavior has been performed into high bit rate data and re-transmits it to the cloud, and the determination unit makes a secondary determination as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of feature points included in the original data or the high bit rate data.

[0007] With this configuration, it is possible to significantly reduce the costs of transferring data from the data transfer system to the cloud, transferring data from the cloud to the user terminal, and storing data in the cloud, while ensuring the reliability of the judgment.

[0008] In another aspect of the present invention, there is provided a data transfer program and a data transfer method corresponding to the above data transfer system. Effect of the Invention

[0009] According to the data transfer system of the present invention, it is possible to determine attention behavior captured in a video on the cloud while reducing data transfer costs and the like. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram of a data transfer system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is an explanatory diagram of a target image according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a block diagram of a data transfer system according to an embodiment of the present invention; [Figure 4] Flowchart of a data transfer system according to an embodiment of the present invention DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A data transfer system 1 according to a first embodiment of the present invention will now be described with reference to FIGS. 1 to 4. FIG.

[0012] As shown in Figs. 1 and 2, the data transfer system 1 transmits data on feature points of a target action object Z captured in a target image Y (in Fig. 2, frames constituting the image) captured by a capture means X to a cloud A via network communication, and judges whether or not the target action has been performed on the cloud A.

[0013] In this embodiment, a human being is used as the target action object Z, and for ease of understanding, the target action object Z is simply displayed using only a skeleton.

[0014] 3, the data transfer system 1 includes an acquisition unit 2, a detection unit 3, a conversion unit 4, a storage unit 5, a judgment unit 6, a transmission unit 7, a learning side acquisition unit 8, a learning side detection unit 9, and a determination unit 10. In this embodiment, the data transfer system 1 is provided integrally with the imaging means X. In this embodiment, the acquisition unit 2, the detection unit 3, the conversion unit 4, the transmission unit 7, the learning side acquisition unit 8, the learning side detection unit 9, and the determination unit 10 are provided on the edge side, and the storage unit 5 and the judgment unit 6 are provided on the cloud A side.

[0015] The acquisition unit 2 acquires a target video Y captured by an imaging means X. In this embodiment, it is assumed that the target video Y is captured by the imaging means X at a predetermined bit rate.

[0016] The detection unit 3 detects feature points of a target action object Z captured in a target image Y.

[0017] There are various possible feature points, but in this embodiment, an example will be described in which joints are detected as feature points.

[0018] When detecting joints as feature points, for example, the following method can be considered.

[0019] First, a "joint identification standard" and a "behavior object identification standard" are stored in a storage unit (which may be the storage unit 5 or another storage unit).

[0020] The "joint identification standard" is for identifying multiple human joints, and indicates the shape, direction, size, etc. for identifying each joint.

[0021] The "Behavior Identification Standard" indicates the "basic postures" of various human variations ("walking", "standing upright", etc.), the "range of motion of each joint", and the "distance between each joint" in a single human.

[0022] After detecting multiple joints that meet the above-mentioned "joint identification criteria," it is possible to identify the joints contained in each target action object Z by referring to the "action object identification criteria."

[0023] In this embodiment, feature points are detected collectively from multiple frames constituting the target video Y (after linking the chronological order), but they may also be detected individually for each frame.

[0024] The conversion unit 4 converts the original data relating to the detected feature points into low bit rate data and transmits it to the cloud A.

[0025] For example, it is conceivable to convert the original data into low bit rate data by lowering the frame rate, but the original data may also be converted into low bit rate data by "reducing the image quality," "compressing the data," "deleting unnecessary data," etc., and these are also included in the "bit rate conversion" of this invention.

[0026] The storage unit 5 is provided on the cloud A and stores the displacements of the feature points of the action object when the action object performs the action of interest. When detecting a joint as the feature point, it is considered to store the displacements of a plurality of joints.

[0027] Examples of the behavior of interest include falling, punching, kicking, and the like, and a plurality of behaviors may be stored without being limited to one. Also, as the behavior of interest, a "premonitory behavior" such as shoplifting may be stored. Moreover, the data transfer system 1 according to this embodiment does not exclude storing in the storage unit 5 the displacement of the feature point information when behavior other than the behavior of interest (walking, stopping, etc.) occurs.

[0028] The determination unit 6 is provided on the cloud A and performs a primary determination as to whether or not attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points included in the low bit rate data.

[0029] For example, when the displacement of the detected feature point matches the displacement of the feature point of the attention behavior for a predetermined period or more, it may be determined that "the attention behavior has been performed."

[0030] The transmission unit 7 can transmit the result of the determination to the user terminal B that can communicate with the cloud A when it is determined that the attention behavior has been performed.

[0031] User terminal B may be a smartphone, a PC, or the like, and the user can learn from the determination result transmitted to user terminal B that noteworthy behavior, such as abnormal behavior, has occurred.

[0032] In order to enable the user to check the actual situation of the attention behavior, etc., the data transfer system 1 or the image capture means X may buffer the video corresponding to the attention behavior, and the user terminal B may be able to access the video.

[0033] Here, in this embodiment, when the judgment unit 6 judges in the primary judgment that attention behavior has been performed, it transmits a detection signal indicating that to the conversion unit 4, and in response to the detection signal, the conversion unit 4 re-transmits original data corresponding to the low bit-rate data for which it has been determined that attention behavior has been performed to cloud A, or converts the low bit-rate data for which it has been determined that attention behavior has been performed into high bit-rate data and re-transmits it to cloud A.

[0034] Regarding the decision of whether to transmit original data or high bit rate data, if real-time performance is important, original data that does not require conversion time to high bit rate data can be transmitted, whereas if cost reduction is important, high bit rate data (with a lower bit rate than the original data) can be transmitted.

[0035] Then, the determination unit 6 performs a secondary determination as to whether or not attention behavior has been performed, based on the displacement of the stored feature points and the displacement of the feature points included in the original data or the high bit rate data.

[0036] With this configuration, a primary determination is first made as to whether or not attention behavior has occurred based on the low bit rate data, and if it is determined that attention behavior has occurred, a secondary determination is made as to whether or not attention behavior has occurred based on the original data or the high bit rate data.This makes it possible to significantly reduce the costs associated with data transfer from the data transfer system 1 to cloud A, data transfer from cloud A to user terminal B, and data storage in cloud A while ensuring the reliability of the determination.

[0037] In addition, in order to prevent different or overlapping judgment results from being sent to user terminal B, it is preferable that when it is determined in the primary judgment that attention behavior has been performed, the transmission unit 7 transmits the judgment result to user terminal B when it is also determined in the secondary judgment that attention behavior has been performed.

[0038] Incidentally, in a configuration in which a primary determination is made based on the displacement of feature points contained in low bit rate data, there may be cases in which the low bit rate causes a failure to determine attention behavior at the time of primary determination.

[0039] Therefore, in this embodiment, a learning side acquisition unit 8, a learning side detection unit 9, and a determination unit 10 are further provided, and these are used to prevent failure to determine attention behavior at the time of primary determination due to the influence of a low bit rate.

[0040] The learning side acquisition unit 8 acquires sample videos (a plurality of sample time-series images) captured by imaging means X that is installed so as to capture a predetermined range.

[0041] The learning side detection unit 9 detects the behavior of the sample behaving object captured in the sample video.

[0042] The behavior of the sample behavior body can be detected by storing the displacement of feature points (such as the movement of each joint) when the sample behavior body performs a specified behavior in a memory unit (which can be memory unit 5 or another memory unit), detecting the feature points in the same manner as detection unit 3, and detecting that "the specified behavior has been performed" if the displacement of the detected feature point information matches the displacement of the stored feature points by a specified amount or more.

[0043] In this embodiment, the learning-side acquisition unit 8 and the learning-side detection unit 9 are respectively shared by the acquisition unit 2 and the detection unit 3, but these may be provided separately.

[0044] The determination unit 10 determines one or more “normal behaviors” within a predetermined range based on the multiple behaviors detected by the learning-side detection unit 9. In this embodiment, the determined “normal behaviors” are stored in the storage unit 5 as attention behaviors.

[0045] "Normal behavior" can be determined based on various criteria, but for example, it is possible to determine that behavior that has a predetermined (threshold) or higher proportion among all detected behaviors is "normal behavior."

[0046] Then, in the primary determination, the determination unit 6 determines that attention behavior has been performed when the displacement of the feature points included in the low bit rate data does not correspond to "normal behavior."

[0047] As a result, rather than determining specific behaviors such as falling, punching, kicking, etc., the initial determination is made within a broad range of what does not correspond to "normal behavior," thereby preventing the failure to determine noteworthy behaviors at the time of the initial determination due to the influence of a low bit rate.

[0048] Of course, in addition to "normal behaviors," specific behaviors such as falling, punching, kicking, etc. may be stored in the memory unit 5 as notable behaviors, and the occurrence of these specific notable behaviors may be determined in parallel.

[0049] Next, the flow of data transfer according to this embodiment will be described with reference to the flowchart of FIG.

[0050] First, when a target image Y is acquired (S1), feature points of a target action object Z captured in the target image Y are detected (S2).

[0051] Next, the original data regarding the detected feature points is converted into low bit rate data and transmitted to cloud A (S3).

[0052] Next, in the judgment unit 6 on cloud A, a primary judgment is made as to whether or not attention behavior has been performed based on the displacements of the feature points stored in the memory unit 5 and the displacements of the feature points included in the low bit rate data transmitted in S3 (S4).

[0053] If it is determined that attention behavior has occurred (S4: YES), a detection signal is sent from cloud A to conversion unit 4 (S5), and the original data or high bit rate data is re-sent to cloud A in conversion unit 4 (S6).

[0054] Then, in the judgment unit 6 on cloud A, a secondary judgment is made as to whether or not attention behavior has been performed based on the displacement of the feature points stored in the memory unit 5 and the displacement of the feature points included in the original data or the high bit rate data transmitted in S6 (S7), and if it is judged that attention behavior has been performed (S7: YES), the judgment result is transmitted to the user terminal B (S8).

[0055] As described above, in the data transfer system 1 according to this embodiment, a primary determination is first made as to whether or not attention behavior has occurred based on low bit rate data, and if it is determined that attention behavior has occurred, a secondary determination is made as to whether or not attention behavior has occurred based on the original data or high bit rate data.

[0056] With this configuration, it is possible to significantly reduce the costs of transferring data from the data transfer system 1 to cloud A, transferring data from cloud A to user terminal B, and storing data in cloud A, while ensuring the reliability of the judgment.

[0057] Furthermore, in the data transfer system 1 according to this embodiment, if the feature points contained in the low bit rate data do not correspond to "normal behavior" in the primary determination, it is determined that an attention behavior has been performed.

[0058] With this configuration, rather than determining specific behaviors such as falling, punching, kicking, etc., the initial determination is made within a broad range of "not equivalent to 'normal behavior,'" thereby preventing the failure to determine noteworthy behaviors at the time of the initial determination due to the effect of a low bit rate.

[0059] In addition, in the data transfer system 1 according to this embodiment, if it is determined in the primary determination that attention behavior has occurred, and if it is also determined in the secondary determination that attention behavior has occurred, the result of the determination is transmitted to the user terminal B.

[0060] According to this configuration, different or overlapping determination results are prevented from being transmitted to user terminal B.

[0061] The data transfer system of the present invention is not limited to the above-described embodiment, and various modifications and improvements are possible within the scope of the claims.

[0062] For example, in the above embodiment, the image capturing means X is provided integrally with the data transfer system 1, but this does not exclude the image capturing means X being a separate entity.

[0063] The present invention is also applicable to a program and method (executed in cooperation with a computer and a cloud) that corresponds to the processing performed by each component as a controller, and to a recording medium storing the program. In the case of a recording medium, the program is installed in a computer or the like. Here, the recording medium storing the program may be a non-transient recording medium. A CD-ROM or the like is conceivable as a non-transient recording medium, but is not limited thereto. [Explanation of symbols]

[0064] 1 Data Transfer System 2 Acquisition part 3. Detection section 4. Conversion section 5 Storage section 6 Judgment section 7. Transmitter 8 Learning side acquisition unit 9 Learning side detector 10 Decision Section A Cloud B User terminal X Shooting method Y Target video Z Target Action

Claims

1. An acquisition unit that acquires a target image captured by an imaging means; A detection unit that detects feature points of a target action object captured in the target image; a conversion unit that converts original data related to the detected feature points into low bit rate data and transmits the data to a cloud; A storage unit provided on the cloud that stores displacement of a feature point of a behavior object when the behavior object performs a behavior of interest; a determination unit that is provided on the cloud and that performs a primary determination as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points included in the low bit rate data; a transmission unit capable of transmitting a result of the determination to a user terminal capable of communicating with the cloud when it is determined that the attention behavior has been performed; Equipped with When the determination unit determines that an attention behavior has been performed in the primary determination, the determination unit transmits a detection signal indicating that the attention behavior has been performed to the conversion unit; the conversion unit, in response to the detection signal, transmits original data corresponding to the low bit rate data in which it is determined that the attention behavior has been performed to the cloud again, or converts the low bit rate data in which it is determined that the attention behavior has been performed to high bit rate data and transmits the high bit rate data to the cloud again; The data transfer system is characterized in that the determination unit performs a secondary determination of whether or not the attention behavior has occurred based on the displacement of the stored feature points and the displacement of feature points contained in the original data or the high bit rate data.

2. A learning side acquisition unit that acquires a sample video captured by the imaging means installed to capture a predetermined range; A learning side detection unit that detects the behavior of a sample behavior object captured in the sample video; A determination unit that determines one or more normal behaviors in the predetermined range based on the multiple behaviors detected by the learning side detection unit; Further comprising: The system for calling for attention to behavior described in claim 1, characterized in that the judgment unit judges that the attention behavior has been performed when, in the primary judgment, the displacement of feature points contained in the low bit rate data does not correspond to the normal behavior.

3. The system for calling for attention to behavior as described in claim 1, characterized in that when it is determined that the attention behavior has been performed in the first determination, and when it is determined that the attention behavior has also been performed in the second determination, the transmission unit transmits the result of the determination to the user terminal.

4. A program executed in cooperation with a computer and a cloud in which displacements of feature points of a behavioral object when the behavioral object performs a behavior of interest are stored, acquiring, on the computer, an image of a target captured by an image capturing means; Detecting, on the computer, feature points of a target action object captured in the target image; converting original data related to the detected feature points into low bit rate data on the computer and transmitting the data to the cloud; performing a primary determination on the cloud as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points included in the low bit rate data; When it is determined in the first determination that attention behavior has been performed, transmitting a detection signal indicating that to the computer on the cloud; a step of, on the computer, in response to the detection signal, transmitting original data corresponding to the low bit rate data for which it is determined that the attention behavior has been performed to the cloud again, or converting the low bit rate data for which it is determined that the attention behavior has been performed to high bit rate data and transmitting the converted data to the cloud again; performing a secondary determination on the cloud as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of feature points included in the original data or the high bit rate data; A data transfer program comprising:

5. A method executed in cooperation with a computer and a cloud in which displacements of feature points of a behavioral object when the behavioral object performs a behavior of interest are stored, comprising: acquiring, on the computer, an image of a target captured by an image capturing means; Detecting, on the computer, feature points of a target action object captured in the target image; converting original data related to the detected feature points into low bit rate data on the computer and transmitting the data to the cloud; performing a primary determination on the cloud as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of the feature points included in the low bit rate data; When it is determined in the first determination that attention behavior has been performed, transmitting a detection signal indicating that to the computer on the cloud; a step of, on the computer, in response to the detection signal, transmitting original data corresponding to the low bit rate data for which it is determined that the attention behavior has been performed to the cloud again, or converting the low bit rate data for which it is determined that the attention behavior has been performed to high bit rate data and transmitting the converted data to the cloud again; performing a secondary determination on the cloud as to whether or not the attention behavior has been performed based on the displacement of the stored feature points and the displacement of feature points included in the original data or the high bit rate data; A data transfer method comprising:

Images