Detection Method, Detection Device and Behavior Activity Detection System for Human Behavior Activities

Abstract

The present application provides a method for detecting human behavior activities, a detection device and a behavior activity detection system. The method includes: acquiring a plurality of consecutive frames of images to be detected, and performing human detection on each image to be detected to obtain a plurality of first target data groups, where each image to be detected corresponds to at least one first target data group, and the first target data group includes the position information of a first target point and the position information of a second target point, and the first target point and the second target point are on the target diagonal line; using the dense optical flow algorithm to calculate any two adjacent images to be detected to obtain a plurality of corresponding optical flow images, and one optical flow image is calculated from two adjacent images to be detected; determining the motion trajectory of the human body at least according to each optical flow image and the corresponding first target data groups, and determining the behavior activity of the human body at least according to the motion trajectory of the human body, thereby solving the problem in the prior art that it is difficult to accurately detect the behavior activities of the human body.

Description

Technical Field

[0001] This application relates to the field of detecting behavioral activities. Specifically, it relates to a method for detecting human behavioral activities, a detection device, a computer-readable storage medium, and a behavioral activity detection system. Background Art

[0002] The fall detection technology is mainly used to notify the user's contacts in a timely manner when someone falls in a home scenario, so that help and treatment can be provided to the fallen person in a timely manner.

[0003] There are mainly two types of existing fall detection algorithms: one is based on dedicated hardware, such as wearable sensors, depth image cameras, radars, etc.; the other is based on human key point detection, and then the key point coordinates are input into a classifier to determine whether there is a fall event.

[0004] The inventors have learned that the above-mentioned mainstream fall detection methods still have many problems. For example, wearable devices often require multiple sensors to collect different data, need to wear relatively complex devices, the sensor cost is relatively high, and there is also a problem of poor user experience. And the key point-based method has a high dependence on the position of the key points and does not fully utilize the motion information of each part of the human body.

[0005] Therefore, there is an urgent need for a method that can more accurately detect human behavioral activities.

[0006] The above information disclosed in the background art is only used to enhance the understanding of the background art of the technology described in this article. Therefore, the background art may contain certain information that is not prior art known in the country to those skilled in the art. Summary of the Invention

[0007] The main purpose of this application is to provide a method for detecting human behavioral activities, a detection device, a computer-readable storage medium, and a behavioral activity detection system, so as to solve the problem that it is difficult to more accurately detect human behavioral activities in the prior art.

[0008] According to one aspect of an embodiment of the present invention, a method for detecting human behavior activities is provided, including: obtaining a plurality of consecutive frames of images to be detected, and performing human detection on each of the images to be detected to obtain a plurality of first target data groups, where each image to be detected corresponds to at least one first target data group, and the first target data group includes the position information of a first target point and the position information of a second target point, the first target point and the second target point are on a target diagonal line, and the target diagonal line is a diagonal line of the smallest rectangle area including a human body in the detected image to be detected; calculating any two adjacent images to be detected by using a dense optical flow algorithm to obtain a plurality of corresponding optical flow images, and one optical flow image is obtained by calculating two adjacent images to be detected; determining the motion trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, and determining the behavior activity of the human body at least according to the motion trajectory of the human body.

[0009] Optionally, determining the motion trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups includes: in the (N + 1)-th frame of the optical flow image, marking the position corresponding to the first target data group of the (N + 1)-th frame of the image to be detected to obtain the marked (N + 1)-th frame of the optical flow image, where N ≥ 1; determining a corresponding second target data group in the N-th frame of the optical flow image according to the marked (N + 1)-th frame of the optical flow image; determining whether the (N + 1)-th frame of the optical flow image and the N-th frame of the optical flow image correspond to the same human body according to the second target data group corresponding to the N-th frame of the optical flow image and the first target data group corresponding to the N-th frame of the image to be detected; in the case of corresponding to the same human body, determining the motion trajectory of the human body according to the first target data group of the (N + 1)-th frame of the image to be detected and the first target data group of the N-th frame of the image to be detected.

[0010] Optionally, determining whether the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body according to the second target data group corresponding to the optical flow image of the N-th frame and the first target data group corresponding to the image to be detected of the N-th frame includes: determining a first predetermined area corresponding to the first target data group in the optical flow image of the N-th frame, and determining a second predetermined area corresponding to the second target data group in the optical flow image of the N-th frame; taking the intersection of the first predetermined area and the second predetermined area to obtain a first predetermined area, and taking the union of the first predetermined area and the second predetermined area to obtain a second predetermined area; calculating the ratio of the first predetermined area to the second predetermined area to obtain a target ratio; and determining that the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body when the target ratio is greater than or equal to a predetermined value.

[0011] Optionally, determining the behavior of the human body according to at least the movement trajectory of the human body includes: intercepting the corresponding image to be detected according to each first target data group to obtain a corresponding target image, where the target image is an image including one human body, and the first target data group and the target image are in one-to-one correspondence; intercepting the corresponding optical flow images according to the target images in each image to be detected corresponding to the movement trajectory of the human body in chronological order to obtain a plurality of optical flow sub-images; and classifying and judging the plurality of optical flow sub-images to determine the behavior of the human body.

[0012] Optionally, intercepting the corresponding optical flow images according to the target images in each image to be detected corresponding to the movement trajectory of the human body in chronological order to obtain a plurality of optical flow sub-images includes: combining the first target data group corresponding to the target image in the N-th image to be detected with the first target data group corresponding to the target image in the (N + 1)-th image to be detected to obtain a third target data group; determining the largest target area corresponding to the third target data group in the optical flow image of the (N + 1)-th frame, and intercepting the target area to obtain the optical flow sub-image.

[0013] Optionally, after determining the behavior of the human body according to at least the movement trajectory of the human body, the method further includes: determining whether the human body is in a dangerous state according to the behavior of the human body; and sending an alarm message to the intelligent terminal when it is determined that the human body is in the dangerous state.

[0014] Optionally, the behavior of the human body includes at least one of the following: standing, walking, sitting down, and falling.

[0015] According to another aspect of the embodiments of the present invention, there is also provided a detection device for human behavior activities, including: a detection unit, configured to obtain a plurality of consecutive frames of images to be detected, and perform human detection on each of the images to be detected, so as to obtain a plurality of first target data groups, each of the images to be detected corresponding to at least one first target data group, the first target data group including the position information of a first target point and the position information of a second target point, the first target point and the second target point being on a target diagonal line, the target diagonal line being a diagonal line of the smallest rectangular area including a human body in the detected image to be detected; a calculation unit, configured to perform calculation on any two adjacent images to be detected by using a dense optical flow algorithm, so as to obtain a plurality of corresponding optical flow images, one optical flow image being obtained by calculating two adjacent images to be detected; a first determination unit, configured to determine the movement trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, and determine the behavior activity of the human body at least according to the movement trajectory of the human body.

[0016] According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, the computer-readable storage medium including a stored program, wherein the program executes any one of the methods.

[0017] According to yet another aspect of the embodiments of the present invention, there is also provided a behavior activity detection system, including: an image acquisition device, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing any one of the methods.

[0018] In an embodiment of the present invention, in the method for detecting human behavior activities, first, a plurality of consecutive frames of images to be detected are acquired, and human detection is performed on each of the acquired images to be detected, obtaining a plurality of first target data groups. Among them, one image to be detected corresponds to at least one first target data group. Secondly, the dense optical flow algorithm is used to calculate any two adjacent images to be detected, obtaining a plurality of corresponding optical flow images. Finally, at least based on each of the optical flow images and the corresponding first target data groups, the movement trajectory of the human body is determined, and at least based on the movement trajectory of the human body, the behavior activities of the human body are determined. In this solution, human detection is performed on a plurality of consecutive frames of images to be detected, obtaining a plurality of first target data groups, and the dense optical flow algorithm is used to calculate any two adjacent frames of images to be detected, obtaining corresponding optical flow images. Then, based on each optical flow image and the corresponding first target data groups, the movement trajectory of the human body is determined. Finally, at least based on the movement trajectory of the human body, the behavior activities of the human body are determined. Compared with the movement information of the human body measured by a sensor in the prior art, in this solution, relatively sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized relatively fully. Furthermore, it ensures that the determined movement trajectory of the human body is relatively accurate. Then, at least based on the movement trajectory of the human body, the behavior activities of the human body are determined, which ensures that the behavior activities of the human body can be determined relatively accurately, thus solving the problem in the prior art that it is difficult to detect the behavior activities of the human body relatively accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0020] Figure 1 The schematic diagram of the method for detecting human behavior activities according to an embodiment of this application is shown;

[0021] Figure 2 The schematic diagram of the device for detecting human behavior activities according to an embodiment of this application is shown;

[0022] Figure 3 The flowchart of the method for detecting human behavior activities according to an embodiment of this application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail this application.

[0024] To enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances for the embodiments of this application described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0026] As described in the background art, it is difficult to accurately detect human behavior activities in the prior art. To solve the above problems, in a typical implementation manner of this application, a method for detecting human behavior activities, a detection device, a computer-readable storage medium, and a behavior activity detection system are provided.

[0027] According to an embodiment of this application, a method for detecting human behavior activities is provided.

[0028] Figure 1 is a flowchart of a method for detecting human behavior activities according to an embodiment of this application. As Figure 1 shown, the detection method includes the following steps:

[0029] Step S101, obtain a series of consecutive frames of images to be detected, and perform human detection on each of the above-mentioned images to be detected to obtain a plurality of first target data groups. Each of the above-mentioned images to be detected corresponds to at least one first target data group. The above-mentioned first target data group includes the position information of a first target point and the position information of a second target point. The above-mentioned first target point and the above-mentioned second target point are on a target diagonal line. The above-mentioned target diagonal line is a diagonal line of the smallest rectangular area including the human body in the detected above-mentioned image to be detected;

[0030] Step S102, use the dense optical flow algorithm to calculate any two adjacent above-mentioned images to be detected to obtain a plurality of corresponding optical flow images. One of the above-mentioned optical flow images is calculated from two adjacent above-mentioned images to be detected;

[0031] Step S103: Determine the movement trajectory of the human body based on at least each of the above optical flow images and the corresponding first target data groups, and determine the behavior of the human body based on at least the movement trajectory of the human body.

[0032] In the above method for detecting the behavior of a human body, first, obtain a series of consecutive frames of images to be detected, and perform human body detection on each of the obtained images to be detected to obtain a plurality of first target data groups. Among them, one image to be detected corresponds to at least one first target data group. Secondly, use the dense optical flow algorithm to calculate any two adjacent images to be detected, and obtain a plurality of corresponding optical flow images. Finally, determine the movement trajectory of the human body based on at least each of the above optical flow images and the corresponding first target data groups, and determine the behavior of the human body based on at least the movement trajectory of the human body. In this solution, human body detection is performed on a series of consecutive frames of images to be detected to obtain a plurality of first target data groups, and the dense optical flow algorithm is used to calculate any two adjacent frames of images to be detected to obtain corresponding optical flow images. Then, based on each optical flow image and the corresponding first target data group, the movement trajectory of the human body is determined. Finally, based on at least the movement trajectory of the human body, the behavior of the human body is determined. Compared with the movement information of the human body measured by sensors in the prior art, in this solution, more sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized more fully. As a result, the determined movement trajectory of the human body is more accurate. Then, based on at least the movement trajectory of the human body, the behavior of the human body is determined, which ensures that the behavior of the human body can be determined more accurately, thus solving the problem in the prior art that it is difficult to detect the behavior of the human body accurately.

[0033] Specifically, in this application, human body detection is performed on a series of consecutive frames of images to be detected to obtain a plurality of first target data groups. The first target data group can be understood as the position information of the detected human body in the image to be detected. Since the first target data group includes the position information of the first target point and the second target point, that is, a minimum rectangular area including the human body can be formed according to the first target point and the second target point. In the actual application process, for one frame of the image to be detected, if there is only one human body in the image to be detected, one first target data group can be obtained; if there are two human bodies in the image to be detected, two second target data groups can be obtained. That is to say, the number of first target data groups corresponding to one frame of the image to be detected can be determined according to the number of human bodies in the image to be detected.

[0034] In addition, the above-mentioned dense optical flow algorithm can be the Farneback algorithm or the FlowNet algorithm. Of course, it is not limited to the two optical flow algorithms listed, and it can also be other optical flow algorithms in the prior art that can obtain optical flow images.

[0035] In the actual application process, the size of the obtained optical flow image is the same as that of the image to be detected. In addition, the electronic device for obtaining multiple consecutive frames of the image to be detected can be a camera, a smartphone, a smart tablet, or a computer. In this application, the method for obtaining multiple consecutive frames of the image to be detected is not limited, and only multiple frames of the image to be detected need to be obtained. Of course, a video can also be obtained, and the obtained video is processed to obtain multiple consecutive frames of the image to be detected.

[0036] Specifically, the method for performing human body detection on the obtained multiple consecutive frames of the image to be detected can be a deep learning method or a machine learning method. For example, specifically, it can be a detection method combining Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM), or the YOLO detection algorithm, or the Faster RCNN detection algorithm (Faster Regions with CNN features), etc.

[0037] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0038] In order to more accurately determine the movement trajectory of the human body, in an embodiment of the present application, the movement trajectory of the human body is determined at least based on each of the above optical flow images and the corresponding first target data groups, including: in the (N + 1)-th frame of the above optical flow image, marking the position corresponding to the first target data group of the above to-be-detected image in the (N + 1)-th frame to obtain the marked (N + 1)-th frame of the above optical flow image, where N ≥ 1; determining the corresponding second target data group in the N-th frame of the above optical flow image according to the marked (N + 1)-th frame of the above optical flow image; determining whether the (N + 1)-th frame of the above optical flow image and the N-th frame of the above optical flow image correspond to the same human body according to the second target data group corresponding to the N-th frame of the above optical flow image and the first target data group corresponding to the N-th frame of the above to-be-detected image; and in the case of corresponding to the same human body, determining the movement trajectory of the human body according to the first target data group of the (N + 1)-th frame of the above to-be-detected image and the first target data group of the N-th frame of the above to-be-detected image.

[0039] Specifically, optical flow calculation is performed on any two adjacent frames of the above to-be-detected images to obtain the corresponding optical flow images. For example, the optical flow image corresponding to the (N + 1)-th frame of the to-be-detected image can be obtained through the N-th frame and the (N + 1)-th frame of the to-be-detected images. Therefore, each to-be-detected image corresponds to at least one group of first target data groups and an optical flow image. That is, the position information of the human body can be marked in the corresponding optical flow image according to the first target data group. Since the optical flow image has the movement information of each pixel point, the position information of the human body in the N-th frame of the optical flow image can be traced back from the (N + 1)-th frame of the optical flow image, which is the second target data group (i.e., the predicted position information of the human body in the N-th frame of the optical flow image). Then, it is determined whether it corresponds to the same human body according to the second target data group of the N-th frame of the optical flow image and the first target data group corresponding to the N-th frame of the to-be-detected image (since the optical flow image and the to-be-detected image are in one-to-one correspondence, it can also be understood as the first target data group corresponding to the N-th frame of the optical flow image). In the case of corresponding to the same human body, the movement trajectory of the human body is determined according to the first target data group of the (N + 1)-th frame of the to-be-detected image and the first target data group of the N-th frame of the to-be-detected image, which ensures that the movement trajectory of the human body can be more accurately determined.

[0040] Of course, in the case where it is determined that the (N + 1)-th frame of the above optical flow image and the N-th frame of the above optical flow image do not correspond to the same human body, a movement trajectory of this human body can be newly created.

[0041] In another embodiment of the present application, determining whether the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body according to the second target data group corresponding to the optical flow image of the N-th frame and the first target data group corresponding to the image to be detected of the N-th frame includes: determining a first predetermined area corresponding to the first target data group in the optical flow image of the N-th frame, and determining a second predetermined area corresponding to the second target data group in the optical flow image of the N-th frame; taking the intersection of the first predetermined area and the second predetermined area to obtain a first predetermined area, and taking the union of the first predetermined area and the second predetermined area to obtain a second predetermined area; calculating the ratio of the first predetermined area to the second predetermined area to obtain a target ratio; when the target ratio is greater than or equal to a predetermined value, determining that the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body. In this embodiment, when the target ratio is greater than or equal to the predetermined value, it is determined that the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body, which ensures that it is relatively simple and efficient to determine that the optical flow image of the (N + 1)-th frame and the optical flow image of the N-th frame correspond to the same human body.

[0042] In yet another embodiment of the present application, determining the behavior of the human body at least according to the movement trajectory of the human body includes: intercepting the corresponding image to be detected according to each first target data group to obtain a corresponding target image, where the target image is an image including one human body, and the first target data group and the target image are in one-to-one correspondence; sequentially intercepting the corresponding optical flow images according to the target images in the images to be detected corresponding to the movement trajectory of the human body to obtain a plurality of optical flow sub-images; classifying and judging the plurality of optical flow sub-images to determine the behavior of the human body. In this embodiment, according to the target images in the images to be detected corresponding to the movement trajectory of the human body, the corresponding optical flow images are intercepted to obtain a plurality of optical flow sub-images, and then the obtained plurality of optical flow sub-images are classified and judged to determine the behavior of the human body. In this solution, instead of classifying and judging the entire corresponding optical flow image, the optical flow sub-images are classified and judged, which can reduce the interference generated by other areas in the optical flow image and further ensure that the behavior of the human body can be determined relatively accurately.

[0043] Specifically, according to the time sequence in which the human body's movement trajectory appears in a video or multiple frames of images to be detected, and through the target images corresponding to the images to be detected, the regions corresponding to the target images in the corresponding optical flow images are intercepted to obtain multiple optical flow sub-images. To make those skilled in the art understand this solution more clearly, the following is illustrated by a specific example. For example, when the above-mentioned human body's movement trajectory starts from the 3rd frame and ends at the 50th frame, after determining the human body's movement trajectory, according to the time sequence, based on the target image corresponding to the 3rd frame of the image to be detected, the region corresponding to the above-mentioned target image is intercepted in the 3rd frame of the optical flow image to obtain the optical flow sub-image corresponding to the 3rd frame of the optical flow image, and subsequent interceptions are performed frame by frame. That is to say, according to each target image corresponding to the 3rd to 50th frames of the images to be detected, interceptions are made from the 3rd to 50th frames of the optical flow images, so that multiple optical flow sub-images can be obtained, and then the obtained multiple optical flow sub-images are classified, thereby determining the above-mentioned human body's behavior activities.

[0044] In the actual application process, the obtained multiple optical flow sub-images can also be input into a CRNN (Convolutional Recurrent Neural Network) or a Conv-LSTM (Convolutional Long Short Term Memory Network) in chronological order for classification to obtain the above-mentioned human body's behavior activities.

[0045] In order to further ensure that the above-mentioned human body's behavioral activities can be determined more accurately subsequently, in another embodiment of the present application, in chronological order, according to each of the above-mentioned target images in each of the above-mentioned to-be-detected images corresponding to the above-mentioned human body's motion trajectory, each corresponding optical flow image is intercepted to obtain a plurality of optical flow sub-images, including: combining the above-mentioned first target data group corresponding to the above-mentioned target image in the Nth frame of the above-mentioned to-be-detected image with the above-mentioned first target data group corresponding to the above-mentioned target image in the (N + 1)th frame of the above-mentioned to-be-detected image to obtain a third target data group; determining the largest target area corresponding to the above-mentioned third target data group in the (N + 1)th frame of the above-mentioned optical flow image, and intercepting the above-mentioned target area to obtain the above-mentioned optical flow sub-image. In this embodiment, the first target data group of the Nth frame of the to-be-detected image and the first target data group of the (N + 1)th frame of the to-be-detected image are combined to obtain a third target data group, and then according to the third target data group, an interception is performed in the (N + 1)th frame of the optical flow image to obtain the optical flow sub-image. That is, in this solution, the first target data groups of two adjacent frames are combined to obtain a third target data group with a larger range, and then an interception is performed in the corresponding optical flow image. This ensures that relatively sufficient motion information of the above-mentioned human body can be obtained, but the entire optical flow image is not used. That is, this solution can minimize the interference generated by other areas and ensure that sufficient motion information of the human body is obtained.

[0046] Specifically, for example, the first position point of the first target data group in the Nth frame of the to-be-detected image is (2, 1), and the second position point is (4, 3). The first position point of the first target data group in the (N + 1)th frame of the to-be-detected image is (3, 0), and the second position point is (5, 3). Then the first position point of the combined third target data group is (2, 1) and the second position point is (5, 3). That is to say, the rectangular area formed by the third target data group is the largest rectangular area that can be formed by taking the union of the two first target data groups.

[0047] In one embodiment of the present application, after at least determining the above-mentioned human body's behavioral activities according to the above-mentioned human body's motion trajectory, the method further includes: determining whether the above-mentioned human body is in a dangerous state according to the above-mentioned human body's behavioral activities; in the case of determining that the above-mentioned human body is in the above-mentioned dangerous state, sending an alarm message to the intelligent terminal, which ensures that a human body in a dangerous state can be treated in time.

[0048] In another embodiment of the present application, the above-mentioned human body's behavioral activities include at least one of the following: standing, walking, sitting down, and falling.

[0049] The embodiments of the present application also provide a detection device for human behavior activities. It should be noted that the detection device for human behavior activities in the embodiments of the present application can be used to execute the detection method for human behavior activities provided in the embodiments of the present application. The following introduces the detection device for human behavior activities provided in the embodiments of the present application.

[0050] Figure 2 It is a schematic diagram of the detection device for human behavior activities according to the embodiments of the present application. As Figure 2 shown, the detection device includes:

[0051] A detection unit 10, configured to obtain a plurality of consecutive frames of images to be detected, perform human detection on each of the above images to be detected, and obtain a plurality of first target data groups. Each of the above images to be detected corresponds to at least one first target data group. The first target data group includes the position information of a first target point and the position information of a second target point. The first target point and the second target point are on the target diagonal line, and the target diagonal line is a diagonal line of the smallest rectangular area including the human body in the detected image to be detected;

[0052] A calculation unit 20, configured to perform calculations on any two adjacent images to be detected by using a dense optical flow algorithm to obtain a plurality of corresponding optical flow images. One of the above optical flow images is obtained by calculating two adjacent images to be detected;

[0053] A first determination unit 30, configured to determine the movement trajectory of the human body at least according to each of the above optical flow images and the corresponding first target data groups, and determine the behavior activity of the human body at least according to the movement trajectory of the human body.

[0054] In the above-described detection device for human behavior activities, the detection unit is used to obtain a plurality of consecutive frames of images to be detected, and perform human detection on each of the above-mentioned images to be detected, obtaining a plurality of first target data groups. Each of the above-mentioned images to be detected corresponds to at least one first target data group. The above-mentioned first target data group includes the position information of a first target point and the position information of a second target point. The above-mentioned first target point and the above-mentioned second target point are on a target diagonal line, and the above-mentioned target diagonal line is a diagonal line of the smallest rectangular area including a human body in the detected above-mentioned image to be detected; the calculation unit is used to perform calculations on any two adjacent above-mentioned images to be detected by using the dense optical flow algorithm, obtaining a plurality of corresponding optical flow images, and one of the above-mentioned optical flow images is obtained by calculating two adjacent above-mentioned images to be detected; the first determination unit is used to determine the movement trajectory of the above-mentioned human body at least according to each of the above-mentioned optical flow images and the corresponding each of the above-mentioned first target data groups, and determine the behavior activity of the above-mentioned human body at least according to the movement trajectory of the above-mentioned human body. In this solution, human detection is performed on a plurality of consecutive frames of images to be detected obtained, obtaining a plurality of first target data groups, and calculations are performed on any two adjacent frames of images to be detected by using the dense optical flow algorithm, obtaining corresponding optical flow images. Then, according to each optical flow image and the corresponding each first target data group, the movement trajectory of the human body is determined. Finally, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined. Compared with the movement information of the human body measured by a sensor in the prior art, in this solution, relatively sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized relatively sufficiently. Furthermore, it ensures that the determined movement trajectory of the human body is relatively accurate. Then, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined, which ensures that the behavior activity of the human body can be determined relatively accurately, thus solving the problem in the prior art that it is difficult to detect the behavior activity of the human body relatively accurately.

[0055] Specifically, in this application, human detection is performed on a plurality of consecutive frames of images to be detected obtained, obtaining a plurality of first target data groups. The above-mentioned first target data group can be understood as the position information of the detected human body in the above-mentioned image to be detected. Since the above-mentioned first target data group includes the position information of a first target point and the above-mentioned position information of a second target point, that is, a smallest rectangular area including a human body can be formed according to the first target point and the second target point. In the actual application process, for a frame of image to be detected, if there is only one human body in the image to be detected, one first target data group can be obtained. If there are two human bodies in the image to be detected, two second target data groups can be obtained. That is to say, the number of first target data groups corresponding to a frame of image to be detected can be determined according to the number of human bodies in the image to be detected.

[0056] In addition, the above-mentioned dense optical flow algorithm can be the Farneback algorithm or the FlowNet algorithm. Of course, it is not limited to the two optical flow algorithms listed, and it can also be other optical flow algorithms in the prior art that can obtain optical flow images.

[0057] In the actual application process, the size of the above-mentioned obtained optical flow image is the same as that of the image to be detected. In addition, the electronic device for acquiring multiple consecutive frames of the image to be detected can be a camera, a smartphone, a smart tablet, or a computer. In this application, the method for acquiring multiple consecutive frames of the image to be detected is not limited, as long as multiple frames of the image to be detected are acquired. Of course, a video can also be acquired, and the acquired video can be processed to obtain multiple consecutive frames of the image to be detected.

[0058] Specifically, the method for performing human body detection on the acquired multiple consecutive frames of the image to be detected can be a deep learning method or a machine learning method. For example, specifically, it can be a detection method combining Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM), or the YOLO detection algorithm, or the Faster RCNN detection algorithm (Faster Regions with CNN features), etc.

[0059] In order to more accurately determine the movement trajectory of the human body, in an embodiment of this application, the above-mentioned first determination unit includes a marking module, a first determination module, a second determination module, and a third determination module. Among them, the above-mentioned marking module is used to mark the position corresponding to the first target data group of the image to be detected in the (N + 1)-th frame in the (N + 1)-th frame of the above-mentioned optical flow image, to obtain the marked (N + 1)-th frame of the above-mentioned optical flow image, where N ≥ 1; the above-mentioned first determination module is used to determine the corresponding second target data group in the N-th frame of the above-mentioned optical flow image according to the marked (N + 1)-th frame of the above-mentioned optical flow image; the above-mentioned second determination module is used to determine whether the (N + 1)-th frame of the above-mentioned optical flow image and the N-th frame of the above-mentioned optical flow image correspond to the same above-mentioned human body according to the above-mentioned second target data group corresponding to the N-th frame of the above-mentioned optical flow image and the above-mentioned first target data group corresponding to the N-th frame of the above-mentioned image to be detected; the above-mentioned third determination module is used to determine the movement trajectory of the above-mentioned human body according to the above-mentioned first target data group of the (N + 1)-th frame of the above-mentioned image to be detected and the above-mentioned first target data group of the N-th frame of the above-mentioned image to be detected when they correspond to the same above-mentioned human body.

[0060] Specifically, optical flow calculation is performed on any two adjacent frames of the to-be-detected images to obtain corresponding optical flow images. For example, through the Nth frame of the to-be-detected image and the (N + 1)th frame of the to-be-detected image, the optical flow image corresponding to the (N + 1)th frame of the to-be-detected image can be obtained. Therefore, each to-be-detected image corresponds to at least one group of first target data groups and an optical flow image. That is, according to the first target data group, the position information of the above-mentioned human body can be marked in the corresponding optical flow image. Since the optical flow image has the motion information of each pixel point, the position information of the above-mentioned human body in the optical flow image of the Nth frame can be traced back from the optical flow image of the (N + 1)th frame, which is the second target data group (that is, the predicted position information of the above-mentioned human body in the optical flow image of the Nth frame). Then, according to the second target data group of the optical flow image of the Nth frame and the first target data group corresponding to the Nth frame of the to-be-detected image (since the optical flow image corresponds one-to-one with the to-be-detected image, that is, it can also be understood as the first target data group corresponding to the optical flow image of the Nth frame), it is determined whether they correspond to the same human body. In the case of corresponding to the same human body, according to the first target data group of the (N + 1)th frame of the to-be-detected image and the first target data group of the Nth frame of the to-be-detected image, the motion trajectory of the human body is determined, which ensures that the motion trajectory of the human body can be determined more accurately.

[0061] Of course, in the case where it is determined that the optical flow image of the (N + 1)th frame and the optical flow image of the Nth frame do not correspond to the same human body, a new motion trajectory of this human body can be created.

[0062] In another embodiment of the present application, the second determination module includes a first determination sub-module, a generation sub-module, a calculation sub-module, and a second determination sub-module. Among them, the first determination sub-module is used to determine a first predetermined area corresponding to the first target data group in the optical flow image of the Nth frame, and determine a second predetermined area corresponding to the second target data group in the optical flow image of the Nth frame; the generation sub-module is used to take the intersection of the first predetermined area and the second predetermined area to obtain a first predetermined area, and take the union of the first predetermined area and the second predetermined area to obtain a second predetermined area; the calculation sub-module is used to calculate the ratio of the first predetermined area to the second predetermined area to obtain a target ratio; the second determination sub-module is used to determine that the optical flow image of the (N + 1)th frame and the optical flow image of the Nth frame correspond to the same human body when the target ratio is greater than or equal to a predetermined value. In this embodiment, when the target ratio is greater than or equal to a predetermined value, it is determined that the optical flow image of the (N + 1)th frame and the optical flow image of the Nth frame correspond to the same human body, which ensures that it can be determined more simply and efficiently that the optical flow image of the (N + 1)th frame and the optical flow image of the Nth frame correspond to the same human body.

[0063] In another embodiment of the present application, the above-mentioned first determination unit further includes a first interception module, a second interception module, and a fourth determination module. Among them, the first interception module is configured to intercept the corresponding image to be detected according to each of the above-mentioned first target data groups to obtain a corresponding target image, where the target image is an image including one human body, and the first target data group corresponds to the target image one by one; the second interception module is configured to intercept each of the corresponding optical flow images according to each of the target images in each of the images to be detected corresponding to the movement trajectory of the human body in chronological order to obtain a plurality of optical flow sub-images; the fourth determination module is configured to classify and judge the plurality of optical flow sub-images to determine the behavior of the human body. In this embodiment, according to each target image in each image to be detected corresponding to the movement trajectory of the human body, each corresponding optical flow image is intercepted to obtain a plurality of optical flow sub-images, and then the obtained plurality of optical flow sub-images are classified and judged to determine the behavior of the human body. In this solution, instead of classifying and judging the entire corresponding optical flow image, the optical flow sub-images are classified and judged, which can reduce the interference generated by other regions in the optical flow image, and further ensure that the behavior of the human body can be determined more accurately.

[0064] Specifically, according to the chronological order in which the movement trajectory of the human body appears in a video or multiple frames of images to be detected, and through the target images corresponding to the images to be detected, the regions corresponding to the target images in the corresponding optical flow images are intercepted to obtain a plurality of optical flow sub-images. To make those skilled in the art understand this solution more clearly, the following is illustrated by a specific example. For example, when the movement trajectory of the human body starts from the 3rd frame and ends at the 50th frame, after determining the movement trajectory of the human body, according to the chronological order, according to the target image corresponding to the 3rd frame image to be detected, the region corresponding to the target image is intercepted in the 3rd frame optical flow image to obtain the optical flow sub-image corresponding to the 3rd frame optical flow image, and subsequent interceptions are performed frame by frame. That is, according to each target image corresponding to the 3rd to 50th frame images to be detected, interceptions are made from the 3rd to 50th frame optical flow images, so that a plurality of optical flow sub-images can be obtained, and then the obtained plurality of optical flow sub-images are classified, so that the behavior of the human body can be determined.

[0065] In the actual application process, the obtained plurality of optical flow sub-images can also be input into a CRNN (Convolutional Recurrent Neural Network) or a Conv-LSTM (Convolutional Long Short Term Memory Network) in chronological order for classification to obtain the behavior of the human body.

[0066] In order to further ensure that the subsequent behavior activities of the above-mentioned human body can be determined more accurately, in another embodiment of the present application, the second interception module includes a merging sub-module and an interception sub-module. Among them, the merging sub-module is used to merge the first target data group corresponding to the target image in the Nth frame of the to-be-detected image with the first target data group corresponding to the target image in the (N + 1)th frame of the to-be-detected image to obtain a third target data group; the interception sub-module is used to determine the largest target area corresponding to the third target data group in the (N + 1)th frame of the optical flow image, and intercept the target area to obtain the optical flow sub-image. In this embodiment, the first target data group of the Nth frame of the to-be-detected image and the first target data group of the (N + 1)th frame of the to-be-detected image are merged to obtain a third target data group, and then, according to the third target data group, interception is performed in the (N + 1)th frame of the optical flow image to obtain the optical flow sub-image. That is, in this solution, the first target data groups of two adjacent frames are merged to obtain a third target data group with a larger range, and then interception is performed in the corresponding optical flow image. This ensures that relatively sufficient motion information of the above-mentioned human body can be obtained, but the entire optical flow image is not used. That is, this solution can minimize the interference generated by other areas and ensure that sufficient motion information of the human body can be obtained.

[0067] Specifically, for example, the first position point of the first target data group in the Nth frame of the to-be-detected image is (2, 1), and the second position point is (4, 3). The first position point of the first target data group in the (N + 1)th frame of the to-be-detected image is (3, 0), and the second position point is (5, 3). Then, the first position point of the third target data group obtained after merging is (2, 1), and the second position point is (5, 3). That is to say, the rectangular area formed by the third target data group is the largest rectangular area that can be formed by taking the union of the two first target data groups.

[0068] In an embodiment of the present application, the detection device further includes a second determination unit and a sending unit. Among them, the second determination unit is used to determine whether the human body is in a dangerous state according to the behavior activity of the human body after at least determining the behavior activity of the human body according to the motion trajectory of the human body; the sending unit is used to send an alarm message to the smart terminal when it is determined that the human body is in the dangerous state, so as to ensure that the human body in the dangerous state can be treated in time.

[0069] In another embodiment of the present application, the behavior activities of the above-mentioned human body include at least one of the following: standing, walking, sitting down, and falling.

[0070] In order to enable those skilled in the art to more clearly and clearly understand the technical solution of the present application, the following will be described in conjunction with specific embodiments:

[0071] Embodiment

[0072] As Figure 3 shown, it is a flowchart of the detection method for human behavior activities in this application. The specific detection process is as follows: First, obtain a continuous multi-frame image or video to be detected; Second, perform human detection on each image to be detected to obtain at least one first target data group, that is, perform human detection on a frame of the image to be detected. If one human is detected in the image to be detected, one first target data group is obtained. If multiple humans are detected in the image to be detected, multiple first target data groups are obtained; Third, use the dense optical flow algorithm to calculate the optical flow for any two adjacent frames of the image to be detected to obtain multiple optical flow images. Then, determine the motion trajectory of the human according to each optical flow image and each first target data group; Finally, intercept the corresponding image to be detected according to each first target data group to obtain a target image. According to the target image, intercept in the corresponding optical flow image to obtain multiple optical flow sub-images. Then, input the optical flow sub-images corresponding to the target images belonging to the same motion trajectory into the CRNN or Conv-LSTM network to determine the human behavior activities.

[0073] The above-mentioned detection device for human behavior activities includes a processor and a memory. The above-mentioned detection unit, calculation unit, first determination unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.

[0074] The processor contains a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the problem that it is difficult to accurately detect human behavior activities in the prior art can be solved.

[0075] The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory includes at least one storage chip.

[0076] An embodiment of the present invention provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the above-mentioned detection method for human behavior activities is implemented.

[0077] An embodiment of the present invention provides a processor, and the above-mentioned processor is used to run a program. When the above-mentioned program runs, the above-mentioned detection method for human behavior activities is executed.

[0078] In a typical embodiment of the present application, a behavior activity detection system is further provided, including: an image acquisition device, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing any one of the above methods.

[0079] The above behavior activity detection system can execute any one of the above detection methods. In the above detection method, first, obtain a continuous multi-frame of images to be detected, and perform human body detection on each of the obtained images to be detected to obtain a plurality of first target data groups, wherein at least one of the above first target data groups corresponds to one of the above images to be detected. Secondly, use the dense optical flow algorithm to calculate any two adjacent images to be detected to obtain a plurality of corresponding optical flow images; finally, determine the movement trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, and determine the behavior activity of the human body at least according to the movement trajectory of the human body. In this solution, human body detection is performed on the obtained continuous multi-frame of images to be detected to obtain a plurality of first target data groups, and the dense optical flow algorithm is used to calculate any two adjacent frames of images to be detected to obtain corresponding optical flow images. Then, according to each optical flow image and the corresponding first target data groups, the movement trajectory of the human body is determined. Finally, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined. Compared with the movement information of the human body measured by sensors in the prior art, in this solution, more sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized more fully, and thus ensures that the determined movement trajectory of the human body is more accurate. Then, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined, which ensures that the behavior activity of the human body can be determined more accurately, thereby solving the problem in the prior art that it is difficult to detect the behavior activity of the human body more accurately.

[0080] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements at least the following steps:

[0081] Step S101, obtain a continuous multi-frame of images to be detected, and perform human body detection on each of the above images to be detected to obtain a plurality of first target data groups. Each of the above images to be detected corresponds to at least one first target data group. The above first target data group includes the position information of a first target point and the position information of a second target point. The above first target point and the above second target point are on the target diagonal line. The above target diagonal line is a diagonal line of the smallest rectangular area including the human body in the detected image to be detected.

[0082] Step S102: Use the dense optical flow algorithm to calculate any two adjacent to-be-detected images, obtaining multiple corresponding optical flow images, where one of the optical flow images is calculated from two adjacent to-be-detected images.

[0083] Step S103: Determine the movement trajectory of the human body based on at least each of the optical flow images and the corresponding first target data groups, and determine the behavior of the human body based on at least the movement trajectory of the human body.

[0084] The device in this article can be a server, PC, PAD, mobile phone, etc.

[0085] This application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with at least the following method steps:

[0086] Step S101: Obtain multiple consecutive frames of to-be-detected images, and perform human body detection on each of the to-be-detected images, obtaining multiple first target data groups. Each to-be-detected image corresponds to at least one first target data group. The first target data group includes the position information of a first target point and the position information of a second target point. The first target point and the second target point are on the target diagonal line, and the target diagonal line is a diagonal line of the smallest rectangle area including the human body in the detected to-be-detected image.

[0087] Step S102: Use the dense optical flow algorithm to calculate any two adjacent to-be-detected images, obtaining multiple corresponding optical flow images, where one of the optical flow images is calculated from two adjacent to-be-detected images.

[0088] Step S103: Determine the movement trajectory of the human body based on at least each of the optical flow images and the corresponding first target data groups, and determine the behavior of the human body based on at least the movement trajectory of the human body.

[0089] In the above embodiments of the present invention, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0090] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the above division of units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0091] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0093] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in each embodiment of the present invention. The foregoing storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs, and other media that can store program codes.

[0094] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0095] 1) In the method for detecting human behavior activities of the present application, first, a series of consecutive frames of images to be detected are obtained, and human detection is performed on each of the obtained images to be detected, resulting in a plurality of first target data groups. Among them, at least one first target data group corresponds to one of the images to be detected. Secondly, the dense optical flow algorithm is used to calculate any two adjacent images to be detected, obtaining a plurality of corresponding optical flow images. Finally, at least based on each of the optical flow images and the corresponding first target data groups, the movement trajectory of the human body is determined, and at least based on the movement trajectory of the human body, the behavior activities of the human body are determined. In this solution, human detection is performed on a series of consecutive frames of images to be detected, resulting in a plurality of first target data groups. The dense optical flow algorithm is used to calculate any two adjacent frames of images to be detected, obtaining the corresponding optical flow images. Then, based on each optical flow image and the corresponding first target data group, the movement trajectory of the human body is determined. Finally, at least based on the movement trajectory of the human body, the behavior activities of the human body are determined. Compared with the movement information of the human body measured by sensors in the prior art, in this solution, more sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized more fully. As a result, the determined movement trajectory of the human body is more accurate. Then, at least based on the movement trajectory of the human body, the behavior activities of the human body are determined, which ensures that the behavior activities of the human body can be determined more accurately, thus solving the problem in the prior art that it is difficult to detect the behavior activities of the human body more accurately.

[0096] 2) In the detection device for human body behavior activities of the present application, the detection unit is used to obtain a continuous multi-frame of images to be detected, and perform human body detection on each of the above-mentioned images to be detected, obtaining a plurality of first target data groups. Each of the above-mentioned images to be detected corresponds to at least one first target data group. The above-mentioned first target data group includes the position information of a first target point and the position information of a second target point. The above-mentioned first target point and the above-mentioned second target point are on the target diagonal line. The above-mentioned target diagonal line is a diagonal line of the smallest rectangle area including the human body in the detected above-mentioned image to be detected; the calculation unit is used to calculate any two adjacent above-mentioned images to be detected by using the dense optical flow algorithm, obtaining a plurality of corresponding optical flow images. One of the above-mentioned optical flow images is obtained by calculating two adjacent above-mentioned images to be detected; the first determination unit is used to determine the movement trajectory of the above-mentioned human body at least according to each of the above-mentioned optical flow images and the corresponding each of the above-mentioned first target data groups, and determine the behavior activity of the above-mentioned human body at least according to the movement trajectory of the above-mentioned human body. In this solution, human body detection is performed on the obtained continuous multi-frame of images to be detected, obtaining a plurality of first target data groups, and any two adjacent frames of images to be detected are calculated by using the dense optical flow algorithm, obtaining the corresponding optical flow images. Then, according to each optical flow image and the corresponding each first target data group, the movement trajectory of the human body is determined. Finally, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined. Compared with the movement information of the human body measured by a sensor in the prior art, in this solution, more sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized more fully. Furthermore, it ensures that the determined movement trajectory of the human body is relatively accurate. Then, at least according to the movement trajectory of the human body, the behavior activity of the human body is determined, which ensures that the behavior activity of the human body can be determined relatively accurately, thus solving the problem that it is difficult to detect the behavior activity of the human body accurately in the prior art.

[0097] 3) The behavior activity detection system of the present application can execute any one of the above detection methods. In the above detection methods, first, a plurality of consecutive frames of images to be detected are obtained, and human body detection is performed on each of the obtained images to be detected, obtaining a plurality of first target data groups. Among them, at least one of the above first target data groups corresponds to one of the above images to be detected. Secondly, the dense optical flow algorithm is used to calculate any two adjacent images to be detected, obtaining a plurality of corresponding optical flow images. Finally, at least based on each of the above optical flow images and the corresponding first target data groups, the movement trajectory of the human body is determined, and at least based on the movement trajectory of the human body, the behavior activity of the human body is determined. In this solution, human body detection is performed on a plurality of consecutive frames of images to be detected, obtaining a plurality of first target data groups, and the dense optical flow algorithm is used to calculate any two adjacent frames of images to be detected, obtaining the corresponding optical flow images. Then, based on each optical flow image and the corresponding first target data groups, the movement trajectory of the human body is determined. Finally, at least based on the movement trajectory of the human body, the behavior activity of the human body is determined. Compared with the movement information of the human body measured by sensors in the prior art, in this solution, more sufficient movement information of the human body can be obtained through the optical flow images, which ensures that the movement information of each part of the human body can be utilized more fully. Furthermore, it ensures that the determined movement trajectory of the human body is relatively accurate. Then, at least based on the movement trajectory of the human body, the behavior activity of the human body is determined, which ensures that the behavior activity of the human body can be determined relatively accurately, thus solving the problem in the prior art that it is difficult to detect the behavior activity of the human body relatively accurately.

[0098] The foregoing is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting human behavior activities, characterized in that, Including: Obtain consecutive multiple frames of images to be detected, perform human detection on each of the images to be detected, and obtain multiple first target data groups. Each image to be detected corresponds to at least one first target data group. The first target data group includes the position information of a first target point and the position information of a second target point. The first target point and the second target point are on a target diagonal line, and the target diagonal line is a diagonal line of the smallest rectangle area including a human body in the detected image to be detected; Use the dense optical flow algorithm to calculate any two adjacent images to be detected, and obtain multiple corresponding optical flow images. One optical flow image is obtained by calculating two adjacent images to be detected; Determine the movement trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, and determine the behavior of the human body at least according to the movement trajectory of the human body.

2. The method according to claim 1, wherein Determine the movement trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, including: In the (N + 1)-th frame of the optical flow image, mark the positions corresponding to the first target data group of the (N + 1)-th frame of the image to be detected, and obtain the marked (N + 1)-th frame of the optical flow image, where N ≥ 1; According to the marked (N + 1)-th frame of the optical flow image, determine the corresponding second target data group in the N-th frame of the optical flow image; According to the second target data group corresponding to the N-th frame of the optical flow image and the first target data group corresponding to the N-th frame of the image to be detected, determine whether the (N + 1)-th frame of the optical flow image and the N-th frame of the optical flow image correspond to the same human body; In the case of corresponding to the same human body, determine the movement trajectory of the human body according to the first target data group of the (N + 1)-th frame of the image to be detected and the first target data group of the N-th frame of the image to be detected.

3. The method according to claim 2, wherein According to the second target data group corresponding to the N-th frame of the optical flow image and the first target data group corresponding to the N-th frame of the image to be detected, determine whether the (N + 1)-th frame of the optical flow image and the N-th frame of the optical flow image correspond to the same human body, including: Determine a first predetermined area corresponding to the first target data group in the N-th frame of the optical flow image, and determine a second predetermined area corresponding to the second target data group in the N-th frame of the optical flow image; Take the intersection of the first predetermined area and the second predetermined area to obtain a first predetermined area, and take the union of the first predetermined area and the second predetermined area to obtain a second predetermined area; Calculate the ratio of the first predetermined area to the second predetermined area to obtain a target ratio; In the case where the target ratio is greater than or equal to a predetermined value, determine that the (N + 1)-th frame of the optical flow image and the N-th frame of the optical flow image correspond to the same human body.

4. The method according to claim 1, characterized in that Determine the behavior of the human body at least according to the movement trajectory of the human body, including: Intercept the corresponding to-be-detected image according to each of the first target data groups to obtain a corresponding target image, where the target image is an image including one human body, and the first target data groups and the target images are in one-to-one correspondence; Intercept the corresponding optical flow images according to the target images in each of the to-be-detected images corresponding to the movement trajectory of the human body in chronological order to obtain a plurality of optical flow sub-images; Classify and judge the plurality of optical flow sub-images to determine the behavior of the human body.

5. The method according to claim 4, wherein Intercept the corresponding optical flow images according to the target images in each of the to-be-detected images corresponding to the movement trajectory of the human body in chronological order to obtain a plurality of optical flow sub-images, including: Merge the first target data group corresponding to the target image in the Nth frame of the to-be-detected image with the first target data group corresponding to the target image in the (N + 1)th frame of the to-be-detected image to obtain a third target data group; Determine the largest target area corresponding to the third target data group in the (N + 1)th optical flow image, and intercept the target area to obtain the optical flow sub-image.

6. The method according to any one of claims 1 to 5, characterized in that After at least determining the behavior of the human body according to the movement trajectory of the human body, the method further includes: Determine whether the human body is in a dangerous state according to the behavior of the human body; In the case of determining that the human body is in the dangerous state, send an alarm message to the intelligent terminal.

7. The method according to claim 6, wherein The behavior of the human body includes at least one of the following: standing, walking, sitting down, falling.

8. A detection device for human behavior activities, characterized in that, Including: A detection unit, configured to obtain a plurality of consecutive frames of to-be-detected images, and perform human body detection on each of the to-be-detected images to obtain a plurality of first target data groups, where each to-be-detected image corresponds to at least one first target data group, and the first target data group includes the position information of a first target point and the position information of a second target point, and the first target point and the second target point are on the target diagonal line, and the target diagonal line is a diagonal line of the smallest rectangular area including the human body in the detected to-be-detected image; A calculation unit, configured to calculate any two adjacent to-be-detected images by using a dense optical flow algorithm to obtain a plurality of corresponding optical flow images, and one optical flow image is obtained by calculating two adjacent to-be-detected images; A first determination unit, configured to determine the movement trajectory of the human body at least according to each of the optical flow images and the corresponding first target data groups, and determine the behavior of the human body at least according to the movement trajectory of the human body.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where the program executes the method according to any one of claims 1 to 7.

10. A behavior activity detection system, characterized in that, Including: An image acquisition device, one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include the method for executing any one of claims 1 to 7.

Images