Activity detection method, device, electronic device and storage medium

Abstract

The present application discloses an activity detection method, device, electronic device and storage medium, and relates to the field of data processing technology. The method comprises: obtaining a picture to be detected, determining an object frame corresponding to an object to be detected from the picture to be detected, and determining multiple key points of the object to be detected from the picture to be detected, and calculating the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected, so that the object to be detected does not need to wear relevant activity detection equipment and does not need to obtain the personal information of the object to be detected, so as to determine the activity of the object to be detected, simplify the calculations related to activity detection, and improve the calculation speed.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to an activity detection method, device, electronic device and storage medium. Background Art

[0002] With the development of society and the improvement of living standards, people's demand for daily health status monitoring is becoming increasingly urgent, making activity detection and other aspects a research hotspot for many scholars. At present, users can wear auxiliary equipment to detect the amount of activity of users and determine the amount of activity, but it is troublesome for users to wear auxiliary equipment, which reduces the user experience. Summary of the invention

[0003] In view of the above problems, the present application proposes an activity detection method, device, electronic device and storage medium to solve the above problems.

[0004] In a first aspect, an embodiment of the present application provides an activity detection method, the method comprising: acquiring a picture to be detected; determining an object frame corresponding to an object to be detected from the picture to be detected, and determining multiple key points of the object to be detected from the picture to be detected; and calculating the activity data consumption of the object to be detected based on changes in the object frame corresponding to the object to be detected obtained by tracking and monitoring the object to be detected, as well as changes in multiple key points of the object to be detected.

[0005] In the second aspect, an embodiment of the present application provides an activity detection device, which includes: a module for acquiring a picture to be detected, used to acquire a picture to be detected; a key point determination module, used to determine an object frame corresponding to an object to be detected from the picture to be detected, and determine multiple key points of the object to be detected from the picture to be detected; an activity quantity calculation module, used to calculate the activity data consumption of the object to be detected based on changes in the object frame corresponding to the object to be detected obtained by tracking and monitoring the object to be detected, and changes in multiple key points of the object to be detected.

[0006] In some embodiments, the module for determining data corresponding to the objects to be detected includes: multiple sub-modules for determining information corresponding to the objects to be detected, wherein: multiple sub-modules for determining data corresponding to the objects to be detected are used to determine the object frames corresponding to each of the multiple objects to be detected from the picture to be detected, and determine multiple key points of each of the multiple objects to be detected from the picture to be detected.

[0007] In some embodiments, the module for determining data corresponding to the object to be detected further includes: a feature information extraction submodule and a submodule for determining the object to be detected, wherein: the feature information extraction submodule is used to obtain the object to be determined from the image to be detected and extract feature information of the object to be determined. The submodule for determining the object to be determined is used to determine the object to be determined as the object to be detected when the feature information of the object to be determined matches the preset feature information.

[0008] In some embodiments, the module for determining data corresponding to the object to be detected further includes: an object frame determination submodule and a key point determination submodule, wherein: the object frame determination submodule is used to determine the object frame corresponding to the object to be detected from the image to be detected based on a human body detection algorithm. The key point determination submodule is used to determine multiple key points of the object to be detected from the image to be detected based on a human body key point detection algorithm.

[0009] In some embodiments, the activity amount calculation module includes: an object frame average size acquisition submodule, a plurality of key point displacement acquisition submodules, and an activity amount acquisition submodule, wherein: the object frame average size acquisition submodule is used to acquire the average size of the object frame corresponding to the object to be detected in each two adjacent frames of images. The plurality of key point displacement acquisition submodules are used to acquire the displacement of multiple key points of the object to be detected in each two adjacent frames of images. The activity amount acquisition submodule is used to calculate the activity consumption data of the object to be detected based on the average size of the object frame corresponding to the object to be detected in each two adjacent frames of images, and the displacement of multiple key points of the object to be detected in each two adjacent frames of images.

[0010] In some embodiments, the activity acquisition submodule includes: a relative displacement acquisition unit and an activity acquisition unit of the object to be detected, wherein: the relative displacement acquisition unit is used to obtain the relative displacement of the object to be detected in each two adjacent frames of images by using the ratio of the displacement of multiple key points of the object to be detected corresponding to each two adjacent frames of images to the average size of the object frame corresponding to the object to be detected. The activity acquisition unit of the object to be detected is used to accumulate the relative displacement of the object to be detected in each two adjacent frames of images to obtain the activity consumption data of the object to be detected.

[0011] In some embodiments, the activity volume calculation module further includes: a total activity volume acquisition submodule, a second duration acquisition submodule, and an average activity volume acquisition submodule for the space to be detected, wherein: the total activity volume acquisition submodule is used to calculate the total activity consumption data of the multiple objects to be detected based on the changes in the object frames corresponding to the multiple objects to be detected that are tracked and monitored, and the changes in the multiple key points of the multiple objects to be detected. The second duration acquisition submodule is used to obtain the duration of any object to be detected among the multiple objects to be detected in the screen to be detected as the second duration. The average activity volume acquisition submodule for the space to be detected is used to obtain the average activity volume of the multiple objects to be detected in the space to be detected by using the ratio of the total activity volume of the multiple objects to be detected to the second duration.

[0012] In some embodiments, the momentum determination device further includes: a first duration acquisition module and an average activity acquisition module of the object to be detected, wherein: the first duration acquisition module is used to obtain the duration of the object to be detected in the image to be detected as the first duration. The average activity acquisition module of the object to be detected is used to obtain the average activity of the object to be detected by using the ratio of the activity consumption data of the object to be detected to the first duration.

[0013] In some embodiments, the first duration acquisition module includes: a frame number and frame rate acquisition submodule and a first duration calculation module, wherein: the frame number and frame rate acquisition submodule is used to obtain the number of frames in which the object to be detected exists in the picture to be detected, and the frame rate of the picture to be detected. The first duration calculation module is used to obtain the duration of the object to be detected existing in the picture to be detected based on the frame number and the frame rate, as the first duration.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, and the program code can be called by a processor to execute the above method.

[0016] The activity detection method, device, electronic device and storage medium provided in the embodiments of the present application obtain a picture to be detected, determine an object frame corresponding to an object to be detected from the picture to be detected, and determine multiple key points of the object to be detected from the picture to be detected, and calculate the activity consumption data of the object to be detected based on changes in the object frame corresponding to the object to be detected obtained by tracking and monitoring the object to be detected, as well as changes in multiple key points of the object to be detected. This eliminates the need for the object to be detected to wear relevant activity detection equipment and does not require the acquisition of personal information of the object to be detected, thereby achieving the determination of the activity amount of the object to be detected, while simplifying calculations related to activity detection and improving calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram showing an application scenario of the activity detection method provided in an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of multiple key points of an object to be detected provided by an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram showing the changes of the object to be detected corresponding to each two adjacent frames of images provided in an embodiment of the present application is shown;

[0023] Figure 6 This application shows Figure 4 The flowchart of step S250 of the people counting method shown is as follows;

[0024] Figure 7 A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0025] Figure 8 This application shows Figure 6 The flowchart of step S340 of the people counting method shown in FIG.

[0026] Fig. 9A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0027] Fig.10 A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0028] Fig.11 A schematic diagram of a flow chart of an activity detection method provided by an embodiment of the present application is shown;

[0029] Fig.12 A module block diagram of an activity detection device provided in an embodiment of the present application is shown;

[0030] Fig.13 A block diagram of an electronic device according to an embodiment of the present application for executing an activity detection method according to an embodiment of the present application is shown;

[0031] Fig.14 A storage unit for storing or carrying program codes for implementing the activity detection method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0033] With the development of society and the improvement of living standards, people's demand for daily health status monitoring is becoming increasingly urgent, making activity detection and other aspects a research hotspot for many scholars. At present, most activity detection is achieved by wearing activity detection equipment, collecting specific biological signals, and determining the amount of activity through related calculations. However, it requires wearing activity detection equipment and taking specific biological signals, and the related calculations of activity detection are relatively complicated.

[0034] In response to the above problems, the inventors have discovered after long-term research and proposed the activity detection method, device, electronic device and storage medium provided in the embodiments of the present application, which do not require the subject to be detected to wear a detection device or obtain the personal information of the subject to be detected, and can determine the activity amount of the subject to be detected, while simplifying the calculations related to activity detection and improving the calculation speed. The specific activity detection method is described in detail in the subsequent embodiments.

[0035] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an activity detection method provided by an embodiment of the present application. In a specific embodiment, the activity detection method is applied to Fig.10 The activity detection device 200 and the electronic device 100 equipped with the activity detection device 200 are shown in FIG. Fig.11 ). The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device used in this embodiment can be a smart phone, a tablet computer, a desktop computer, a wearable electronic device, etc., which is not limited here. Figure 1 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0036] Step S110: Acquire the image to be detected.

[0037] In some embodiments, the electronic device may be connected to a camera. The camera may capture the image to be detected, and the electronic device obtains the image to be detected captured by the camera through the connection with the camera. Figure 2 , Figure 2 A schematic diagram of an application scenario of the activity detection method provided in an embodiment of the present application is shown. An electronic device 100 can be connected to a camera 200 to enable data interaction between the electronic device 100 and the camera 200, wherein the data may include a picture to be detected, which is not limited here. Among them, the number of cameras 200 is at least one, and the at least one camera 200 is connected to the electronic device 100 respectively, for example, one electronic device 100 is connected to multiple cameras 200, which is not limited here. Among them, the electronic device 100 and the camera 200 can be connected wirelessly via Bluetooth, Wifi or ZigBee, and the electronic device 100 and the camera 200 can also be connected via a wired method such as a USB data cable.

[0038] The camera 200 may include but is not limited to an analog camera, a digital camera, an anti-peeping camera, a night vision camera, a driver-based camera, and a driver-free camera.

[0039] In some embodiments, a camera may be provided in the electronic device, so the electronic device can obtain the image to be detected through the built-in camera of the electronic device, wherein the image to be detected refers to the image to be detected. As an implementable manner, the electronic device may be installed with various video software, and the electronic device may control the camera to collect and obtain the image to be detected through the installed video software, wherein the specific video software is not limited here.

[0040] In some embodiments, the user can set the conditions for obtaining the screen to be detected in the electronic device, wherein the conditions for obtaining the screen to be detected can be a certain time period and / or the state of the object to be detected, etc., which are not limited here. For example, the user sets that when it is in the time period from 14:00 to 17:00, the electronic device obtains the screen to be detected, and when the time is in the time period from 00:00 to 06:00, the electronic device does not obtain the screen to be detected. For another example, the user sets that when it is detected that the object to be detected is in a moving state, the electronic device obtains the screen to be detected, and when it is detected that the object to be detected is in a stationary state, the electronic device does not obtain the screen to be detected.

[0041] In some embodiments, the electronic device may also acquire the image to be detected when a camera connected to the electronic device or a camera built into the electronic device detects the appearance of an object to be detected.

[0042] In some implementations, the electronic device may also directly acquire the picture to be detected.

[0043] Step S120: determining an object frame corresponding to the object to be detected from the picture to be detected, and determining a plurality of key points of the object to be detected from the picture to be detected.

[0044] In some embodiments, the screen to be detected may include one object to be detected, or may include multiple objects to be detected. It should be noted that the object to be detected refers to the target object to be detected in the screen to be detected. Specifically, the object to be detected may include humans or animals, etc., which are not limited here. When the object to be detected is a human, the electronic device can determine the object frame corresponding to the object to be detected from the screen to be detected by a human body detection algorithm. As a way, the electronic device can obtain the object frames corresponding to all objects to be detected in the screen to be detected, and determine the multiple key points of the object to be detected by a human body key point detection algorithm for the objects to be detected corresponding to all object frames. As another way, the user can select the object frame corresponding to one of the objects to be detected or the object frames corresponding to several objects to be detected from the object frames corresponding to multiple objects to be detected in the electronic device, wherein the specific number selected is not limited here, and the object to be detected corresponding to one object frame selected or the object to be detected corresponding to several object frames is determined by a human body key point detection algorithm for the multiple key points of the selected object to be detected. Wherein, the number of key points of the object to be detected is not limited here.

[0045] See also Figure 3 , Figure 3 A schematic diagram of multiple key points of an object to be detected provided in an embodiment of the present application is shown. Figure 3The image includes an object to be detected 300 and a head key point 301, a left shoulder key point 302, a right shoulder key point 303, a left elbow key point 304, a right elbow key point 305, a left wrist key point 306, a right wrist key point 307, a left hip key point 308, a right hip key point 309, a left knee key point 310, a right knee key point 311, a left ankle key point 312 and a right ankle key point 313.

[0046] Step S130: Calculate the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected.

[0047] In some embodiments, the electronic device can monitor all the images in the acquired image to be detected, track and monitor the object to be detected that appears in the image to be detected, and calculate the activity consumption data of the object to be detected based on the changes in the object frame corresponding to the object to be detected and the changes in multiple key points of the object to be detected. Among them, the activity consumption data may include the activity amount change data of the object to be detected, the number of steps of the object to be detected, the calorie consumption of the object to be detected, and the heart rate value of the object to be detected. It can be understood that the activity consumption data in this embodiment refers to the activity amount change data of the object to be detected, and specifically can be the activity amount change data of the object to be detected during the time it appears in the image to be detected.

[0048] In some embodiments, the electronic device may select a portion of the acquired image to be detected to detect the object to be detected that appears in the selected portion of the image, and calculate the activity consumption data of the object to be detected based on the changes in the object frame corresponding to the detected object to be detected and the changes in multiple key points of the object to be detected. The selection of the portion of the image may be performed by the user on the electronic device.

[0049] An activity detection method, device, electronic device and storage medium provided by an embodiment of the present application obtain a picture to be detected, determine an object frame corresponding to an object to be detected from the picture to be detected, and determine multiple key points of the object to be detected from the picture to be detected, and calculate the activity amount of the object to be detected based on monitored changes in the object frame corresponding to the object to be detected and changes in the multiple key points of the object to be detected. This eliminates the need for the object to be detected to wear a detection device and does not require the object to be detected to obtain personal information, thereby achieving determination of the activity amount of the object to be detected, while simplifying calculations related to activity detection and improving calculation speed.

[0050] See also Figure 4 , Figure 4FIG. 1 is a flow chart of an activity detection method provided by an embodiment of the present application. In this embodiment, the image to be detected includes at least two frames of images. Figure 4 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0051] Step S210: Acquire the image to be detected.

[0052] Step S220: determining an object frame corresponding to the object to be detected from the picture to be detected, and determining a plurality of key points of the object to be detected from the picture to be detected.

[0053] For the detailed description of step S210 to step S220, please refer to step S110 to step S120, which will not be repeated here.

[0054] Step S230: obtaining the average size of the object frame corresponding to the object to be detected in each of two adjacent frames of images.

[0055] In this embodiment, an object frame corresponding to the object to be detected is obtained, the object frame is detected in every two adjacent frames of images, and the average size of the object frame in every two adjacent frames of images is obtained.

[0056] In some embodiments, the formula for calculating the average size of the object frame in each of two adjacent frames of images can be: Where M represents the average size of the object frame in each of two adjacent frames, w1 represents the width of the object frame in the previous frame, h1 represents the height of the object frame in the previous frame, w2 represents the width of the object frame in the next frame, and h2 represents the height of the object frame in the next frame. Figure 5 As shown, the width of the previous frame object frame 310 is w1 and the height is h1, and the width of the next frame object frame 320 is w2 and the height is h2.

[0057] It can be understood that when calculating the average size of the object frame corresponding to the object to be detected in the three frames of images, the average size of the object frame in the two adjacent frames of the first frame and the second frame can be calculated, and then the average size of the object frame in the two adjacent frames of the second frame and the third frame can be calculated. Based on the average size of the object frame calculated above, the average size of the object frame in each two adjacent frames is calculated.

[0058] Step S240: obtaining the displacements of a plurality of key points of the object to be detected in each of two adjacent frames of images.

[0059] In this embodiment, by obtaining the object frame corresponding to the object to be detected, multiple key points of the object to be detected are confirmed, and the displacement of the multiple key points of the object to be detected in each two adjacent frames of images is obtained. Wherein, displacement refers to the change in position of an object. It can be understood that the displacement in this embodiment refers to the change in position between the key points corresponding to the object to be detected in two adjacent frames of images, which can be specifically the difference between the position of the key points corresponding to the object to be detected in the current frame image and the position of the key points corresponding to the object to be detected in the next frame image.

[0060] In some embodiments, the formula for calculating the displacement of multiple key points of the object to be detected in each of two adjacent frames of images can be: Among them, S represents the displacement of multiple key points of the object to be detected in each two adjacent frames of images, z represents that the object to be detected has z key points, x1 i Represented as the x-coordinate of the i-th key point of the object to be detected in the previous frame, x2 i Represented as the x-coordinate of the i-th key point of the object to be detected in the next frame, y1 i Represented as the y coordinate of the i-th key point of the object to be detected in the previous frame, y2 i It is represented as the y coordinate of the i-th key point of the object to be detected in the next frame. Figure 5 As shown, the coordinates of the key point 301A of the previous frame are (x11, y11), the coordinates of the key point 301B of the next frame are (x21, y21), S1 is the displacement of the key point 301 of the object to be detected in each two adjacent frames, the coordinates of the key point 302A of the previous frame are (x12, y12), the coordinates of the key point 302B of the next frame are (x22, y22), S2 is the displacement of the key point 302 of the object to be detected in each two adjacent frames, so the displacement of multiple key points of the object to be detected in each two adjacent frames is

[0061] It can be understood that when calculating multiple key points of the object to be detected in three frames of images, the displacement of multiple key points of the object to be detected in two adjacent frames of the first frame and the second frame is calculated, and then the displacement of multiple key points of the object to be detected in two adjacent frames of the second frame and the third frame is calculated. According to the displacement of the multiple key points of the object to be detected in each two adjacent frames of images calculated above, the displacement of the multiple key points of the object to be detected in the three frames of images is calculated.

[0062] Step S250: Calculate the activity consumption data of the object to be detected based on the average size of the object frame in each two adjacent frames of the image and the displacement of multiple key points of the object to be detected in each two adjacent frames of the image.

[0063] In this embodiment, the activity consumption data of the object to be detected can be obtained by calculation based on the average size of the object frame corresponding to the object to be detected in each two adjacent frames of the image, and the displacement of multiple key points of the object to be detected in each two adjacent frames of the image.

[0064] In some embodiments, the electronic device can directly calculate the average size of the object frame corresponding to the object to be detected in each two adjacent frames of the image and the displacement of multiple key points of the object to be detected in each two adjacent frames of the image to obtain the activity consumption data of the object to be detected.

[0065] In some embodiments, when the electronic device detects that the object to be detected stops moving, it can also calculate the average size of the object frame corresponding to the object to be detected in each two adjacent frames of the image, and the displacement of multiple key points of the object to be detected in each two adjacent frames of the image, to obtain the activity consumption data of the object to be detected.

[0066] In some embodiments, a user can set the detection in the electronic device to calculate the activity amount of the object to be detected after a period of time, without obtaining data from two frames of images to calculate the activity amount of the object to be detected. After a period of time, the average size of the object box corresponding to the object to be detected obtained during this period in each two adjacent frames of images, as well as the displacement of multiple key points of the object to be detected in each two adjacent frames of images can be calculated to obtain the activity consumption data of the object to be detected, which can reduce the amount of calculation.

[0067] See also Figure 6 , Figure 6 This application shows Figure 4 The flowchart of step S250 of the method for counting people is shown in FIG. Figure 6 The process shown is described in detail, and the method may specifically include the following steps:

[0068] Step S251: obtaining the relative displacement of the object to be detected between each two adjacent frames of images by using the ratio of the displacement of multiple key points of the object to be detected corresponding to each two adjacent frames of images to the average size of the object frame corresponding to the object to be detected.

[0069] In some embodiments, the relative displacement of the object to be detected in each of two adjacent frames of images is obtained by using the ratio of the displacement of the multiple key points of the object to be detected corresponding to each of two adjacent frames of images to the average size of the object frame corresponding to the object to be detected, that is, the displacement of the multiple key points of the object to be detected corresponding to each of two adjacent frames of images is divided by the average size of the object frame corresponding to the object to be detected, which can be expressed as a calculation formula Wherein, D is the relative displacement of the object to be detected in each two adjacent frames, S is the displacement of multiple key points of the object to be detected corresponding to each two adjacent frames, and M is the average size of the object frame corresponding to the object to be detected. For example, the image to be detected includes three frames, the displacement of multiple key points of the object to be detected corresponding to the two adjacent frames of the first frame and the second frame is S1, the displacement of multiple key points of the object to be detected in the two adjacent frames of the second frame and the third frame is S2, the average size of the object frame in the two adjacent frames of the first frame and the second frame is M1, and the average size of the object frame in the two adjacent frames of the second frame and the third frame is M2, then the relative displacement of the two adjacent frames of the first frame and the second frame is Then the relative displacement between the first frame image and the second frame image is

[0070] Step S252: Accumulate the relative displacement of the object to be detected in every two adjacent frames of images to obtain activity consumption data of the object to be detected.

[0071] In some embodiments, the relative displacement of the object to be detected in each of two adjacent frames of images is accumulated to obtain the activity consumption data of the object to be detected. The formula for calculating the activity consumption data can be: Among them, A represents the activity consumption data of the object to be detected, NUM represents the number of frames in which the object to be detected appears in the image to be detected, and D i It is expressed as the relative displacement of the object to be detected in the i-th adjacent frame image.

[0072] Compared with the activity detection method provided by an embodiment of the present application, Figure 1 The activity detection method shown in this embodiment can obtain the activity consumption data of the object to be detected by obtaining the average size of the object frame in each two adjacent frames of images, and the displacement of multiple key points of the object to be detected in each two adjacent frames of images, thereby simplifying the relevant calculations of activity detection, and at the same time, there is no need for the object to be detected to wear relevant equipment for detecting the amount of activity.

[0073] See also Figure 7 , Figure 7 FIG. 1 is a flow chart of an activity detection method provided by an embodiment of the present application. Figure 7 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0074] Step S310: Acquire the image to be detected;

[0075] Step S320: determining an object frame corresponding to the object to be detected from the picture to be detected, and determining a plurality of key points of the object to be detected from the picture to be detected.

[0076] Step S330: Calculate the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected.

[0077] For the detailed description of step S310 to step S320, please refer to step S110 to step S130, which will not be repeated here.

[0078] Step S340: obtaining a duration during which the object to be detected exists in the picture to be detected as a first duration.

[0079] In some embodiments, the electronic device can directly obtain the duration of time that the object to be detected exists in the screen to be detected, and use the obtained duration as the first duration. The electronic device can be provided with a display screen, which can display the first duration to prompt the user of the duration of the appearance of the object to be detected. As a method, the electronic device can also obtain the duration of time that the object to be detected exists in the screen to be detected after obtaining the activity amount of the object to be detected, and use the obtained duration as the first duration. As another method, the electronic device can also obtain the duration of time that the object to be detected exists in the screen to be detected when the average activity amount is required.

[0080] See also Figure 8 , Figure 8 This application shows Figure 7 The flowchart of step S340 of the method for counting people is shown in FIG. Figure 8 The process shown is described in detail, and the method may specifically include the following steps:

[0081] Step S341: Acquire the number of frames in which the object to be detected exists in the picture to be detected, and the frame rate of the picture to be detected.

[0082] In some embodiments, the electronic device can directly obtain the number of frames in which the object to be detected exists in the picture to be detected and the frame rate of the picture to be detected. The electronic device can also obtain the number of frames in which the object to be detected exists in the picture to be detected and the frame rate of the picture to be detected after obtaining the activity amount of the object to be detected. The electronic device can also obtain the number of frames in which the object to be detected exists in the picture to be detected and the frame rate of the picture to be detected when the average activity amount is needed.

[0083] Step S342: Based on the number of frames and the frame rate, the duration for which the object to be detected exists in the picture to be detected is obtained as the first duration.

[0084] In this embodiment, the duration of the existence of the object to be detected in the picture to be detected is calculated according to the number of frames in which the object to be detected exists in the picture to be detected and the frame rate of the picture to be detected. It can be understood that the duration of the existence of the object to be detected in the picture to be detected is the number of hours that the object to be detected exists in the picture to be detected. The calculation formula is: Wherein, H represents the duration of the existence of the object to be detected in the picture to be detected, that is, the first duration, NUM represents the number of frames in which the object to be detected appears in the picture to be detected, and FPS represents the frame rate of the picture to be detected. Wherein, the first duration refers to the duration of the existence of the object to be detected in the picture to be detected.

[0085] Step S350: Obtain the average activity amount of the object to be detected by using the ratio of the activity consumption data of the object to be detected to the first duration.

[0086] In some embodiments, the average activity amount of the object to be detected can be obtained by using the ratio of the activity consumption data of the object to be detected to the first duration, that is, the activity amount of the object to be detected divided by the first duration. The formula for calculating the average activity amount can be: Among them, L represents the average activity of the detection object, A represents the activity of the object to be detected, and H represents the duration that the object to be detected exists in the picture to be detected, which is the first duration.

[0087] Compared with the activity detection method provided by an embodiment of the present application, Figure 1 The activity detection method shown in this embodiment can calculate the average activity of the object to be detected by the activity amount of the object to be detected and the duration of the object to be detected in the image to be detected, thereby determining the average activity amount of the object to be detected and simplifying the relevant calculations of activity detection.

[0088] See also Fig. 9 , Fig. 9 The flowchart of the activity detection method provided by an embodiment of the present application is shown. In this embodiment, the image to be detected is used to represent the image of the space to be detected, and the space to be detected includes multiple objects to be detected. Fig. 9 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0089] Step S410: Acquire the image to be detected.

[0090] For the detailed description of step S410, please refer to step S110, which will not be repeated here.

[0091] Step S420: determining object frames corresponding to the multiple objects to be detected respectively from the picture to be detected, and determining multiple key points of the multiple objects to be detected respectively from the picture to be detected.

[0092] In some embodiments, the screen to be detected may include multiple objects to be detected, and the electronic device can determine object frames corresponding to multiple objects to be detected from the screen to be detected through a human body detection algorithm. The electronic device can obtain object frames corresponding to all objects to be detected in the screen to be detected, and determine multiple key points corresponding to all objects to be detected corresponding to all object frames through a human body key point detection algorithm, wherein the number of key points of the objects to be detected is not limited here.

[0093] Step S430: Calculate the total activity consumption data of the multiple objects to be detected based on the changes of the object frames corresponding to the multiple objects to be detected and the changes of the multiple key points of the multiple objects to be detected.

[0094] In some embodiments, the electronic device can monitor the screen where multiple objects to be detected appear in the screen to be detected, monitor the changes of the object frames corresponding to the multiple objects to be detected, and the changes of multiple key points of the multiple objects to be detected, calculate the relative displacement of the multiple objects to be detected appearing in each two frames of the screen in each adjacent two frames, and obtain the total activity consumption data of the multiple objects to be detected. The calculation formula for the relative displacement of the multiple objects to be detected in each adjacent two frames of the image can be: Where DD represents the relative displacement of multiple objects to be detected in each two adjacent frames of images, N represents the number of objects to be detected in the image to be detected, and D j It is expressed as the displacement of multiple key points of the jth object to be detected in each two adjacent frames of images, and then the total activity consumption data of multiple objects to be detected are calculated according to the average relative displacement of the multiple objects to be detected. The calculation formula can be Among them, A represents the total activity consumption data of multiple objects to be detected, NUM represents the number of frames in which any object to be detected appears in the image to be detected, and DD i It is expressed as the relative displacement of the object to be detected in the i-th pair of adjacent frames.

[0095] In some embodiments, the electronic device can monitor the changes of the object frames corresponding to the multiple objects to be detected and the changes of the multiple key points of the multiple objects to be detected in the picture where the multiple objects to be detected appear in the picture to be detected, calculate the relative displacement of the multiple objects to be detected appearing in every two frames in each adjacent two frames, and divide it by the number of objects to be detected appearing in the picture to be detected to obtain the average relative displacement of the multiple objects to be detected. The calculation formula for the average relative displacement of the multiple objects to be detected can be Where DD represents the average relative displacement of multiple objects to be detected, D jIt is represented by the displacement of multiple key points of the jth object to be detected in each two adjacent frames of the image, N represents the number of objects to be detected in the image to be detected, and then the total activity consumption data of multiple objects to be detected is calculated based on the average relative displacement of the multiple objects to be detected. The calculation formula is: Among them, A represents the total activity consumption data of multiple objects to be detected, NUM represents the number of frames in which any object to be detected appears in the image to be detected, and DD i It is expressed as the average relative displacement of the object to be detected in the i-th pair of adjacent frames.

[0096] Step S440: obtaining a time duration during which any object to be detected among the plurality of objects to be detected and calculated exists in the picture to be detected as a second time duration.

[0097] In some embodiments, the electronic device can directly obtain the duration of any object to be detected existing in the image to be detected, and use the obtained duration as the second duration, wherein the second duration refers to the total duration of all objects to be detected in the image to be detected appearing in the image to be detected.

[0098] In some embodiments, the electronic device may be provided with a display screen, which may display the second duration to prompt a user of the duration for which multiple objects to be detected appear. As one method, the electronic device may also obtain the duration for which multiple objects to be detected exist in the screen to be detected after obtaining the activity amounts of multiple objects to be detected, and use the obtained duration as the second duration. As another method, the electronic device may also obtain the duration for which multiple objects to be detected exist in the screen to be detected when the average activity amount of the space to be detected is required.

[0099] In some embodiments, the number of frames in which any object to be detected exists in the picture to be detected and the frame rate of the picture to be detected are obtained, and the duration of the object to be detected existing in the picture to be detected is calculated. It can be understood that the duration of the object to be detected existing in the picture to be detected is the number of hours that the object to be detected exists in the picture to be detected. The calculation formula can be: Wherein, H represents the duration that any object to be detected exists in the picture to be detected, that is, the second duration, NUM represents the number of frames in which the object to be detected appears in the picture to be detected, and FPS represents the frame rate of the picture to be detected.

[0100] Step S450: using the ratio of the total activity amount of the plurality of objects to be detected to the second time length, obtaining the average activity amount of the plurality of objects to be detected in the space to be detected.

[0101] In some embodiments, the average activity of the space to be detected can be obtained by using the ratio of the total activity of the multiple objects to be detected to the second time length, that is, the total activity of the multiple objects to be detected divided by the second time length. The calculation formula can be: Among them, L represents the average activity of the detection object, A represents the activity consumption data of multiple objects to be detected, and H represents the duration that any object to be detected exists in the picture to be detected, which is the second duration.

[0102] Compared with the activity detection method provided by an embodiment of the present application, Figure 1 The activity detection method shown in this embodiment can detect multiple objects to be detected in a picture to be detected, and determine the activity of the multiple objects to be detected by monitoring the changes in the object frames corresponding to the multiple objects to be detected and the changes in the multiple key points of the multiple objects to be detected. Then, the duration of the presence of any object to be detected in the picture to be detected is obtained, and the average activity of the space to be detected is obtained, thereby determining the average activity of the space to be detected. The average activity of the space to be detected can be calculated without identifying the objects to be detected, and at the same time, the relevant calculations of activity detection are simplified, thereby improving the calculation speed.

[0103] See also Fig.10 , Fig.10 FIG. 1 is a flow chart of an activity detection method provided by an embodiment of the present application. Fig.10 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0104] Step S510: Acquire the picture to be detected.

[0105] For the detailed description of step S510, please refer to step S110, which will not be repeated here.

[0106] Step S520: obtaining an object to be determined from the image to be detected, and extracting feature information of the object to be determined.

[0107] In some implementations, the object to be determined may be acquired from the picture to be detected by a human body detection algorithm, and feature information of the object to be determined acquired from the picture to be detected may be extracted by a human body recognition algorithm.

[0108] Step S530: When the feature information of the object to be determined matches the preset feature information, the object to be determined is determined as the object to be detected.

[0109] In some embodiments, the electronic device may pre-set and store preset feature information and a similarity threshold, compare the feature information of the object to be determined with the preset feature information, obtain the similarity, and compare the similarity with the similarity threshold. When the similarity between the feature information of the object to be determined and the preset feature information is greater than the similarity threshold, it means that the feature information of the object to be determined matches the preset feature information, and the object to be determined is determined as the object to be detected; when the similarity between the feature information of the object to be determined and the preset feature information is less than the similarity threshold, it means that the feature information of the object to be determined does not match the preset feature information, and the object to be determined is not determined as the object to be detected.

[0110] Step S540: determining an object frame corresponding to the object to be detected from the image to be detected, and determining a plurality of key points of the object to be detected from the image to be detected.

[0111] Step S550: Calculate the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected.

[0112] For the detailed description of step S540 to step S550, please refer to step S120 to step S130, which will not be repeated here.

[0113] Compared with the activity detection method provided by an embodiment of the present application, Figure 1 The activity detection method shown in this embodiment can obtain the object to be determined from the picture to be detected and extract the characteristic information of the detection object, identify the object to be detected according to the preset characteristic information, and thus realize the detection of the activity amount of the object to be detected separately by identifying the object to be detected in the picture to be detected.

[0114] See also Fig.11 , Fig.11 FIG. 1 is a flow chart of an activity detection method provided by an embodiment of the present application. Fig.11 The process shown is described in detail, and the activity detection method may specifically include the following steps:

[0115] Step S610: Acquire the picture to be detected.

[0116] For the detailed description of step S610 , please refer to step S110 , which will not be repeated here.

[0117] Step S620: Based on a human body detection algorithm, determine an object frame corresponding to the object to be detected from the image to be detected.

[0118] In some implementations, the object frame of the object to be detected can be determined from the image to be detected by a human body detection algorithm, wherein the human body detection algorithm may include a YOLO (You Only Look Once) algorithm and an SSD (Single Shot Multi Box Detector) algorithm, etc., which are not limited here.

[0119] Step S630: Based on a human body key point detection algorithm, a plurality of key points of the object to be detected are determined from the image to be detected.

[0120] In some implementations, a plurality of key points of an object to be detected may be determined from a picture to be detected by using a human key point detection algorithm, wherein the human key point detection algorithm may include an OpenPose algorithm, an MSCOCO algorithm, an MPII algorithm, etc., which are not limited here.

[0121] Step S640: Calculate the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected.

[0122] For the detailed description of step S640, please refer to step S130, which will not be repeated here.

[0123] Compared with the activity detection method provided by an embodiment of the present application, Figure 1 The activity detection method shown in this embodiment can confirm the object frame of the object to be detected through a human body detection algorithm and determine multiple key points of the object to be detected through a key point detection algorithm, and calculate the activity amount of the object to be detected by monitoring the changes in the object frame corresponding to the object to be detected and the changes in the multiple key points of the object to be detected, so as to determine the activity amount of the object to be detected by analyzing the picture to be detected, without the need for the object to be detected to wear relevant activity detection equipment and without the need to obtain the personal information of the object to be detected, thereby protecting the privacy information of the object to be detected.

[0124] The present application also provides an application scenario, and the above activity detection method can be applied in the application scenario as follows:

[0125] After the camera in the smart home system captures an image, it performs an activity check on the image through a gateway, server or terminal. After calculating the amount of human activity, it generates an exercise report and sends it to the user terminal. Further, based on the amount of human activity and user identity information such as age, appropriate exercise guidance is pushed to the user, such as indoor yoga, aerobics and other recommended exercises. If the user issues a corresponding exercise command, the various smart devices in the current home environment can be controlled to execute the scene mode corresponding to the exercise.

[0126] After the camera in the smart home system captures the image, it checks the activity of the image through the gateway, server or terminal. After calculating the amount of human activity, it can also output a daily exercise report to the user and give reasonable suggestions and adjustments based on the user's daily exercise amount.

[0127] After the camera in the smart home system captures an image, it performs an activity check on the image through a gateway, server, or terminal. After calculating multiple human activity amounts in the captured image, the multiple human activity amounts are sorted, and an exercise amount ranking list can be output to the user.

[0128] See also Fig.12 , Fig.12 The module block diagram of the activity detection device 400 provided in the embodiment of the present application is shown in FIG. Fig.12 The block diagram shown in FIG. 4 is used to illustrate that the activity detection device 400 includes: a detection picture acquisition module 410, a corresponding data determination module 420 for the object to be detected, and an activity amount calculation module 430, wherein:

[0129] The to-be-detected picture acquisition module 410 is used to acquire the to-be-detected picture.

[0130] The module 420 for determining data corresponding to the object to be detected is used to determine an object frame corresponding to the object to be detected from the picture to be detected, and to determine a plurality of key points of the object to be detected from the picture to be detected.

[0131] Furthermore, the to-be-detected object corresponding data determination module 420 includes: a plurality of to-be-detected object corresponding information determination submodules, wherein:

[0132] The submodule for determining data corresponding to multiple objects to be detected is used to determine object frames corresponding to each of the multiple objects to be detected from the picture to be detected, and to determine multiple key points of each of the multiple objects to be detected from the picture to be detected.

[0133] Furthermore, the to-be-detected object corresponding data determination module 420 further includes: a feature information extraction submodule and a to-be-detected object determination submodule, wherein:

[0134] The feature information extraction submodule is used to obtain the object to be determined from the picture to be detected and extract feature information of the object to be determined.

[0135] The to-be-detected object determination submodule is used to determine the to-be-determined object as the to-be-detected object when the feature information of the to-be-determined object matches the preset feature information.

[0136] Furthermore, the to-be-detected object corresponding data determination module 420 further includes: an object frame determination submodule and a key point determination submodule, wherein:

[0137] The object frame determination submodule is used to determine the object frame corresponding to the object to be detected from the picture to be detected based on a human body detection algorithm.

[0138] The key point determination submodule is used to determine a plurality of key points of the object to be detected from the image to be detected based on a human key point detection algorithm.

[0139] The activity calculation module 430 is used to calculate the activity consumption data of the object to be detected based on the changes of the object frame corresponding to the object to be detected and the changes of multiple key points of the object to be detected obtained by tracking and monitoring the object to be detected.

[0140] Furthermore, the activity amount calculation module 430 includes: an object frame average size acquisition submodule, a plurality of key point displacement acquisition submodules and an activity amount acquisition submodule, wherein:

[0141] The object frame average size acquisition submodule is used to obtain the average size of the object frame corresponding to the object to be detected in each of two adjacent frames of images.

[0142] The multiple key point displacement acquisition submodule is used to acquire the displacement of multiple key points of the object to be detected in each two adjacent frames of images.

[0143] The activity acquisition submodule is used to calculate the activity consumption data of the object to be detected based on the average size of the object frame corresponding to the object to be detected in each two adjacent frames of the image, and the displacement of multiple key points of the object to be detected in each two adjacent frames of the image.

[0144] Furthermore, the activity amount acquisition submodule includes: a relative displacement acquisition unit and an activity amount acquisition unit of the object to be detected, wherein:

[0145] The relative displacement acquisition unit is used to obtain the relative displacement of the object to be detected in each two adjacent frames of images by using the ratio of the displacement of multiple key points of the object to be detected corresponding to each two adjacent frames of images to the average size of the object frame corresponding to the object to be detected.

[0146] The activity amount acquisition unit of the object to be detected is used to accumulate the relative displacement of the object to be detected in each two adjacent frames of images to obtain the activity consumption data of the object to be detected.

[0147] Furthermore, the activity volume calculation module 430 further includes: a total activity volume acquisition submodule, a second duration acquisition submodule, and an average activity volume acquisition submodule of the space to be detected, wherein:

[0148] The total activity acquisition submodule is used to calculate the total activity consumption data of the multiple objects to be detected based on the changes in the object frames corresponding to the multiple objects to be detected and the changes in the multiple key points of the multiple objects to be detected obtained by tracking and monitoring the objects to be detected.

[0149] The second duration acquisition submodule is used to acquire the duration of time that any object to be detected among the multiple objects to be detected exists in the picture to be detected as the second duration.

[0150] The average activity amount acquisition submodule of the space to be detected is used to obtain the average activity amount of the multiple objects to be detected in the space to be detected by using the ratio of the total activity amount of the multiple objects to be detected to the second time length.

[0151] Furthermore, the momentum determination device 400 further includes: a first duration acquisition module and an average activity acquisition module of the object to be detected, wherein:

[0152] The first duration acquisition module is used to acquire the duration for which the object to be detected exists in the picture to be detected as the first duration.

[0153] The average activity amount acquisition module of the object to be detected is used to obtain the average activity amount of the object to be detected by using the ratio of the activity consumption data of the object to be detected and the first time length.

[0154] Furthermore, the first duration acquisition module includes: a frame number and frame rate acquisition submodule and a first duration calculation module, wherein:

[0155] The frame number and frame rate acquisition submodule is used to acquire the frame number in which the object to be detected exists in the picture to be detected, and the frame rate of the picture to be detected.

[0156] The first duration calculation module is used to obtain the duration of time that the object to be detected exists in the picture to be detected based on the number of frames and the frame rate, as the first duration.

[0157] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0158] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0159] See also Fig.13, which shows a structural block diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0160] Among them, the processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 110 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0161] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0162] See also Fig.14, which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 500 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0163] The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 510 can be compressed, for example, in an appropriate form.

[0164] In summary, the activity detection method, device, electronic device and storage medium provided in the embodiments of the present application obtain a picture to be detected, determine an object frame corresponding to an object to be detected from the picture to be detected, and determine multiple key points of the object to be detected from the picture to be detected, and calculate the activity amount of the object to be detected based on the monitored changes in the object frame corresponding to the object to be detected and the changes in the multiple key points of the object to be detected. This eliminates the need for the object to be detected to wear a detection device and does not require the personal information of the object to be detected, thereby achieving the determination of the activity amount of the object to be detected, while simplifying the calculations related to activity detection and improving the calculation speed.

[0165] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes several instructions for a terminal (which can be an intelligent gateway, mobile phone, computer, server, air conditioner or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0167] The various embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which belong to the scope of protection of the present application.

Claims

1. An activity detection method, characterized in that: The method comprises: Acquire a picture to be detected; the picture to be detected includes at least two frames of images; Determine an object frame corresponding to the object to be detected from the picture to be detected, and determine a plurality of key points of the object to be detected from the picture to be detected; In the process of tracking and monitoring the object to be detected, obtaining an average size of an object frame corresponding to the object to be detected in each of two adjacent frames of images, and a displacement of a plurality of key points of the object to be detected in each of two adjacent frames of images; Based on the average size and the displacement, activity consumption data of the object to be detected is calculated.

2. The method according to claim 1, characterized in that The calculating the activity consumption data of the object to be detected based on the average size and the displacement includes: The relative displacement of the object to be detected between two adjacent frames of images is obtained by using the ratio of the displacement of multiple key points of the object to be detected corresponding to two adjacent frames of images to the average size of the object frame corresponding to the object to be detected; The relative displacement of the object to be detected in each two adjacent frames of images is accumulated to obtain the activity consumption data of the object to be detected.

3. The method according to claim 1, characterized in that After calculating the activity consumption data of the object to be detected based on the average size and the displacement, the method further includes: Acquire the duration of time that the object to be detected exists in the picture to be detected as a first duration; The average activity amount of the object to be detected is obtained by using the ratio of the activity consumption data of the object to be detected to the first time length.

4. The method according to claim 3, characterized in that The obtaining, as a first duration, a duration for which the object to be detected exists in the picture to be detected includes: Obtaining the number of frames in which the object to be detected exists in the picture to be detected, and the frame rate of the picture to be detected; Based on the number of frames and the frame rate, a duration during which the object to be detected exists in the picture to be detected is acquired as the first duration.

5. The method according to any one of claims 1 to 4, characterized in that: The to-be-detected picture is used to represent a picture of a to-be-detected space, wherein the to-be-detected space includes a plurality of to-be-detected objects, and the determining of an object frame corresponding to the to-be-detected object from the to-be-detected picture, and determining a plurality of key points of the to-be-detected object from the to-be-detected picture, comprises: Determine the object frames corresponding to the multiple objects to be detected from each of the images to be detected, and determine the multiple key points of the multiple objects to be detected from each of the images to be detected; Based on the average size of the object frames of the multiple objects to be detected in each of the two adjacent frames of images, and the displacement of the multiple key points of the multiple objects to be detected in each of the two adjacent frames of images, the total activity consumption data of the multiple objects to be detected are calculated.

6. The method according to claim 5, characterized in that After calculating the total activity consumption data of the plurality of objects to be detected based on the average size of the object frames of the plurality of objects to be detected in each of the two adjacent frames of images and the displacement of the plurality of key points of the plurality of objects to be detected in each of the two adjacent frames of images, the method further includes: Acquire a time duration during which any object to be detected among the multiple objects to be detected exists in the picture to be detected as a second time duration; The average activity amount of the multiple objects to be detected in the space to be detected is obtained by using the ratio of the total activity amount of the multiple objects to be detected to the second time length.

7. The method according to any one of claims 1 to 4, characterized in that: The step of determining an object frame corresponding to the object to be detected from the picture to be detected, and determining a plurality of key points of the object to be detected from the picture to be detected includes: Acquire the object to be determined from the picture to be detected, and extract feature information of the object to be determined; When the feature information of the object to be determined matches the preset feature information, the object to be determined is determined as the object to be detected.

8. The method according to any one of claims 1 to 4, characterized in that: The step of determining an object frame corresponding to the object to be detected from the picture to be detected, and determining a plurality of key points of the object to be detected from the picture to be detected includes: Based on a human body detection algorithm, determining an object frame corresponding to the object to be detected from the image to be detected; Based on a human body key point detection algorithm, multiple key points of the object to be detected are determined from the image to be detected.

9. An activity detection device, characterized in that: The device comprises: A module for acquiring a picture to be detected, used for acquiring a picture to be detected; the picture to be detected includes at least two frames of images; A module for determining data corresponding to the object to be detected, used to determine an object frame corresponding to the object to be detected from the picture to be detected, and to determine a plurality of key points of the object to be detected from the picture to be detected; The activity calculation module is used to obtain the average size of the object frame corresponding to the object to be detected in each two adjacent frames of images, and the displacement of multiple key points of the object to be detected in each two adjacent frames of images during the process of tracking and monitoring the object to be detected; based on the average size and the displacement, calculate the activity consumption data of the object to be detected.

10. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 8.

Images