Human motion recognition method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure provide a method, device, electronic device and computer-readable medium for human motion recognition, the method comprising: obtaining a video to be recognized; performing key point detection on the nth video frame and the n+1th video frame to obtain skeleton key point data of a target human body in the nth video frame and skeleton key point data in the n+1th video frame; tracking a target human body according to the skeleton key point data of the target human body in the nth video frame to obtain a predicted position of the target human body in the n+1th video frame; determining a skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; determining a motion recognition result of the target human body according to the skeleton key point trajectory list of the target human body. The methods, devices, electronic devices and computer-readable medium for human motion recognition provided by the embodiments of the present disclosure can obtain accurate motion recognition results.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a method, device, electronic device and computer-readable medium for human motion recognition. Background Art

[0002] At present, deep learning technology has achieved very impressive results in many aspects of computer vision (such as target detection and recognition). In the field of human posture and action recognition, domestic and foreign scholars have also proposed various deep learning-based network models for human posture and action recognition. Through reference and testing, it is found that these networks improve the action recognition performance of the network by increasing the depth or width of the network. However, our experiments on a large number of existing deep learning models have proved that the generalization ability of human posture estimation based on deep learning has certain limitations. At the same time, the human posture estimation algorithm based on deep learning also has the problems of huge number of parameters, high computational complexity, and poor real-time performance, so it is difficult to put it into practice.

[0003] Therefore, a new human motion recognition method, device, electronic device and computer-readable medium are needed.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure. Summary of the invention

[0005] In view of this, the embodiments of the present disclosure provide a human motion recognition method, device, electronic device and computer-readable medium, which can quickly obtain accurate motion recognition results.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to a first aspect of an embodiment of the present disclosure, a method for human motion recognition is proposed, the method comprising: obtaining a video to be recognized, the video to be recognized comprising N video frames, where N is an integer greater than 0; performing key point detection on the nth video frame and the n+1th video frame to obtain skeleton key point data of a target human body in the nth video frame and skeleton key point data in the n+1th video frame, where n<N; performing target tracking on the target human body according to the skeleton key point data of the target human body in the nth video frame to obtain a predicted position of the target human body in the n+1th video frame; determining a skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; and determining a motion recognition result of the target human body according to the skeleton key point trajectory list of the target human body.

[0008] In an exemplary embodiment of the present disclosure, the action recognition result includes a coarse classification action recognition result; wherein, determining the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body includes: obtaining a sample video and a coarse classification action annotation of the sample human body in the sample video, the coarse classification action annotation including still and moving; determining a first variance mean of the sample human body with stillness and a second variance mean of the sample human body with movement according to the skeleton key point trajectory list of the sample human body; determining a target classification threshold according to the first variance mean and the second variance mean; determining the coarse classification action recognition result of the target human body according to the skeleton key point trajectory list of the target human body and the target classification threshold.

[0009] In an exemplary embodiment of the present disclosure, determining the coarse classification action recognition result of the target human body according to the skeleton key point trajectory list of the target human body and the target classification threshold includes: determining the body center point of the target human body in N video frames according to the skeleton key point trajectory list of the target human body; determining the center point variance of the target human body according to the body center point of the target human body in N video frames; determining the coarse classification action recognition result of the target human body according to the target classification threshold and the center point variance of the target human body.

[0010] In an exemplary embodiment of the present disclosure, determining a target classification threshold based on the first variance mean and the second variance mean includes: determining the minimum value of the first variance mean and the second variance mean; determining the absolute value of the difference between the first variance mean and the second variance mean; and determining the target classification threshold based on the sum of the minimum value and the absolute value.

[0011] In an exemplary embodiment of the present disclosure, the action recognition result includes an accurate classification action recognition result; wherein, determining the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body also includes: determining the key points to be identified of the action to be accurately classified; based on the key points to be identified, sampling the skeleton key point data of the target human body in the nth video frame to obtain sampled key point data; determining a first vector and a second vector according to the sampled key point data; determining the accurate classification action recognition result of the target human body in the nth video frame according to the angle between the first vector and the second vector; determining the accurate classification action recognition result of the target human body according to the accurate classification action recognition results of the target human body in N video frames.

[0012] In an exemplary embodiment of the present disclosure, target tracking is performed on a target human body according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame, including: determining the skeleton boundary region of the target human body in the nth video frame according to the skeleton key point data of the target human body in the nth video frame; determining the reference features of the target human body in the nth video frame according to the histogram normalized features of the skeleton boundary region of the target human body in the nth video frame; determining the reference features of the target human body in the nth video frame within a preset area range of the center point position of the skeleton boundary region of the target human body in the nth video frame Randomly generate k potential positions of the target human body in the n+1th video frame, where k is an integer greater than 0; calculate and obtain reference features of the k potential positions of the target human body in the n+1th video frame; calculate k similarities between the reference features of the target human body and the reference features of the k potential positions; determine the potential position corresponding to the maximum value of the k similarities as the updated center point position of the skeleton boundary area of ​​the target human body in the n+1th video frame; determine the predicted position of the target human body in the n+1th video frame according to the updated center point position of the skeleton boundary area of ​​the target human body in the n+1th video frame.

[0013] In an exemplary embodiment of the present disclosure, determining a skeleton key point trajectory list of a target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame includes: determining the skeleton boundary area of ​​each skeleton key point data in the n+1th video frame; calculating the area overlap rate between the predicted position of the target human body in the n+1th video frame and each skeleton boundary area in the n+1th video frame; determining the skeleton key point data corresponding to the skeleton boundary area with the largest area overlap rate in the n+1th video frame as the skeleton key point data of the target human body in the n+1th video frame; and determining the skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in N video frames.

[0014] In an exemplary embodiment of the present disclosure, determining a skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in N video frames includes: if the maximum value of the area overlap rate between the predicted position of the target human body in the n+1th video frame and each skeleton boundary area in the n+1th video frame is less than an area overlap rate threshold, then determining the skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in the 1st to nth video frames; if there is a skeleton key point data in the n+1th video frame, and the area overlap rate of the predicted position of the target human body in the nth video frame is less than the area overlap rate threshold, then determining the skeleton key point data as the skeleton key point data of a newly added target human body, and determining the skeleton key point trajectory list of the newly added target human body according to the skeleton key point data of the newly added target human body in the n+1th to Nth video frames.

[0015] According to a second aspect of an embodiment of the present disclosure, a human motion recognition device is proposed, which includes: a video acquisition module, used to acquire a video to be recognized, wherein the video to be recognized includes N video frames, where N is an integer greater than 0; a key point detection module, used to perform key point detection on the nth video frame and the n+1th video frame, and obtain skeleton key point data of a target human body in the nth video frame and skeleton key point data of the n+1th video frame, where n<N; a target tracking module, used to perform target tracking on the target human body according to the skeleton key point data of the target human body in the nth video frame, and obtain a predicted position of the target human body in the n+1th video frame; a trajectory list determination module, used to determine a skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; and a motion recognition module, used to determine a motion recognition result of the target human body according to the skeleton key point trajectory list of the target human body.

[0016] According to the third aspect of an embodiment of the present disclosure, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the human motion recognition methods described above.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the human motion recognition method as described in any one of the above items is implemented.

[0018] According to the human action recognition method, device, electronic device and computer-readable medium provided by certain embodiments of the present disclosure, for N video frames in the video to be recognized, after key point detection, the target human body is tracked according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame; the skeleton key point data in the n+1th video frame can be compared with the predicted position of the target human body in the n+1th video frame to determine the skeleton key point trajectory list of the target human body, and realize the inter-frame tracking of multiple target human bodies in multiple consecutive video frames. Based on the skeleton key point trajectory list of the target human body, a multi-frame analysis of the dynamic action of the target human body can quickly obtain a more accurate action recognition result.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. The drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work.

[0021] Figure 1 The present invention is a system block diagram of a method and device for human motion recognition according to an exemplary embodiment.

[0022] Figure 2 The figure is a flow chart of a method for recognizing human motion according to an exemplary embodiment.

[0023] Figure 3 is a flow chart of a method for recognizing human motion according to another exemplary embodiment.

[0024] Figure 4 is a flow chart of a method for recognizing human motion according to yet another exemplary embodiment.

[0025] Figure 5 is a flow chart of a method for recognizing human motion according to yet another exemplary embodiment.

[0026] Figure 6 is a flow chart of a method for recognizing human motion according to yet another exemplary embodiment.

[0027] Figure 7 It is a schematic diagram of the key point data of the human skeleton.

[0028] Figure 8It is a diagram of human body movements.

[0029] Fig. 9 The figure is a block diagram of a human motion recognition device according to an exemplary embodiment.

[0030] Fig.10 A block diagram schematically shows an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0032] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0033] The accompanying drawings are only schematic illustrations of the present invention, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0034] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0035] The main problem to be solved in human gesture recognition is how to analyze and process the video sequences collected by the camera or sensor so that the computer can "understand" the human body's movements and behaviors in the video; video-based human gesture recognition has broad application prospects in security monitoring, human-computer interaction, virtual reality and other fields. For many artificial intelligence systems, human gesture recognition or human behavior understanding is indispensable. For example, for the compliance judgment of the inspection behavior of the inspection personnel in the computer room, the human gesture recognition technology can be used to identify and understand the human body's movements in the monitoring video of the specified inspection time period, automatically complete the inspection compliance check, and also automatically detect malicious and abnormal behaviors.

[0036] Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0037] Figure 1 The present invention is a system block diagram of a method and device for human motion recognition according to an exemplary embodiment.

[0038] In the system 100 of the human motion recognition method and device, the server 105 may be a server that provides various services, such as a background management server (only for example) that provides support for the human motion recognition system operated by the user using the terminal devices 101, 102, and 103 through the network 104. The background management server may analyze and process the received data such as the human motion recognition request, and feed back the processing result (such as the motion recognition result - only for example) to the terminal device.

[0039] The server 105 may be a physical server, or may be composed of multiple servers, a part of the server 105 may be used as a human motion recognition task submission system in the present disclosure, for obtaining a task to execute a human motion recognition command; and a part of the server 105 may also be used as a human motion recognition system in the present disclosure, for obtaining a video to be recognized, wherein the video to be recognized includes N video frames, where N is an integer greater than 0; key point detection is performed on the nth video frame and the n+1th video frame to obtain skeleton key point data of a target human body in the nth video frame and skeleton key point data in the n+1th video frame, where n<N; target tracking is performed on the target human body according to the skeleton key point data of the target human body in the nth video frame to obtain a predicted position of the target human body in the n+1th video frame; a skeleton key point trajectory list of the target human body is determined according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; and a motion recognition result of the target human body is determined according to the skeleton key point trajectory list of the target human body.

[0040] Figure 21 is a flowchart of a human motion recognition method according to an exemplary embodiment. The human motion recognition method provided by the embodiment of the present disclosure can be executed by any electronic device with computing and processing capabilities, such as terminal devices 101, 102, 103 and / or server 105. In the following embodiments, the server execution method is used as an example for illustration, but the present disclosure is not limited to this. The human motion recognition method provided by the embodiment of the present disclosure may include steps S202 to S210.

[0041] like Figure 2 As shown, in step S202, a video to be identified is obtained, and the video to be identified includes N video frames, where N is an integer greater than 0.

[0042] In the embodiment of the present disclosure, the video to be identified is, for example, a video stream file, or, for example, a video clip file.

[0043] In step S204, key point detection is performed on the nth video frame and the n+1th video frame to obtain skeleton key point data of the target human body in the nth video frame and skeleton key point data in the n+1th video frame, where n<N.

[0044] In the embodiment of the present disclosure, n is an integer greater than 0 and less than N. The target human body in each video frame may include one or more. The skeleton key point data of each target human body may include 18 skeleton key points. The skeleton key point data may be as follows: Figure 7 As shown, there are 18 skeleton key points numbered 0 to 17. The skeleton key point data of each target human body can be expressed as follows: [[x1, y1, score1, x2, y2, score2, …, x18, y18, score18], (xi, yi) represents the coordinates of the i-th skeleton key point, i∈[1,18], scorei represents the confidence of the key point, and scorei=0 represents that the i-th skeleton key point is invisible.

[0045] The skeleton key point data detected in n video frames belong to the target human body, and the skeleton key point data detected in n+1 video frames do not necessarily belong to the target human body.

[0046] In step S206, the target human body is tracked according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the (n+1)th video frame.

[0047] In the embodiment of the present disclosure, target tracking may be, for example, a particle filter algorithm, but the present disclosure does not make any special limitation to this.

[0048] In step S208, a skeleton key point trajectory list of the target human body is determined according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame.

[0049] In the embodiment of the present disclosure, for example, the overlap rate of the skeleton key point data in the n+1th video frame and the predicted position area of ​​the target human body in the n+1th video frame can be calculated, and the skeleton key point data of the target human body in the n+1th video frame can be determined according to the calculation result, and then the skeleton key point trajectory list of the target human body can be generated according to the skeleton key point data of the target human body in the nth and n+1th video frames.

[0050] In step S210, the action recognition result of the target human body is determined according to the skeleton key point trajectory list of the target human body.

[0051] In the disclosed embodiment, a multi-frame analysis of the dynamic motion of the target human body may be performed based on the target human body's skeleton key point trajectory list, thereby obtaining a more accurate motion recognition result.

[0052] According to the human action recognition method provided by the embodiment of the present disclosure, for N video frames in the video to be recognized, after key point detection, the target human body is tracked according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame; the skeleton key point data in the n+1th video frame can be compared with the predicted position of the target human body in the n+1th video frame to determine the skeleton key point trajectory list of the target human body, and realize the inter-frame tracking of multiple target human bodies in multiple consecutive video frames. Based on the skeleton key point trajectory list of the target human body, the dynamic action of the target human body is analyzed in multiple frames, and a relatively accurate action recognition result can be quickly obtained.

[0053] Moreover, the above method is not limited by the defects of the deep learning human posture estimation algorithm, and has simple calculation, low calculation amount and high real-time performance.

[0054] Figure 3 is a flow chart of a method for recognizing human motion according to another exemplary embodiment. Figure 2 The step S210 of the embodiment may further include steps S302 to S308. In the embodiment of the present disclosure, the action recognition result may include a coarse classification action recognition result.

[0055] like Figure 3 As shown, in step S302, a sample video and a rough classification action annotation of a sample human body in the sample video are obtained, and the rough classification action annotation includes stillness and movement.

[0056] In the disclosed embodiment, the sample human body of the sample video has coarse classification action annotations and a list of skeleton key point trajectories of the sample human body in the sample video. The sample human body and the target human body may be different human bodies.

[0057] In step S304, a first variance mean of a stationary sample human body and a second variance mean of a moving sample human body are determined according to the skeleton key point trajectory list of the sample human body.

[0058] In the embodiment of the present disclosure, a first variance list of a stationary sample human body can be determined according to a list of skeleton key point trajectories of the stationary sample human body: sj1, sj2, ..., sj m ; (wherein the variance calculation is, for example, to obtain the variance of each stationary sample human body by calculating the coordinates of the center point of the skeleton key point, and the variance calculation formula is, for example, as shown in formula (1)) According to the skeleton key point trajectory list of the moving sample human body, a second variance list of the moving sample human body is determined: sy1, sy2, ..., sy m . And the first variance list of the static sample human body is averaged to obtain the first variance mean of the static sample human body; the second variance list of the moving sample human body is averaged to obtain the second variance mean of the moving sample human body. Wherein, m is the number of elements in the skeleton key point trajectory list of the sample human body, that is, the number of video frames in which the sample human body appears in the sample video.

[0059]

[0060] Among them, s 2 is the variance, s is the standard deviation, X i is the i-th element in the skeleton key point trajectory list (i.e. the horizontal or vertical coordinate of the center point), is the mean, calculated as follows:

[0061]

[0062] In step S306, a target classification threshold is determined according to the first variance mean and the second variance mean.

[0063] Among them, the minimum value of the first variance mean and the second variance mean can be determined; the absolute value of the difference between the first variance mean and the second variance mean can be determined; and the target classification threshold can be determined according to the sum of the minimum value and the absolute value. Specifically, the target classification threshold can be calculated by the following formula.

[0064] threshold=min(mean0,mean1)+abs(mean0-mean1) (3)

[0065] Among them, mean0 is the first variance mean, and mean1 is the second variance mean.

[0066] In step S308, a coarse classification action recognition result of the target human body is determined according to the skeleton key point trajectory list of the target human body and the target classification threshold.

[0067] In the disclosed embodiment, the center point of the target human body in N video frames can be determined based on the target human body's skeleton key point trajectory list; the center point variance of the target human body can be determined based on the target human body's center point in N video frames; and the first action labeling recognition result of the target human body can be determined based on the target classification threshold and the center point variance of the target human body.

[0068] The calculation method of the central point variance of the target human body can refer to the calculation method of formula (1). If the central point variance of the target human body is greater than the target classification threshold, the rough classification action recognition result of the target human body is confirmed to be moving, otherwise it is still.

[0069] According to the human motion recognition method provided by the embodiment of the present disclosure, a target classification threshold is obtained based on statistical calculation of sample videos, and then the skeleton key point trajectory list of the target human body is judged according to the target classification threshold to obtain a coarse classification recognition result of the target human body. Accurate coarse classification recognition results can be obtained based on accurate target classification thresholds, thereby improving the reliability of classification results.

[0070] Figure 4 4 is a flowchart of a method for human motion recognition according to another exemplary embodiment. The method for human motion recognition provided by the embodiment of the present disclosure may include steps S402 to S410. In the embodiment of the present disclosure, the motion recognition result may also include an accurate classification motion recognition result.

[0071] like Figure 4 As shown, in step S402, key points to be identified for the action to be accurately classified are determined.

[0072] In the disclosed embodiment, the action to be precisely classified may be determined based on the recognition result of the coarsely classified action of the target human body; and the key points to be recognized of the target human body in each action to be precisely classified may be determined.

[0073] For example, when the result of the rough classification action recognition is still, it can be determined that the actions to be accurately classified at least include: standing, sitting. When the result of the rough classification action recognition is moving, it can be determined that the actions to be accurately classified at least include: walking, jumping. For another example, for the action to be accurately classified: sitting, its key points to be recognized are, for example, Figure 7 Points 1, 8, 9, 10, 11, 12, 13 are shown. A schematic diagram of key point data of a standing human skeleton can be, for example, Figure 8 As shown in (a), the schematic diagram of key point data of a sitting human skeleton can be, for example, Figure 8 As shown in (b), the schematic diagram of key point data of a walking human skeleton can be, for example, Figure 8 (c) as shown.

[0074] In step S404, based on the key points to be identified, the skeleton key point data of the target human body in the nth video frame is sampled to obtain sampled key point data.

[0075] In step S406, a first vector and a second vector are determined according to the sampled key point data.

[0076] Among them, Figure 7 As shown, taking the action to be accurately classified: sitting as an example, the first vector is, for example, point 1 to point 8, and the second vector is point 8 to point 9. For another example, the first vector is point 1 to point 11, and the second vector is point 11 to 12. For another example, the first vector is point 9 to point 8, and the second vector is point 9 to 10. For another example, the first vector is point 12 to point 11, and the second vector is point 12 to 13. The first vector and the second vector may include one or more of the above enumerated cases.

[0077] In step S408, the accurate classification action recognition result of the target human body in the nth video frame is determined according to the angle between the first vector and the second vector.

[0078] Continuing with the above example, when the angle between the first vector and the second vector is close to 180 degrees (i.e., the difference from 180 degrees is less than the difference threshold), the accurate classification action recognition result is considered to be standing; when the angle between the first vector and the second vector is close to 90 degrees (i.e., the difference from 90 degrees is less than the difference threshold), the accurate classification action recognition result is considered to be sitting.

[0079] The angle r between the first vector A and the second vector B can be expressed as: r=acos(cos(r))*180 / PI.

[0080] Among them, PI=3.14159265, cos(r)=A·B / (|A|*|B|).

[0081] In step S410, the accurate classification action recognition result of the target human body is determined according to the accurate classification action recognition results of the target human body in N video frames.

[0082] According to the human motion recognition method provided by the embodiment of the present disclosure, the action to be accurately classified is determined based on the coarse classification action recognition result, which can narrow the judgment range of accurate classification, thereby saving calculation amount and improving the efficiency of motion recognition.

[0083] Figure 5 FIG. 5 is a flow chart of a method for human motion recognition according to another exemplary embodiment. The method for human motion recognition provided by the embodiment of the present disclosure may include steps S502 to S514.

[0084] like Figure 5As shown, in step S502, the skeleton boundary region of the target human body in the nth video frame is determined according to the skeleton key point data of the target human body in the nth video frame.

[0085] In the embodiment of the present disclosure, the skeleton boundary area rect is, for example, a rectangular area, and the rectangular area can be expressed as rect=[xleft, ytop, width, height].

[0086] Wherein, xleft (upper left endpoint X coordinate) = min[x1, x2, ..., x18];

[0087] Ytop (Y coordinate of the upper left endpoint) = min[y1, y2, ..., y18];

[0088] Width = |max[x1,x2,…,x18]–min[x1,x2,…,x18]|;

[0089] Height = |max[y1, y2, …, y18] – min[y1, y2, …, y18]|.

[0090] In step S504, the reference features of the target human body in the nth video frame are determined according to the histogram normalization features of the skeleton boundary region of the target human body in the nth video frame.

[0091] In the disclosed embodiment, for example, the color histogram normalization feature of the rect area is calculated by using calcHist of opencv as the reference feature F of the target human body in the nth video frame.

[0092] In step S506, k potential positions of the target human body in the n+1th video frame are randomly generated within a preset area of ​​the center point position of the skeleton boundary region of the target human body in the nth video frame, where k is an integer greater than 0.

[0093] In the embodiment of the present disclosure, the preset area range is, for example, a range with a radius r and a center point position of a skeleton boundary area of ​​a target human body in the nth video frame as the center of a circle. R can be, for example, 10, but this is only an example.

[0094] In step S508, reference features of k potential positions of the target human body in the (n+1)th video frame are calculated and obtained.

[0095] In the disclosed embodiments, for example, the potential position can be used as the center point to determine the potential boundary area of ​​each potential position (similar to the skeleton boundary area, that is, the potential boundary area is determined based on the center point, width Width and height Height), and the color histogram normalized features of the potential boundary area of ​​each potential position in the n+1th video frame are calculated to obtain the reference feature Fi of each potential position.

[0096] In step S510, k similarities between the reference features of the target human body and the reference features of the k potential positions are calculated.

[0097] In step S512, the potential position corresponding to the maximum value among the k similarities is determined as the updated center point position of the skeleton boundary region of the target human body in the (n+1)th video frame.

[0098] In step S514, the predicted position of the target human body in the n+1th video frame is determined according to the updated center point position of the skeleton boundary region of the target human body in the n+1th video frame.

[0099] In an embodiment of the present disclosure, the predicted position of the target human body in the n+1th video frame can be determined based on the updated center point position, width and height of the skeleton boundary area of ​​the target human body in the n+1th video frame. The predicted position is an area corresponding to the skeleton boundary area of ​​the target human body in the nth video frame.

[0100] Figure 6 FIG. 1 is a flow chart of a method for human motion recognition according to another exemplary embodiment. The method for human motion recognition provided by the embodiment of the present disclosure may include steps S602 to S608.

[0101] like Figure 6 As shown, in step S602, the skeleton boundary area of ​​each skeleton key point data in the (n+1)th video frame is determined.

[0102] In the embodiment of the present disclosure, the method for determining the skeleton boundary area can be found in Figure 5 Detailed explanation of step S502.

[0103] In step S604, the area overlap ratio between the predicted position of the target human body in the n+1th video frame and each skeleton boundary region in the n+1th video frame is calculated.

[0104] In step S606, the skeleton key point data corresponding to the skeleton boundary region with the maximum area overlap rate in the n+1th video frame is determined as the skeleton key point data of the target human body in the n+1th video frame.

[0105] When the maximum area overlap rate is greater than the area overlap rate threshold, the skeleton key point data corresponding to the skeleton boundary region with the maximum area overlap rate in the n+1th video frame can be determined as the skeleton key point data of the target human body in the n+1th video frame.

[0106] In step S608, a skeleton key point trajectory list of the target human body is determined according to the skeleton key point data of the target human body in N video frames.

[0107] In the disclosed embodiment, the skeleton key point data of the target human body in at least one of the first to Nth video frames can be obtained in sequence, and then a skeleton key point trajectory list of the target human body can be obtained. The skeleton key point trajectory list describes the skeleton key point data of the target human body in different video frames.

[0108] According to the human motion recognition method provided by the embodiment of the present disclosure, based on the calculation of the area overlap rate of the predicted position and each skeleton boundary area in the n+1th video frame, each target human body can be accurately tracked, and a list of skeleton key point trajectories of each target human body can be accurately generated according to the tracking results, thereby facilitating the subsequent accurate recognition of the target human body's motion based on the skeleton key point trajectory list of multiple frames.

[0109] In an exemplary embodiment, in step S608, if the maximum value of the area overlap rate between the predicted position of the target human body in the n+1th video frame and the area overlap rate of each skeleton boundary region in the n+1th video frame is less than the area overlap rate threshold, the skeleton key point trajectory list of the target human body is determined based on the skeleton key point data of the target human body in the 1st to nth video frames.

[0110] In this case, it can be considered that the target human body has disappeared since the n+1th video frame, and there is no need to track it subsequently. Therefore, the target human body's skeleton key point trajectory list can be directly determined based on the target human body's skeleton key point data in the 1st to nth video frames.

[0111] If there is a skeleton key point data in the n+1th video frame and the area overlap rate of the predicted position of the target human body in the nth video frame is less than the area overlap rate threshold, then the skeleton key point data is determined as the skeleton key point data of the newly added target human body, and the skeleton key point trajectory list of the newly added target human body is determined based on the skeleton key point data of the newly added target human body in the n+1th to Nth video frames.

[0112] In this case, it can be considered that the person corresponding to the skeleton key point data of the newly added target human body is the new person appearing in the n+1th video frame, who has not appeared before the n+1th video frame. Therefore, the skeleton key point trajectory list of the newly added target human body can be directly determined based on the skeleton key point data of the newly added target human body in the n+1th to Nth video frames.

[0113] The human motion recognition method provided in this application can solve four problems existing in the prior art.

[0114] (1) Application scope limitations: This paper aims to solve the limitations of the current motion recognition algorithm (only supporting motion recognition of a single person and a single background), realize the simultaneous motion recognition of multiple people in ordinary monitoring scenarios, and achieve the goal of real-time performance.

[0115] (2) Weak generalization ability: This paper aims to solve the problem of low generalization ability of the current mainstream human action recognition algorithm based on model training, and achieve the accuracy of human action recognition in multiple scenarios without training models (statistics + skeleton angle calculation).

[0116] (3) High computational complexity: This solution solves the problem of high computational complexity and poor real-time performance when using deep learning technology to recognize human posture and motion.

[0117] (4) High equipment requirements and high cost: This method solves the complexity of the current motion recognition algorithm, which requires a depth-of-field camera to obtain the three-dimensional coordinate information of the human body to complete motion recognition. This method can achieve the purpose of human posture and motion recognition based on ordinary surveillance cameras, thereby reducing the cost of actual applications.

[0118] It should be clearly understood that the present disclosure describes how to form and use specific examples, but the principles of the present disclosure are not limited to any details of these examples. On the contrary, based on the teachings of the contents disclosed in the present disclosure, these principles can be applied to many other embodiments.

[0119] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments are implemented as a computer program executed by a central processing unit (CPU). When the computer program is executed by the central processing unit (CPU), the above functions defined by the above method provided by the present disclosure are executed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0120] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0121] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0122] Fig. 9 is a block diagram of a human motion recognition device according to an exemplary embodiment. Fig. 9 The human motion recognition device 90 provided in the embodiment of the present disclosure may include: a video acquisition module 902 , a key point detection module 904 , a target tracking module 906 , a trajectory list determination module 908 and a motion recognition module 910 .

[0123] In the human motion recognition device 90 , the video acquisition module 902 may be used to acquire a video to be recognized, where the video to be recognized includes N video frames, where N is an integer greater than 0.

[0124] The key point detection module 904 may be used to perform key point detection on the nth video frame and the n+1th video frame to obtain skeleton key point data of the target human body in the nth video frame and skeleton key point data of the n+1th video frame, where n<N.

[0125] The target tracking module 906 may be used to track the target human body according to the skeleton key point data of the target human body in the nth video frame, and obtain the predicted position of the target human body in the (n+1)th video frame.

[0126] The trajectory list determination module 908 may be used to determine the skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame.

[0127] The action recognition module 910 may be used to determine the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body.

[0128] According to the human action recognition device provided by the embodiment of the present disclosure, for N video frames in the video to be recognized, after key point detection, the target human body is tracked according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame; the skeleton key point data in the n+1th video frame can be compared with the predicted position of the target human body in the n+1th video frame to determine the skeleton key point trajectory list of the target human body, and realize the inter-frame tracking of multiple target human bodies in multiple consecutive video frames. Based on the skeleton key point trajectory list of the target human body, a multi-frame analysis of the dynamic action of the target human body can quickly obtain a relatively accurate action recognition result.

[0129] In an exemplary embodiment, the action recognition result may include a coarse classification action recognition result; wherein, the action recognition module 910 may include: a sample video acquisition unit, which can be used to obtain a sample video and a coarse classification action annotation of a sample human body in the sample video, and the coarse classification action annotation includes static and moving; a variance calculation unit, which can be used to determine a first variance mean of a static sample human body and a second variance mean of a moving sample human body according to a list of skeleton key point trajectories of the sample human body; a target classification threshold calculation unit, which can be used to determine a target classification threshold according to the first variance mean and the second variance mean; an action recognition unit, which can be used to determine the coarse classification action recognition result of the target human body according to the list of skeleton key point trajectories of the target human body and the target classification threshold.

[0130] In an exemplary embodiment, the action recognition unit may include: a center point determination subunit, which can be used to determine the center point of the target human body in N video frames based on the skeleton key point trajectory list of the target human body; a center point variance calculation subunit, which can be used to determine the center point variance of the target human body based on the center point of the target human body in N video frames; a coarse classification action recognition subunit, which can be used to determine the coarse classification action recognition result of the target human body based on the target classification threshold and the center point variance of the target human body.

[0131] In an exemplary embodiment, the target classification threshold calculation unit may include: a variance minimum calculation subunit, which can be used to determine the minimum value of the first variance mean and the second variance mean; a difference absolute value calculation subunit, which can be used to determine the absolute value of the difference between the first variance mean and the second variance mean; a target classification threshold calculation subunit, which can be used to determine the target classification threshold based on the sum of the minimum value and the absolute value.

[0132] In an exemplary embodiment, the action recognition result may include an accurate classification action recognition result; wherein, the action recognition module 910 may also include: a key point determination unit to be identified, which can be used to determine the key points to be identified of the action to be accurately classified; a key point data sampling unit, which can be used to sample the skeleton key point data of the target human body in the nth video frame based on the key points to be identified to obtain sampled key point data; a vector determination unit, which can be used to determine the first vector and the second vector according to the sampled key point data; a first subunit of accurate classification action recognition, which can be used to determine the accurate classification action recognition result of the target human body in the nth video frame according to the angle between the first vector and the second vector; a second subunit of accurate classification action recognition, which can be used to determine the accurate classification action recognition result of the target human body according to the accurate classification action recognition results of the target human body in N video frames.

[0133] In an exemplary embodiment, the target tracking module 906 may include: an nth region determination unit, which can be used to determine the skeleton boundary region of the target human body in the nth video frame based on the skeleton key point data of the target human body in the nth video frame; a reference feature calculation unit, which can be used to determine the reference features of the target human body in the nth video frame based on the histogram normalized features of the skeleton boundary region of the target human body in the nth video frame; a potential position generation unit, which can be used to randomly generate k potential positions of the target human body in the n+1th video frame within a preset area range of the center point position of the skeleton boundary region of the target human body in the nth video frame, where k is greater than 0. integer; a potential feature calculation unit, which can be used to calculate the reference features of k potential positions of the target human body in the n+1th video frame; a similarity calculation unit, which can be used to calculate k similarities between the reference features of the target human body and the reference features of the k potential positions; an updated center point calculation unit, which can be used to determine the potential position corresponding to the maximum value of the k similarities as the updated center point position of the skeleton boundary area of ​​the target human body in the n+1th video frame; a predicted position determination unit, which can be used to determine the predicted position of the target human body in the n+1th video frame according to the updated center point position of the skeleton boundary area of ​​the target human body in the n+1th video frame.

[0134] In an exemplary embodiment, the trajectory list determination module 908 may include: an n+1th area determination unit, which can be used to determine the skeleton boundary area of ​​each skeleton key point data in the n+1th video frame; an area overlap rate calculation unit, which can be used to calculate the area overlap rate between the predicted position of the target human body in the n+1th video frame and each skeleton boundary area in the n+1th video frame; an inter-frame tracking unit, which can be used to determine the skeleton key point data corresponding to the skeleton boundary area with the largest area overlap rate in the n+1th video frame as the skeleton key point data of the target human body in the n+1th video frame; and a trajectory list determination unit, which can be used to determine the skeleton key point trajectory list of the target human body based on the skeleton key point data of the target human body in N video frames.

[0135] In an exemplary embodiment, the trajectory list determination unit may include: a target human body disappearance processing subunit, which can be used to determine the skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in the 1st to nth video frames if the maximum value of the area overlap rate between the predicted position of the target human body in the n+1th video frame and the area overlap rate of each skeleton boundary area in the n+1th video frame is less than the area overlap rate threshold; a target human body addition processing subunit, which can be used to determine the skeleton key point data as the skeleton key point data of a newly added target human body if the area overlap rate between a skeleton key point data in the n+1th video frame and the predicted position of the target human body in the nth video frame is less than the area overlap rate threshold, and determine the skeleton key point trajectory list of the newly added target human body according to the skeleton key point data of the newly added target human body in the n+1th to Nth video frames.

[0136] Refer to the following Fig.10 The electronic device 1000 according to this embodiment of the present invention will be described. Fig.10 The electronic device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0137] like Fig.10 As shown, the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010, the at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0138] The storage unit stores program codes, which can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1010 can perform the following steps: Figure 2 or Figure 3 or Figure 4 or Figure 5 or Figure 6 Follow the steps shown in .

[0139] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202 , and may further include a read-only storage unit (ROM) 10203 .

[0140] The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205, such program modules 10205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0141] Bus 1030 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0142] The electronic device 1000 may also communicate with one or more external devices 1100 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1050. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1060. As shown, the network adapter 1060 communicates with other modules of the electronic device 1000 via a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0144] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0145] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0146] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0147] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0148] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0149] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0150] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and concept of the present disclosure are indicated by the claims.

Claims

1. A human motion recognition method, characterized in that: include: Obtain a video to be identified, where the video to be identified includes N video frames, where N is an integer greater than 0; Perform key point detection on the nth video frame and the n+1th video frame to obtain skeleton key point data of the target human body in the nth video frame and skeleton key point data in the n+1th video frame, where n<N; Tracking the target human body according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame; Determine a skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; Determine the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body; The action recognition result includes a coarse classification action recognition result; Determining the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body includes: Obtaining a sample video and a rough classification action annotation of a sample human body in the sample video, wherein the rough classification action annotation includes stillness and movement; Determine a first variance mean of a stationary sample human body and a second variance mean of a moving sample human body according to the skeleton key point trajectory list of the sample human body; Determining a target classification threshold according to the first variance mean and the second variance mean; The rough classification action recognition result of the target human body is determined according to the skeleton key point trajectory list of the target human body and the target classification threshold.

2. The method according to claim 1, characterized in that Determining the rough classification action recognition result of the target human body according to the skeleton key point trajectory list of the target human body and the target classification threshold comprises: Determine the center point of the target human body in N video frames according to the skeleton key point trajectory list of the target human body; Determine the center point variance of the target human body according to the center point of the target human body in N video frames; A coarse classification action recognition result of the target human body is determined according to the target classification threshold and the central point variance of the target human body.

3. The method according to claim 1, characterized in that Determining a target classification threshold according to the first variance mean and the second variance mean includes: The maximum value between the first variance mean and the second variance mean is determined as a target classification threshold.

4. The method according to claim 1, characterized in that The action recognition result includes an accurate classification action recognition result; wherein, determining the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body also includes: Determine the key points to be identified for the action to be accurately classified; Based on the key points to be identified, sampling the skeleton key point data of the target human body in the nth video frame to obtain sampled key point data; Determine a first vector and a second vector according to the sampling key point data; Determining an accurate classification action recognition result of the target human body in the nth video frame according to the angle between the first vector and the second vector; The accurate classification action recognition result of the target human body is determined according to the accurate classification action recognition results of the target human body in N video frames.

5. The method according to claim 1, characterized in that Tracking the target human body according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame includes: Determine a skeleton boundary region of the target human body in the nth video frame according to the skeleton key point data of the target human body in the nth video frame; Determining a reference feature of the target human body in the nth video frame according to a histogram normalized feature of a skeleton boundary region of the target human body in the nth video frame; Randomly generate k potential positions of the target human body in the n+1th video frame within a preset area of ​​the center point position of the skeleton boundary area of ​​the target human body in the nth video frame, where k is an integer greater than 0; Calculate and obtain reference features of k potential positions of the target human body in the n+1th video frame; Calculating k similarities between the reference features of the target human body and the reference features of k potential positions; The potential position corresponding to the maximum value among the k similarities is determined as the updated center point position of the skeleton boundary area of ​​the target human body in the n+1th video frame; The predicted position of the target human body in the n+1th video frame is determined according to the updated center point position of the skeleton boundary region of the target human body in the n+1th video frame.

6. The method according to claim 1, characterized in that Determining the skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame includes: Determine the skeleton boundary area of ​​each skeleton key point data in the n+1th video frame; Calculate the area overlap rate between the predicted position of the target human body in the n+1th video frame and each skeleton boundary area in the n+1th video frame; Determine the skeleton key point data corresponding to the skeleton boundary region with the maximum area overlap rate in the n+1th video frame as the skeleton key point data of the target human body in the n+1th video frame; A skeleton key point trajectory list of the target human body is determined according to the skeleton key point data of the target human body in N video frames.

7. The method according to claim 6, characterized in that Determining a skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in N video frames includes: If the maximum value of the area overlap rate between the predicted position of the target human body in the n+1th video frame and each skeleton boundary area in the n+1th video frame is less than the area overlap rate threshold, then determining the skeleton key point trajectory list of the target human body according to the skeleton key point data of the target human body in the 1st to nth video frames; If there is a skeleton key point data in the n+1th video frame and the area overlap rate with the predicted position of the target human body in the n+1th video frame is less than the area overlap rate threshold, then the skeleton key point data is determined as the skeleton key point data of the newly added target human body, and the skeleton key point trajectory list of the newly added target human body is determined based on the skeleton key point data of the newly added target human body in the n+1th to Nth video frames.

8. A human motion recognition device, characterized in that: include: A video acquisition module, used to acquire a video to be identified, wherein the video to be identified includes N video frames, where N is an integer greater than 0; A key point detection module is used to perform key point detection on the nth video frame and the n+1th video frame to obtain skeleton key point data of the target human body in the nth video frame and skeleton key point data of the n+1th video frame, where n<N; A target tracking module is used to track the target human body according to the skeleton key point data of the target human body in the nth video frame to obtain the predicted position of the target human body in the n+1th video frame; A trajectory list determination module, used to determine the skeleton key point trajectory list of the target human body according to the skeleton key point data in the n+1th video frame and the predicted position of the target human body in the n+1th video frame; An action recognition module, used to determine the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body; The action recognition result includes a coarse classification action recognition result; Determining the action recognition result of the target human body according to the skeleton key point trajectory list of the target human body includes: Obtaining a sample video and a rough classification action annotation of a sample human body in the sample video, wherein the rough classification action annotation includes stillness and movement; Determine a first variance mean of a stationary sample human body and a second variance mean of a moving sample human body according to the skeleton key point trajectory list of the sample human body; Determining a target classification threshold according to the first variance mean and the second variance mean; The rough classification action recognition result of the target human body is determined according to the skeleton key point trajectory list of the target human body and the target classification threshold.

9. An electronic device, characterized in that: include: at least one processor; A storage device for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images