Behavior prediction and early warning system based on two-dimensional image

Abstract

The invention relates to two-dimensional behavior recognition of a natural person in the field of computer vision, and provides a novel behavior recognition method using time-domain three-dimensional skeleton holographic recognition data (Kinect), so that tracking of human body joint positions along with time is possible. The method introduces a definition of a body state; each action is modeled as a sequence of the states. In the mode recognition of the learning stage, Fisher data dimensionality reduction linear discriminant analysis (Fisher LDA) is used for constructing a discriminant feature space to distinguish the body state. Meanwhile, the distance measurement for measuring the distance between points in the multi-dimensional space, namely the distance measurement for measuring the similarity between two points, is used as the proper distance measurement for classifying the non-voluntary action states. Time-domain transformations between body states in each motion are subjected to sequential data modeling by using a statistical model-Hidden Markov Model (HMM) to understand and analyze data with implicit structures. The action recognition (feedback) rate of the result is higher than that of other methods and can reach 96.18%.

Description

Technical Field

[0001] Natural person behavior recognition is a research in the field of computer vision and machine learning, based on the prediction and early warning of actions and behaviors in two-dimensional images. Background Art

[0002] Other systems in the market use RGB videos as input to the recognition system, and various valuable methods and algorithms have been proposed based on behaviors and activities recognized using RGB data.

[0003] However, there are some problems in using red, green and blue (RGB) frames to complete action recognition, such as occlusion and different camera directions, the presence of other objects besides the human body that can be mentioned, and the lack of third dimension information, which is another challenge faced by such methods.

[0004] To address these issues, methods that recognize behaviors from multiple perspectives have been introduced; however, they are usually very computationally expensive and not suitable for real-time recognition.

[0005] Therefore, how to effectively reduce the amount of calculation while more accurately identifying, predicting and warning actions and behaviors is an urgent problem to be solved. Summary of the invention

[0006] In view of the above problems and the introduction of 3D holographic recognition sensors on the market, researchers of the present invention have begun to study 3D data for behavior recognition.

[0007] In addition to capturing red, green and blue frames, the holographic recognition sensor also provides depth and skeleton data.

[0008] So far, different approaches have been proposed for action recognition using either depth or skeleton data.

[0009] Action recognition has many different applications.

[0010] From a certain perspective, all actions can be classified as normal (voluntary) and involuntary movements (see Figure 1 ). Everyday actions, game actions, and interactions between humans and computers can be considered normal actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to describe the non-limiting and non-exhaustive implementation methods of the present invention and to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. In each drawing, unless otherwise specified, the same figure numbers in all drawings refer to the same parts.

[0012] Figure 1 To classify actions / behaviors, they can be divided into two categories: regular / normal and involuntary

[0013] Figure 2 Natural person behavior / motion recognition system in the recognition of falling actions

[0014] Figure 3 Select a joint from the available joints in the skeleton data and display the joints used for correction

[0015] Figure 4 The skeleton is corrected by using the left and right shoulders to align to offset the skeleton orientation.

[0016] Figure 5 Example of Fisher and PCA Directions

[0017] Figure 6 The projection of the state sample on the Fisher space. It can be seen that These states have different distributions

[0018] Figure 7 A three-state hidden Markov model (HMM) model

[0019] Figure 8 Overall structure of the proposed framework

[0020] Fig. 9 Depth Image Sample Dataset

[0021] Fig.10 Examples of each state of state recognition

[0022] Fig.11 Chaotic Matrix State

[0023] Fig.12 Action Chaos Matrix DETAILED DESCRIPTION

[0024] Involuntary movements can occur in different places, such as homes, hospitals, and public places. The most common involuntary movement is a fall, which can happen to patients in hospitals. Elderly people are also prone to dangerous falls, and if falls are detected by elderly care monitoring systems, serious injuries and deaths can be reduced. Proper detection of involuntary movements can prevent problems and confusion in public places.

[0025] In these places, involuntary actions such as falling or being thrown may be caused by accidents or physical conflicts. Compared with normal actions, involuntary actions usually have larger performance differences across various trajectories and different subjects. The characteristics of such involuntary actions are the main challenge in identifying them.

[0026] Although the proposed method can be applied to both normal and involuntary movements, it focuses on handling involuntary movements and attempts to address the above challenges. Figure 2 A human action recognition system for fall detection is described. It can be seen that it is impossible to train the system using all the various fall actions of different subjects. Therefore, the challenge is to recognize any fall action using a limited number of training samples.

[0027] This invention proposes a new method to identify human body movements, especially involuntary movements. The main situation is as follows:

[0028] Compared to most action recognition methods, this work is not feature-based. Rather, it is holistic. In other words, features are not extracted from the skeleton, as in the case of the used joint histograms; the raw features of the skeleton are fed to a so-called body state classifier. The classifier is thus responsible for extracting discriminative features. As is well known in face recognition, holistic approaches have a greater potential for accurate recognition, as it uses all the information and leaves feature extraction to the classifier. Our experiments validate our better performance than feature-based methods in both action and normal scenes;

[0029] This work properly handles involuntary actions by considering the distribution of each body state, which are distributed differently in the joint space;

[0030] Using a statistical model [Hidden Markov Model (HMM)] to model sequential data to handle the different speeds at which unconscious actions are performed;

[0031] This method can be used to identify normal movements as well as involuntary movements;

[0032] Besides outperforming the recognition of various normal and involuntary motions in the dataset, the proposed method achieves a high recognition rate in distinguishing all involuntary motions from the class of normal motions. This is particularly important in cases where only involuntary motion detection is important, such as elderly or patient monitoring.

[0033] The main algorithms of the method proposed in the present invention include human body modeling, action recognition using data dimension reduction linear discriminant analysis (Fisher linear discriminant analysis) and sequence data modeling statistical model (HMM), as well as the data sets used and experimental results.

[0034] According to the importance of action recognition and its numerous applications, this paper proposes many different methods in this field. In particular, in the model methods for different types of visual word bags, the impact of each method on action recognition is studied. These factors include feature extraction, feature preprocessing, codebook generation, feature encoding, pooling, normalization and fusion of these descriptors.

[0035] In this study, genetic programming is applied to the spatiotemporal motion features of action recognition, and features are extracted from color and optical flow sequences for action recognition based on motion optical flow histograms and motion-based histogram descriptors. For difficult problems such as identifying a part of the body that is blocked by others, three-dimensional algorithms and depth maps are considered for recognition.

[0036] In the research and development, we create action graphs by sampling 3D points of depth images to identify natural human behaviors / actions, and project the contours of the human body shape onto different planes for sampling to simulate the position of the human body, and then simulate the state of the human body through the package of these 3D points, which are regarded as nodes of the graph to model the action. Although this method is insensitive to the constant changes in perspective and human body size, it recognizes 90% of the actions and halves the error compared to the 2D method.

[0037] Another aspect of research on recognizing actions in depth images using deep convolutional neural networks is to transfer human posture to a high-level space where the viewpoint remains unchanged, that is, to represent the 5D space of (x, y, z, f, n) as a 2D coordinate space and a 3D color space, where f and n represent time and joint labels, respectively. The combined descriptors of red, green, blue (RGB) and depth images are used to obtain spatiotemporal interest points, and the spatiotemporal information of skeleton joints in depth sequences is encoded into more discriminative deep features, which are used for action recognition.

[0038] The present invention also studies the method of tracking the human body in the image and detecting the position of the human body joints in 3D space by developing holographic recognition sensors and related software, and using this information to identify the action. Different action categories are distinguished by finding the covariance matrix of the joint position during the action, and the support vector machine is used for classification, and a deep motion map is created, and the action recognition is performed through the distance-weighted Tikhonov Matrix.

[0039] On the other hand, a sparse set of body parts is used to model the action as a combination of multimodal features, where the dynamics and appearance of the parts are represented by a set of heterogeneous deep and skeleton-based features. The joints are divided into three categories: stable, mobile, and highly mobile, and the angles of 10 important joints and the vectors connecting the moving joints to the stable joints are used.

[0040] The position of the shoulder and hip joints is used to obtain the body's orientation during the test. The hip joint and the lateral hip joint are used to extract the histogram of other joint positions as feature vectors, and the orthogonal basis vectors are extracted for each frame. LDA is used to reduce the dimension of the feature vector, and the K-means method is used to cluster the feature vectors. Then the spherical angle of the joint is used instead of the position of the joint. A new space is constructed for it based on these vectors and the body orientation of each person, and modeled by HMM. The energy function is used to overcome the problem of both hands or feet doing the same action.

[0041] In order to recognize actions, the actions should be properly modeled. Action modeling depends on various facts, such as the application, the action type, and the classification method. One of the most important applications of action recognition is online recognition, where the recognition should be performed in real time. This type of recognition is the goal of research and development.

[0042] In this category, actions should be modeled so that the model can be updated as the action is completed and eventually the type of action performed can be identified. Therefore, each action should be a sequence of several body states. After that, the body state is simulated using the joint positions in 3D space, which are prepared by the output of the Kinect sensor. The skeleton consists of several joints, and the dataset used in the experiments of the present invention has 25 joints. However, many of these joints are very close to each other and have no important difference in movement; therefore, their information is almost redundant.

[0043] For the actions discussed in the present invention, only 12 important joints are selected from the skeleton, including left and right ankles, left and right knees, left and right wrists, left and right shoulders, head, mid-spine, hips and spine shoulders. The positions of the base of the spine (hips) and left and right shoulders are used for alignment to correctly describe the body state of different people and running. The selected joints and the joints required for correction are as follows Figure 3 State modeling, including skeleton alignment and state classification.

[0044] In order to model and describe the state of the body, an appropriate descriptor should be created. This descriptor models the action as a time sequence of states and tries to recognize the action. The body state is determined as follows. Based on the nature of each action, the body states that constitute the action are inferred and then manually selected and sampled from the training sequence of picture frames. This manual selection is only done in the training phase. In the test phase, each input picture frame is automatically classified by the body state classifier:

[0045] 1. Align the skeleton: The different positions and orientations of the body in the skeleton create the need to align the skeleton. As mentioned earlier, in order to describe the state of the body, 12 joint positions are used in 3D space. In order to cancel the position of the body bones, the position of the hip joint is subtracted from the positions of all joints. This is performed for each frame in the sequence. In addition, the different orientations of the bones or the camera make it difficult or even wrong to identify similar states. Therefore, in order to offset the different orientations of the body bones, the body is rotated around the y-axis so that the vector connecting the left and right shoulders is projected onto the xz plane parallel to the x-axis. With this rotation, the bones will face directly towards the camera. This process is as follows Figure 4 The axis is as shown. Figure 3 As shown. Skeletal alignment is often performed in research; however, the method or joints used for this operation may vary. For example, the left and right hip joints are used for alignment instead of the shoulder joint;

[0046] 2. Create feature vectors: To determine the body state in each frame, an appropriate feature vector is required. Three of the 12 joints are used for alignment, and the remaining nine joints are used to create feature vectors. If represents the coordinates of the mth joint (m={1,...9}), then the original feature vector is Data dimensionality reduction linear discriminant analysis is used to extract discriminant features from the original feature vector. In the data dimensionality reduction linear discriminant analysis method, the dimension of the feature vector is reduced to C-1, where C is the number of states. In the linear discriminant analysis of data dimensionality reduction, the intra-class (S w ) and inter-class (S b )The scattering matrix is: In order to minimize the within-class covariance and maximize the between-class covariance, μ i and μ represent the average value of the i-th state and the average value of the class average, respectively. The linear discriminant analysis projection space for data dimensionality reduction is given by The feature vector is created.

[0047] By projecting it into this space, the eigenvector F of the input skeleton state is obtained. After being projected onto the data dimensionality reduction linear discriminant analysis space, the obtained eigenvectors are positioned relative to each other so that the eigenvectors associated with similar and different states are close and separated, respectively. Through this fact, the identification of the state becomes possible.

[0048] There are other feature reduction methods that can be used for classification. One of the most popular methods in this category is principal component analysis. However, principal component analysis (PCA) It is not always possible to classify data as in linear discriminant analysis. As an example, suppose the distribution of classes is similar to Figure 5 In the present invention, the direction of data dimension reduction linear discriminant analysis (Fisher LDA) is perpendicular to the direction of PCA.

[0049] like Figure 5 As shown, the data dimensionality reduction linear discriminant analysis (Fisher LDA) attempts to minimize the intra-class variance while maximizing the inter-class variance in order to classify them. The resulting feature vectors are used for training and testing body states. Actions will be defined as a time series of multiple specific states. In the testing phase, the body state is identified by finding the minimum distance as described in the next section.

[0050] 3. Finding the minimum distance: In each frame denoted as f, the state of the body should be identified. To achieve this, the distance between the feature vector F of that frame and the mean of the feature vectors of all states is found. The minimum distance determines the frame The state of f. If represents the mean of the eigenvectors of the i-th class, then the state is found to be: where d is the distance measurement function, which can be one of the following:

[0051] Euclidean distance: One of the most popular methods for calculating the distance between two vectors is the Euclidean distance. The present invention will use the Euclidean distance as one of the distance methods. The function of the Euclidean distance can be expressed as: where F j and They are F and The jth component of

[0052] Mahalanobis distance: Since the minimum distance of the state average is used to identify the state, using an appropriate distance has a great impact on the accuracy of recognition. Therefore, the distribution of the final feature vectors in the feature space should be considered and the distance measure defined accordingly.

[0053] If the body states are classified into C classes, the dimension of the final feature (Fisher) vector is C-1. Since the dimensionality of the feature vectors can be high, their distribution in each class cannot be directly visualized for direct analysis. However, the distribution of the feature vectors can be analyzed in higher dimensions by calculating the covariance matrix of the feature vectors. The first two directions of the Fisher space are used here to illustrate the distribution of each of the eight body states defined for the TST dataset. Figure 6The training examples are shown projected onto the space constructed by the first two Fisher directions. As shown, the distribution of the feature vector of each state is different in the two directions.

[0054] The more different the ways in which people perform actions involving a state, the more widely distributed that state is. States with wider distributions are usually states during the completion of involuntary actions. For example, Figure 6 As shown, after projection on the constructed Fisher space, the states associated with normal actions (such as standing and sitting) are less distributed than the states occurring in involuntary actions (such as lying forward and lying backward). In order to cope with the challenge of different distributions of projected states, distance measurement functions other than Euclidean functions should be used to account for these distributions. The Mahalanobis distance takes into account the variance of the distribution in its calculation and is calculated as follows: Where S represents the covariance matrix of the eigenvectors of the class with which the distance is calculated.

[0055] As is evident from equation (5), the covariance matrix S acts as a weighting matrix for each class according to its distribution. That is, the importance of distances in a particular dimension is taken into account when calculating distances. In other words, distances in directions with smaller variance have lower value, yielding S -1 The Mahalanobis distance is actually an extension of the standard deviation of the mean in multidimensional space. The experimental report of the invention shows the superiority of this distance compared with the Euclidean distance.

[0056] Each action can be modeled as a sequence of subsequent states. After identifying the body states using FisherLDA, this work uses HMM to classify the actions. Each action is modeled using a separate HMM. Each HMM has many hidden states with specific transition probabilities between them. For example, a three-state HMM and its transition probabilities are as follows Figure 7 As shown. Each hidden state has a specific emission probability of the emitting body state. The transition and emission probabilities of each HMM are estimated by the well-known Baum-Welch expectation maximization algorithm using the training observations (i.e., the body state sequences). The algorithm starts with an initial assumption about all parameters of the model (i.e., transition and emission probabilities) and then iteratively updates the parameters using the corresponding expectation maximization equation until convergence.

[0057] To reduce the computational cost of the algorithm, the frame rate is reduced by downsampling. A uniform downsampling rate of 20 is used, which is shown to be appropriate based on our experiments (note that the sampling rate of the Kinect V2 sensor is known to be 30 frames per second (fps) under normal lighting conditions and 15 fps under poor lighting conditions. According to the corresponding RGB images of the dataset, the samples in this dataset should be captured under normal lighting conditions, so the original sampling rate of this dataset must be 30fps). After constructing the HMM for each action, the unknown sequence is identified by feeding it to each HMM. After feeding the test sequence of frames to all the trained HMMs. Each HMM outputs the probability of occurrence of the sequence. The maximum probability determines the action of the sequence.

[0058] To gain insights, note that each repetitive cycle of body states can be roughly associated with an HMM state. For example, when having a three-state HMM for classifying actions, the actions sit, grasp, and finally sit consist mainly of the following sequence: - {standing, squatting, sitting on a chair}, -{stand, bend over, stand}, -{standing, squatting, sitting on the ground} Wherein the body state represents a subsequence of repeated body states. In addition, in each subsequence, the number of repetitions of the corresponding body state may vary across subjects and different trials.

[0059] For each action, the sequences used to train the HMM were adjusted to have the same length (number of body states). This equalization was performed by manually repeating the last state completed by the individual so that the total number of states for all actions became equal. It should be noted that this equalization does not compensate for the different lengths and speeds of actions performed by different people or in different trials. The advantage of the HMM in this work is that it only considers the dynamics of the sequences and is insensitive to the speed and length of the various actions. For example, in the TST Fall dataset, there are sequences with lengths ranging from 75 frames to 463 frames and different action speeds, which have been successfully processed and recognized by this method. The overall structure of the proposed framework is shown in Figure 2. Figure 8 shown.

[0060] The TST fall detection dataset is used to verify the effectiveness of this method. The dataset contains two main categories of actions, namely daily living activities and falling actions. Each action is performed three times by eleven different people. Daily living activities include sitting, lying, grabbing, and walking; falling actions include falling forward, falling backward, falling to one side, and falling down. This dataset prepares the joint 3D position information and depth data obtained by Kinect sensor V2, which is more accurate than previous Kinect sensors. In this work, only the skeleton data of this dataset is used for experiments. One of the important goals of human action recognition is monitoring applications, especially for controlling the elderly or patients. The main goal of detecting involuntary actions and the improvements of KinectV2 encourage this work to use the above dataset. Unlike other datasets, involuntary actions such as falls are fully present in this dataset, which makes this database challenging.

[0061] Since the falling action is involuntary, different people will experience different states and conditions from normal actions. Therefore, existing action recognition methods may not necessarily work well for falling actions. In addition, many methods have been proposed to identify falling actions, which focus on features such as speed and acceleration recorded by accelerometer sensors. These features cannot effectively distinguish between normal and involuntary behaviors; therefore, they cannot help identify general actions. Therefore, the main challenge here is to develop a method that can detect and analyze normal and involuntary behaviors and identify them from each other. Several deep image sample datasets of actions in TST are as follows: Fig. 9 shown.

[0062] In the dataset, only actions are labeled; therefore, labeling states should be performed manually. Based on the actions, eight different states are selected and labeled for training and testing the state classification module. The selected states should include the main states of the operations in the dataset and should not contain unnecessary states that are close to other states. The states selected are standing, squatting, lying back, lying forward, lying on the side, bending over, sitting on a chair, and sitting on the ground. Examples of each state are shown in Figure 2. Fig.10 shown.

[0063] The experiment adopts "leave one subject out" cross validation. In each iteration, The entire sample of one person is considered as the test sample, and the samples of other subjects are used to train the system. This type of cross-validation is quite difficult because the system does not see any samples from the test subjects during the training phase. The state recognition experiments are repeated using the two distance methods and the results are listed in Table 1. Note that all ratios reported in this paper are recall rates: Unless the type of rate is mentioned. Table 1. Body state recognition accuracy

[0064] Table 1 shows that the Mahalanobis distance is better than the Euclidean distance. As expected, the recognition rates of squatting, lying forward, and bending are significantly improved using the Mahalanobis distance. The reason is that the variance of the training data for these states is large. And this fact is not taken into account when using Euclidean distance.

[0065] By using the Mahalanobis distance, the recognition efficiency of the bent state is improved at the expense of the recognition rate of the standing state. Figure 6 It can be found that there is an overlapping distribution area between the two states. The Euclidean distance does not consider the distribution of the classes and mostly identifies the overlapping area as the standing state. On the other hand, the Mahalanobis distance mostly identifies the area as the bending state because the change in the standing state is much smaller than that in the bending state. This fact can also be seen from the state confusion matrix of the two distances, as shown in Figure 11.

[0066] In the operation, the action is represented as a sequence of states. Each state in this sequence is identified by projecting into the LDA space and using a distance measure. Then, the probability of each HMM (note that there is an HMM for each specific action) generating the input state sequence is calculated, and the maximum probability determines the recognized action. The number of hidden states in the HMM (note that hidden states are different from body states) affects the recognition performance. Therefore, different hidden states of the HMM were tested and compared to each other. Table 2. The impact of the number of HMM states on the recognition rate

[0067] The results for three different numbers of hidden states for the HMMs are reported in Table 2. The experiments for this table were conducted under the Mahalanobis distance. As expected, the HMM with three hidden states performs better based on the nature of the states and actions in the TST-Fall dataset; therefore, the number of hidden states for the HMMs is considered to be three in this work. It is worth noting that combinations of the optimal number of hidden states for each action were also considered; however, the experiments showed that using a constant number of hidden states for all HMMs yields better performance.

[0068] In the development, the proposed methods mentioned above are compared. Note that all the above methods have been experimented on datasets created using old versions of Kinect sensors, which do not have involuntary actions. A fair comparison is made with the proposed method using the TST dataset. First, for LDA, the states are labeled in the same way as the proposed method. Second, the number of hidden states of the HMMs is chosen to be three based on the actions of the dataset. Third, the best number of clusters for the histogram is found to be eight, which is consistent with the number of state classes in the proposed method. Table 3. Comparison of results between our method and method 8 on the TST dataset

[0069] The results are listed in Table 3, which show that the proposed method using each of the two distance measures outperforms method [8] in all operations. Although method [8] uses LDA and clustering methods to prepare data for training HMM, it makes several states very close to each other by using the histogram concept, which increases the error. For example, in the falling action, the angular positions of the joints are very similar, and their differences are ignored when using the histogram.

[0070] Using the Mahalanobis distance significantly improves the performance, especially in the falling action. In other words, it improves the recognition performance of difficult involuntary states, such as squatting and prone, and improves the overall recognition rate. As mentioned before, the main reason for this fact is that the intrinsic variance of the state is taken into account in the Mahalanobis distance.

[0071] The chaotic matrix of the action is Fig.12 This matrix shows that actions that are similar to each other are sometimes confused and misidentified. Actions such as front, side, and back falls are sometimes confused with each other because their distributions (and thus their behaviors) are similar and broader than other actions, such as Figure 6 As shown. In some cases, such as anomaly detection in actions, this confusion between subgroup actions may not matter. Therefore, another experiment was conducted to treat all falls and normal actions as two different high-level groups. In this case, the recognition rate increased from 88.64% to 96.18%. Table 4. Comparison results considering all anomalies In addition, it can be seen from Table 4 that the false alarm rate is also significantly reduced. This result shows that the possibility of misidentifying a normal action as a fall action is quite low.

[0072] The present invention uses Kinect data to create a feature vector to represent the body state in each frame, and then identifies the body state in the corresponding discriminant Fisher subspace, and classifies and identifies the action by providing the identified body state sequence to HMMs. It is used to identify actions with a certain complexity, such as various types of falling actions, especially in the case of a mixture of normal and involuntary actions. By preprocessing the data using low-dimensional feature vectors, skeleton alignment, etc., the algorithm is stable to the direction of the camera and the amount of calculation is relatively low. By aligning the angle data between joints, more complex and longer actions can be identified and greater stability can be obtained. For the AI ​​selection and sampling of body states, the system can automatically find the basic state of higher-level actions and automatically select body states, thereby enhancing the scalability of the system.

Claims

1. A software recognition and warning system, a new method for identifying actions through two-dimensional images, the method comprising: The system uses joint position recognition (Kinect) data, hidden Markov model (HMM), HMMs (Hidden Markov Model Collection), etc., and is particularly suitable for identifying actions with a certain complexity and issuing warnings for specific actions, such as various types of falling actions, aggressive actions, dangerous actions, etc.

2. The method according to claim 1, characterized in that: Each two-dimensional image is decomposed by the system using a low-dimensional feature vector, and there is no large computational overhead, so it can be used in real scenes and realistic situations. Experiments show that the core algorithm of the system outperforms other methods, especially in the case of a mixture of normal and involuntary behaviors.

3. The method according to claim 2, characterized in that A feature vector is created from the holographic recognition (Kinect) data using joint positions to represent the body state in each frame, and in the proposed method, the body state is then identified in the corresponding data dimension reduction linear discriminant analysis (Fisher) subspace.

4. The method according to claim 3, characterized in that Actions are classified and recognized by feeding the identified body state sequence to an ensemble of HMMs (Hidden Markov Models).

5. The method according to claim 4, characterized in that A Hidden Markov Model (HMM) is used, which is robust to different motion speeds and lengths. In addition, to improve the recognition rate, the Mahalanobis Distance is used to consider a wider distribution of involuntary body states.

6. A software system that pre-processes the data by aligning the skeleton in 2D images, making the algorithm robust to the orientation of the camera. The angles between the joints can be used instead of their positions to gain greater robustness.

7. The system according to claim 6, characterized in that To recognize and identify more complex and longer actions, manual selection / sampling of body states limits the scalability of the system. In a similar way, the body state is automatically selected to automatically find the basic state of higher-level actions.

8. The system according to claim 7, characterized in that Identifying actions of near-body states in addition to different motions, cancelling the motion of the body through alignment is necessary for the approach proposed by Mahalanobis Distance, which omits motion information.

9. The system according to claim 8, characterized in that The identifying of different movements also includes: identifying specific actions and behaviors, and the processing actions are specifically used for: The falling action, aggressive action, etc. are identified.

10. The system according to claim 9, characterized in that The various behaviors of warning also include: warning specific actions and behaviors, and the processing behaviors are also specifically used for: The falling, dodging and defense, dangerous actions, etc. are regarded as the warnings.

Images