Methods for detecting macro activities

Abstract

Computer-implemented method for recognizing macro-activities (M) of a person using a neural network (NN) with the steps: generating in an initial step (S0) a graph embedding (E (G T )) of a graph (G T ), which represents a microactivity representation sequence consisting of a sequence of representations (m) of microactivities, by a graph embedding block (GEB) of the neural network (NN); checking in a first step (S1) by a macro-activity validation unit (MVE) of the neural network (NN) whether the graph embedding (E (G T )) of the graph represents a valid macroactivity (M); concatenating in a second step (S2) the graph embedding (E (G T )) and a representation (m T+1) of a next microactivity, as long as a check carried out in a third step (S3) by a link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1 ) of the next microactivity to the graph embedding (E (G T )) can be appended; and classifying (S4) by an activity classification unit (MKE) of the neural network (NN) the macroactivity (M) consisting of a sequence of micro-activities and recognized as valid in the first step (S1) as soon as the test carried out in the third step (S3) by the link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1 ) of the next microactivity does not match the graph embedding (E (G T )) can be appended.

Description

[0001] The invention relates to a method and a device for detecting macro-activities of a person based on sensor-detected inertial data. State of the art

[0002] Alia, Sayeda & Lago et al. (2021) "Summary of the Cooking Activity Recognition Challenge" describes a method for classifying both microactivities and macroactivities during the implementation of cooking recipes. Sensor data is generated from various accelerometers attached to the entire body of a person cooking. Each sensor dataset comprises signal time windows with a range of three standard deviations, which are labeled with a macroactivity (the implementation of the recipe) and the corresponding microactivities or manual tasks performed during cooking (e.g., taking, cutting, placing, etc.). The classification task for the macroactivities is a single-label task (i.e., one macroactivity label per time window), whereas the classification task for the microactivities is a multi-label task (i.e., a number N of microactivity labels per signal time window).Several teams have developed a number of machine learning-based approaches to accomplish these two tasks. One of these approaches uses a convolutional neural network (CNN) to classify microactivities based on raw data derived from motion data. It classifies micro- and macroactivities that occur within signal time windows, but does not predict connections or links between microactivities that form macroactivities.

[0003] In "Meta-activity recognition: a wearable approach for logic cognition based activity sensing" IEEE INFOCOM 2017, Atlanta, GA, USA, 2017, pages 1-9 doi: 10.1109 / INFOCOM.2017.8057209, an approach for detecting micro-activities is presented. To extract micro-activities from sensor time series, this approach analyzes movement angle profiles, e.g., arm movements. Subsequent angle profiles are separated so that each segment represents a complex macro-activity. Meta-activities are classified using dynamic time warping and meta-activity templates. Complex macro-activities are then classified using a least-edit-distance approach. This conventional approach uses angle profiles and template matching to detect micro- and macro-activities, but does not dynamically combine successive micro-activities into macro-activities based on learned micro-activity relationships.

[0004] Human activities are typically composed of various sub-movements that can occur sequentially, in parallel, or intertwined. For example, the activity of cooking involves taking ingredients, washing them, cutting them, pouring them into a pan, and much more.

[0005] Hitting a racket, such as a table tennis racket, involves a backward arm movement, a forward arm movement, and a backswing. These sub-movements are referred to as micro-activities, while the combined movements of such sub-movements are referred to as macro-activities.

[0006] Conventional human activity recognition (HAR) approaches attempt to classify human activities by evaluating predefined time intervals (time windows) of signal data, with time intervals typically ranging from a few milliseconds to a few minutes.

[0007] Traditionally, macro-activities can be classified based on defined, labeled data sets. In reality, however, the information about when a macro-activity begins and ends is not always predetermined. One challenge, therefore, is to connect successive micro-activity windows in such a way that they potentially form or reveal more general macro-activities. Disclosure of the invention

[0008] According to a first aspect, the invention provides a computer-implemented method for detecting macro-activities (M) of a person by means of a neural network, comprising the steps: Generating, in an initial step of a graph embedding, a graph representing a microactivity representation sequence consisting of a sequence of representations of microactivities by a graph embedding block of the neural network; In a first step, a macro-activity validation unit of the neural network checks whether the graph embedding of the graph generated by the graph embedding block in the initial step represents a valid macro-activity; Concatenating in a second step the graph embedding generated by the graph embedding block in the initial step and a representation of a next microactivity, as long as a check performed in a third step by a link predication unit of the neural network shows that the representation of the next microactivity can be appended to the graph embedding generated by the graph embedding block in the initial step; and Classification by an activity classification unit of the neural network of the macroactivity consisting of a sequence of microactivities and recognized as valid in the first step as soon as the check carried out in the third step by the link prediction unit of the neural network shows that the representation of the next microactivity cannot be appended to the graph embedding generated by the graph embedding block in the initial step.

[0009] The method according to the invention improves the recognition of human activities by using at least one graph neural network (GNN) that processes successive micro-activities and predicts links between these micro-activity sequences that may constitute a macro-activity of the person.

[0010] The method according to the invention comprises checking a current sequence of microactivities (microactivity sequence) and predicting whether a subsequent microactivity can be validly appended to the current microactivity sequence.

[0011] In addition to the detection and validation of microactivity sequences, the method according to the invention can also be used to classify macroactivities (= sequences of microactivities). For this purpose, microactivity sequences are converted into a graph representation, which can be passed on to a graph neural network that processes a unique graph embedding E representing such microactivity sequences.

[0012] The method according to the invention can be used in addition to learned or trained micro-activity classifiers that use either neural networks or other traditional machine learning approaches to classify human activities in sensor-captured inertial data.

[0013] The method according to the invention preferably uses graph neural networks (GNN) to create a representation of the current microactivity sequence via other neural network architectures (e.g., recurrent neural networks (RNN) or convolutional neural networks (CNN), since link prediction (i.e., checking whether a microactivity sequence can be appended to the current microactivity sequence) is a common activity in graph learning and graph theory.

[0014] The method according to the invention can be easily adapted to different underlying network architectures.

[0015] Using the method according to the invention, links L or connections between successive microactivities can be predicted in order to reliably detect various human macroactivities.

[0016] According to a further aspect, the invention provides a neural network for detecting macro-activities comprising: a graph embedding block for generating a graph embedding of a graph representing a microactivity representation sequence consisting of a sequence of representations of microactivities; a macro-activity validation unit for checking whether the graph embedding of the graph generated by the graph embedding block represents a valid macro-activity; a concatenation unit configured to concatenate the graph embedding generated by the graph embedding block and a representation of a next microactivity, as long as a check performed by a link prediction unit of the neural network shows that the representation of the next microactivity can be appended to the graph embedding generated by the graph embedding block; and with an activity classification unit for classifying the macroactivity consisting of a sequence of microactivities and validly recognized as soon as the check performed by the link prediction unit of the neural network shows that the representation of the next microactivity cannot be appended to the graph embedding generated by the graph embedding block.

[0017] The method according to the invention and the neural network NN according to the invention are preferably optimized such that they can be trained to carry out all of the above-mentioned steps at once.

[0018] Although this approach increases the complexity of network training, it enables richer internal feature extraction, from which all subtasks benefit (multi-task learning).

[0019] Furthermore, using a single neural network instead of separate classifiers reduces memory and processing requirements, as parts of the neural network NN can be shared between the different subtasks.

[0020] This is particularly advantageous when the neural network according to the invention is implemented on portable devices, such as smartwatches or the like.

[0021] The inventive method can use graph neural networks (GNN) to create a representation of the current microactivity sequence via other neural network architectures (e.g., recurrent neural networks (RNN) or convolutional neural networks (CNN), since link prediction (i.e., checking whether a microactivity sequence can be appended to the current microactivity sequence) is a common activity in graph learning and graph theory.

[0022] Possible embodiments of the method according to the invention and the neural network according to the invention are explained in more detail below with reference to the attached figures.

[0023] They show: Fig. 1A is a representation of a graph used in the method according to the invention as a basis for detecting macroactivities; Fig. 1 B an adjadence matrix of the Fig. 1 B graphs; Fig. 2 shows a schematic block diagram of a possible embodiment of a neural network according to the invention for detecting macro-activities; Fig. 3 a flowchart illustrating a possible embodiment of a method according to the invention for detecting macro activities.

[0024] As shown in the block diagram according to Fig. As shown in Figure 2, a GNN-based neural network NN according to the invention comprises, in one possible embodiment, in addition to a graph embedding block GEB, a macro-activity validation unit MVE, a link prediction unit LPE and a macro-activity classification unit MKE.

[0025] The Fig. The embodiment of a GNN-based neural network NN according to the invention shown in Figure 2 comprises a macro-activity validation unit MVE. The macro-activity validator MVE of the GNN-based neural network NN is designed to predict whether a sequence of micro-activities is valid. The macro-activity validation unit MVE receives the graph embedding E of the graph G from the graph embedding block GEB and uses a binary classifier to check whether it represents a valid macro-activity M or not. The macro-activity validation unit MVE preferably consists of a number of h fully connected hidden layers FC. h V and a binary output layer FC o V , as in Fig. 2 is shown.

[0026] The Fig. The embodiment of a GNN-based neural network NN according to the invention shown in Figure 2 further comprises a link prediction unit (Link Predictor) LPE, which is designed to predict whether the hidden representation m T+1 the next microactivity or subsequent microactivity to the generated graph embedding (E (G T )) of the graph G T can be appended to the current micro-activity sequence.

[0027] The link prediction unit LPE of the neural network NN receives the generated graph embedding (E (G T )) and the hidden representation m T+1 of the next microactivity. The link prediction unit LPE of the neural network NN concatenates the obtained graph embedding (E (G T )) and the resulting hidden representation m T+1of the next microactivity and then checks whether this concatenation or linking results in a possible microactivity sequence or not. For this purpose, a binary classifier with h hidden layers FC h L and a binary output layer FC o L used.

[0028] The Fig. The embodiment of a GNN-based neural network NN according to the invention shown in Figure 2 also has a macro-activity classification unit MKE (Macro Activity Classifier), which is designed to classify recognized macro-activities M.

[0029] The macro-activity classification unit MKE of the neural network NN receives the graph embedding (E (G T )) and classifies the microactivity sequence recognized as valid using a feedforward network with h hidden layers FC h Mand a multi-class output layer FC o M The macro-activity classification unit (MKE) requires the combination of consecutive micro-activities in an embedding E. Therefore, the provision of the link prediction unit (LPE), which identifies micro-activity sequences, is a prerequisite for evaluation by the macro-activity classification unit (MKE) of the neural network (NN).

[0030] The neural network NN according to the invention can be used for classifying macro-activities, wherein preferably the GNN components, namely the macro-activity validator (MVE), link prediction unit (LPE) and macro-activity classification unit (MKE) are used.

[0031] A prerequisite for the application of graphic neural networks (GNNs) to data is the assignment of data points to a graph G. Therefore, the neural network NN according to the invention converts micro-activity sequences into a graphical representation G.

[0032] Representations m of microactivities can be generated based on sensory data. The sensors can include accelerometers, gyroscopes, or other inertial sensors. The only requirement or limitation for microactivity classifiers is that, in addition to classification, they generate rich and hidden representations of the input signals, which can be used to form a graphical representation G of the microactivity sequences.

[0033] The Fig. The embodiment of the neural network NN according to the invention shown in Figure 2 can be used for a wide variety of applications. It can be implemented in particular in end devices, especially portable or worn end devices, hearing aids, headphones, AR / VR, and other wearable devices. With the help of the neural network NN according to the invention, such devices are enabled to provide more detailed insights and more helpful suggestions for the users of the device via a user interface of the end device, in particular at a higher level of abstraction that includes information from many smaller, more fundamental movements or micro-activities. The devices can have various types of sensors, in particular inertial or acceleration sensors, which provide sensor data regarding the movement or acceleration of body parts of the person.

[0034] The neural network NN according to the invention is also able to recognize components of different activities that are interwoven over time and to relate them to a usage context in a larger picture.

[0035] To convert microactivity sequences into a graphical representation or graph G, a rich representation mt of each microactivity is provided with t ∈ {1,...,T} as the time index. In one possible embodiment, a microactivity classification unit (mKE) is provided, which is designed to generate a rich representation mt of each microactivity.

[0036] A graph G consists of nodes V (also called vertices) and edges that represent the connections between the nodes V. A microactivity sequence consisting of a sequence of microactivities is transformed into a graph G by transforming each representation of a microactivity into a graph node V. t is converted, where the rich representations with the microactivity are the node features of node V t Each node or vertex V of the graph G is unidirectional with the last node V in time T connected as shown in the diagram of Fig. 1 A is shown schematically.

[0037] A resulting graph adjacency matrix A, as shown in Fig. 1B is a zero-filled matrix, with the last column a T filled with ones, as in Fig. 1B. The adjacency matrix A is a square matrix used to represent the relationships between the nodes V of a graph G. The rows and columns of the matrix A correspond to the nodes V of the graph G. The entries of the adjacency matrix A indicate whether or not there is a connection (edge) between the corresponding nodes V. If there is a connection between the nodes Vi and Vj, the entry at position (i, j) and (j, i) of the adjacency matrix A is one. If, however, there is no connection, the entry in the adjacency matrix A at this position is zero. This mapping can be performed easily and quickly, which is particularly necessary for classification by edge devices. Furthermore, this mapping enables efficient processing with the subsequent GNN-based network architecture, as it captures the characteristics of each micro-activity in a single node V. Tof the graph G.

[0038] As shown in the block diagram according to Fig. 2, the GNN architecture used in the method according to the invention essentially consists of a graph embedding block GEB (Graph Embedding Block), a macro-activity validation unit MVE (Macro Activity Validator), a link prediction unit LPE (Link Predictor) and a macro-activity classification unit MKE (Macro Activity Classifier).

[0039] The graph embedding block GEB of the neural network NN receives the generated graph G = (V, A) and, in one possible embodiment, first applies a positional encoding to the nodes (vertices V) of the provided graph G using a positional encoding unit PKE to encode temporal relationships between the micro-activities in the node features of the nodes V of the graph G (PE: Positional Encoding). The graph embedding block GEB of the neural network NN can use a positional encoding function that is widely used in transformer models.

[0040] The updated node features are then transferred to the last node V using a graph attention layer GAT of the graph embedding block GEB. T The aggregation is based on the Fig. 1 B, which is provided to the graph embedding block GEB of the neural network NN, as in Fig. 2 is shown.

[0041] The attention mechanism of the GAT layer of the graph embedding block GEB weights the nodes V according to their importance for the graph embedding E. The attention mechanism in a GAT layer allows the model to assign different weights to the contributions of neighboring nodes of each node V in the graph G, based on their relative importance for the current processing.

[0042] Each node V in the graph G is represented by a vector. These vectors can, for example, represent the features or properties of the nodes V. Initially, a linear transformation is performed for each node V to combine the features and result in a representative vector, which is the transformed vector h V for the node V.

[0043] In the GAT layer of the graph embedding block GEB of the neural network NN, the neighboring nodes are weighted based on their importance for a particular node V. This is achieved by an attention mechanism. For each node V, attention weights are calculated between the node V i and its neighboring node V j These weights can be normalized by an activation function, typically the softmax function: aij=softmax(Leaky ReLU(aT[Whv,Whj]))

[0044] Here, W are weight matrices for the transformation, a is a trainable weight vector, and [Whv, Wh j ] represents the concatenation of the transformed features of v and j.

[0045] The weighted features of the neighboring nodes are combined by the attention weights to obtain an aggregated neighborhood vector hv' for the current node: hv'=∑j∈Neighbors(v)aij⋅Whj

[0046] The aggregated neighborhood vector h v ' is then used to update the final representative vector of the node: hvneu=Leaky ReLU(Whv')

[0047] This step results in an updated representation of node V taking into account the weights of its neighboring nodes.

[0048] The last vertex pooling layer LVP of the graph embedding block GEB of the neural network NN extracts the updated features of the last node V T , which contains the aggregated microactivity sequence information, as a graph embedding E. This cached graph embedding E is then used as input for the following components or units of the GNN architecture, as in Fig. 2 is shown.

[0049] Graph embedding, performed by the graph embedding block GEB, transforms the structured graph data of graph G into a low-dimensional vector space. During graph embedding, each node V of graph G is projected into a vector space in which the geometry of the vector reflects the structural and semantic relationships between the nodes V of graph G. Similar nodes in graph G are located close to each other in the vector space.

[0050] The macro-activity validation unit MVE of the neural network NN receives the graph embedding E of the graph G from the graph embedding block GEB and checks, using a binary classifier, whether it represents a valid macro-activity M or not. The macro-activity validation unit MVE consists of a number of h fully connected hidden layers FC h V and a binary output layer FC o V , as in Fig. 2 is shown.

[0051] The link prediction unit LPE of the neural network NN receives the graph embedding E and the hidden representation m T+1 of the next microactivity, concatenates them, and checks whether this concatenation or linking results in a possible microactivity sequence or not. For this purpose, a binary classifier with h hidden layers FC h L and a binary output layer FC o L used.

[0052] The macro-activity classification unit MKE of the neural network NN receives the graph embedding E from the graph embedding block GEB and classifies the micro-activity sequence using a feedforward network with h hidden layers FC h M and a multi-class output layer FC o MThe macro-activity classification unit (MKE) of the neural network (NN) requires the combination of consecutive micro-activities in an embedding. Therefore, the provision of a link prediction unit (LPE), which identifies micro-activity sequences, is a prerequisite for evaluation by the macro-activity classification unit (MKE).

[0053] Fig. Figure 3 illustrates the classification workflow in a possible embodiment of a method according to the invention based on a GNN-based architecture for the classification of macro-activities M, as described in Fig. 2 is shown.

[0054] In an initial step S0, a graph embedding E (G T ) of a graph G T generated using a graph embedding block GEB of the neural network NN.

[0055] In a first step S1, the macro-activity validation unit MVE of the neural network NN checks whether the generated graph embedding E (G T ) of the graph G T represents a macroactivity M or not.

[0056] Subsequently, in a second step S2, a concatenation unit of the neural network NN concatenates the generated graph embedding E (G T ) and the representation m T+1 the next microactivity.

[0057] In a third step S3, the link prediction unit LPE of the neural network NN is used to check whether the representation m T+1 the next microactivity to the graphene embedding E (G T ) can be attached or a link is possible.

[0058] In a fourth step S4, the test results obtained in steps S1 and S3 are evaluated.

[0059] If (first case F1) the check carried out in the first step S1 by the macro-activity validation unit MVE shows that no macro-activity M is present (ie the macro-activity M is invalid) the check carried out in the third step S3 by the link predication unit LPE shows that a linking or linking is possible (ie link L is valid) then the representation m T+1 the next microactivity to the graph G T appended and the resulting graph G T+1 = (V||m T+1 , , A T+1 ) is used as new input for the initial step S0 (graph embedding), as in Fig. 3 is shown.

[0060] If (second case F2) the check carried out in the first step S1 by the macro-activity validation unit MVE shows that a macro-activity M is present (i.e., the macro-activity M is valid) and the check carried out in the third step S3 by the link prediction unit LPE shows that the linking is not possible (i.e., link L is invalid), then a micro-activity sequence has been found that can be classified with the macro-activity classification unit MKE of the neural network NN. This is followed by a stop or end of the processing of the macro-activity MVE in the third step S3. Fig. 3 shown loop.

[0061] If (third case F3) the check carried out in the first step S1 by the macro-activity validation unit MVE shows that a macro-activity M is present (ie the macro-activity M is valid) and the check carried out in the third step S3 by the link predication unit LPE shows that the linking is possible (ie the link L is valid) then the valid micro-activity sequence is classified with the macro-activity classification unit MKE of the neural network NN and additionally the representation m T+1 the next microactivity to the graph G T appended. The resulting graph G T+1 = (V||m T+1 , , A T+1 ) is used as new input for the initial step S0 (graph embedding), as in Fig. 3 is shown.

[0062] If (fourth case F4) the check performed in the first step S1 by the macro-activity validation unit MVE shows that the current micro-activity sequence does not represent a macro-activity (i.e., the macro-activity M is invalid) and the check performed in the third step S3 by the link predication unit LPE shows that the linking is not possible (i.e., the link L is invalid), then no macro-activity M was found in the current micro-activity sequence. This results in a stop or end of the execution of the macro-activity sequence in Fig. 3 shown loop.

[0063] If multiple macroactivities M are found (see case F3), a heuristic is preferably used to decide which macroactivity label (M-label) should be output to the user via a user interface UINT. For example, the first (shortest) macroactivity M or the last (longest) macroactivity M can be output via a user interface UINT of the end device in which the neural network NN is implemented. Furthermore, the macroactivity M with the highest macroactivity validation probability can be output via a user interface. Furthermore, the macroactivity M with the highest cumulative link predictor probabilities or the macroactivity M with the highest macroactivity classification probability can also be output or returned to the user via a user interface of the end device.

[0064] As described above, the macro-activity validation unit (MVE), link prediction unit (LPE), and macro-activity classification unit (MKE) of the neural network (NN) build on one another. However, some of the units (MVE, LPE, and MKE) can also be used independently. For example, some applications may only require a validation check. In this case, the macro-activity validation unit (MVE) is sufficient to perform this subtask. Other applications may only be interested in the connections between consecutive micro-activities. In this case, providing a link prediction unit (LPE) is sufficient. In one possible embodiment, the various sub-units (MVE, LPE, and MKE) of the neural network (NN) can be selectively activated for different use cases by a control unit (SE).In addition, another classification unit can be added to classify the micro-activities. For example, another neural network (or another machine learning approach) can be used. T+1 to its corresponding microactivity. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Alia, Sayeda & Lago et. Al (2021) “Summary of the Cooking Activity Recognition Challenge

[0002] Meta-activity recognition: a wearable approach for logic cognition based activity sensing" IEEE INFOCOM 2017, Atlanta, GA, USA, 2017, Seiten 1-9 doi: 10.1109 / INFOCOM.2017.8057209

[0003]

Claims

[1] Computer-implemented method for detecting macro-activities (M) of a person using a neural network (NN) with the steps: Creating in an initial step (S0) a graph embedding (E (G T )) of a graph (G T ), which represents a microactivity representation sequence consisting of a sequence of representations (m) of microactivities, by a graph embedding block (GEB) of the neural network (NN); In a first step (S1), a macro-activity validation unit (MVE) of the neural network (NN) checks whether the graph embedding (E (G T )) of the graph represents a valid macroactivity (M); Concatenating in a second step (S2) the graph embedding (E (G T )) and a representation (m T+1) of a next microactivity, as long as a check carried out in a third step (S3) by a link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1 ) of the next microactivity to the graph embedding (E (G T )) can be attached; and Classifying (S4) by an activity classification unit (MKE) of the neural network (NN) the macroactivity (M) consisting of a sequence of micro-activities and recognized as valid in the first step (S1) as soon as the test carried out in the third step (S3) by the link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1 ) of the next microactivity does not match the graph embedding (E (G T )) can be appended. [2] Method according to claim 1, wherein for each microactivity an associated representation (mt) is generated by a microactivity classification unit (mKE). [3] Method according to claim 2, wherein a microactivity sequence consisting of a sequence of microactivities is stored in the graph (G T ) by converting each generated representation (m t ) of a microactivity into an associated node (V t ) of the graph (G T ) is converted, where the representation (m t ) of the microactivity node characteristics of the corresponding node (V t ) of the graph (G T ) form. [4] The method of claim 3, wherein temporal relationships between the micro-activities are encoded in the node features of the nodes of the graph by a position encoding unit (PKE) of the graph embedding block (GEB). [5] Method according to claim 4, wherein the node features of the nodes updated by the position coding unit (PKE) are passed through a graph attention layer (GAT) of the graph embedding block (GEB) to a last node (V T ) are aggregated based on an adjacency matrix (A) of the graph (G). [6] The method of claim 4, wherein a last vertex pooling layer (LVP) of the graph embedding block (GEB) uses the updated features of the last node (V T ), which contains the aggregated microactivity sequence information, as a graph embedding E(G T ) of the graph. [7] Method according to one of the preceding claims 1 to 6, wherein the activity classification unit (MKE) of the neural network (NN) generates a macroactivity label (M-label). [8] Method according to one of the preceding claims 1 to 7, wherein the macroactivity (M) classified by the activity classification unit (MKE) of the neural network (NN) is output via a user interface. [9] Method according to one of the preceding claims 1 to 8, wherein the representations (m) of micro-activities are generated on the basis of sensory acquired data. [10] Neural network (NN) for detecting macro activities (M) with: a graph embedding block (GEB) for generating a graph embedding (E (G T )) of a graph (G T ), which represents a microactivity representation sequence consisting of a sequence of representations (m) of microactivities; a macro-activity validation unit (MVE) to check whether the graph embedding (E (G T )) of the graph (G T) represents a valid macroactivity (M); a concatenation unit designed to combine the graph embedding (E (G T )) and a representation (m T+1 ) of a next microactivity, as long as a check carried out by a link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1 ) of the next microactivity to the graph embedding (E (G T )) can be appended; and with an activity classification unit (MKE) for classifying the macroactivity (M) consisting of a sequence of microactivities and validly recognized as soon as the test carried out by the link prediction unit (LPE) of the neural network (NN) shows that the representation (m T+1) of the next microactivity does not match the graph embedding (E (G T )) can be appended. [11] Terminal with at least one neural network (NN) according to claim 10 integrated therein and with a user interface for outputting the detected macroactivity (M). [12] Computer program product with a data carrier which stores program instructions for carrying out a method according to one of claims 1 to 9.

Images