Little video learning of repetitive human tasks

Abstract

Little-vision learning of repetitive human tasks is provided. Little-vision learning of repetitive human tasks is performed. Sliding window-based temporal segmentation of sensor data is performed for multiple cycles of a repetitive task. Motion alignment of the multiple cycles is performed that maps portions of the multiple cycles to corresponding portions of other cycles of the multiple cycles. Classes are constructed for each corresponding portion of the multiple cycles according to the motion alignment. Meta-training is performed to teach a model from data sampled from a set of labeled human motions and the classes of each corresponding portion, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the multiple cycles. The model is used to perform temporal segmentation of a data stream of sensor data in real-time for predicting motion windows within the data stream.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to few-shot learning of repetitive human tasks. BACKGROUND

[0002] In modern industrial manufacturing, low-cost smart sensors are often used to monitor, analyze, and improve assembly processes. In particular, motion sensors mounted on the body can continuously record high-precision movement signals. Advanced machine learning systems can then be built on top of them to perform anomaly detection, efficiency analysis, and poka-yoke feedback, among others. At each station along the assembly line, operators need to complete a sequence of assembly steps in a standard and timely manner. The human activities involved are often repetitive.

[0003] Few-shot learning is a task in which a classifier is adapted to accommodate new classes that were not seen during training, given only a few examples of each of these classes. Types of few-shot learning models include matching networks and prototypical networks. One problem with systems that perform few-shot learning is overfitting, in which the error of the training set is driven to a small value, but new data presented to the network gives a large error that is typically unusable. SUMMARY

[0004] In one or more illustrative examples, a method for few-shot learning of repetitive human tasks is performed. A sliding window based temporal segmentation of sensor data is performed for a plurality of cycles of a repetitive task, the plurality of cycles including a reference cycle and one or more operational cycles. Motion alignment of the plurality of cycles is performed, the motion alignment mapping portions of the plurality of cycles to corresponding portions of other cycles of the plurality of cycles. Classes are constructed for each corresponding portion of the plurality of cycles according to the motion alignment. Meta-training is performed to teach a model from data sampled from a set of labeled human motions and the classes of each corresponding portion, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the plurality of cycles. The model is used to perform temporal segmentation of a data stream of sensor data in real-time for predicting movement windows within the data stream.

[0005] In one or more illustrative examples, a system for few-shot learning of repetitive human tasks includes a memory configured to store a few-shot analysis application and motion capture data including a reference period and one or more operational periods; and a processor operatively connected to the memory. The processor is configured to execute the few-shot analysis application to perform a sliding window based temporal segmentation of sensor data for a plurality of periods of a repetitive task, the plurality of periods including the reference period and the one or more operational periods; perform motion alignment of the plurality of periods, the motion alignment mapping portions of the plurality of periods to corresponding portions of other periods in the plurality of periods; construct a class for each corresponding portion of the plurality of periods according to the motion alignment; perform meta-training to teach a model according to data sampled from a set of labeled human motions and the class of each corresponding portion, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the plurality of periods; and perform temporal segmentation of a data stream of sensor data in real-time using the model for predicting motion windows within the data stream.

[0006] In one or more illustrative examples, a non-transitory computer-readable medium includes instructions of a few-shot analysis application that, when executed by one or more processors, cause the one or more processors to perform a sliding window based temporal segmentation of sensor data for a plurality of periods of a repetitive task, the plurality of periods including a reference period and one or more operational periods; perform motion alignment of the plurality of periods, the motion alignment mapping portions of the plurality of periods to corresponding portions of other periods in the plurality of periods; construct a class for each corresponding portion of the plurality of periods according to the motion alignment; perform meta-training to teach a model according to data sampled from a set of labeled human motions and the class of each corresponding portion, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the plurality of periods; and perform temporal segmentation of a data stream of sensor data in real-time using the model for predicting motion windows within the data stream. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 FIGURE illustrates a schematic diagram of an example embodiment of a system for performing few-shot learning of repetitive human tasks.

[0008] Figure 2 FIGURE illustrates an example setup of performance of repetitive human activities;

[0009] Figure 3 FIGURE illustrates an example temporal segmentation of motion activities using a sliding window approach;

[0010] Figure 4 FIGURE illustrates an example of motion phase recognition under a few-shot learning framework;

[0011] Figure 5 FIGURE illustrates an example task construction method with respect to a training set;

[0012] Figure 6 FIGURE illustrates a bidirectional long short-term memory architecture for computing fixed-length embeddings; and

[0013] Figure 7 FIGURE illustrates an example process for performing few-shot learning of repetitive human tasks. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure are described herein. It is to be understood, however, that the disclosed embodiments are merely examples and other embodiments can take various forms and alternate forms. The drawings are not necessarily to scale; the dimensions of some features can have been exaggerated or minimized to illustrate details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the embodiments. As those skilled in the art will appreciate, the various features illustrated and described in connection with any one figure can be combined with features illustrated and described in connection with one or more other figures, to produce a variety of other embodiments. Combinations of features illustrated and described in connection with the figures are not limited to the embodiments actually illustrated but are intended to be combinable with one another to produce a variety of other embodiments.

[0015] Operators at stations along an assembly line can complete a sequence of assembly steps. These assembly steps can be referred to as motion phases. An ordered sequence of motion phases can be referred to as a cycle in a repetitive activity. A method is described that provides real-time identification of motion phases. Because the method identifies motion phases in real-time, it provides a user with a no-latency feedback functionality, which is desirable in implementing a complete error-proofing system.

[0016] Figure 1 FIGURE illustrates a schematic diagram of an example embodiment of a system 100 for performing few-shot learning of repetitive human tasks. The system 100 includes a processor 102 operatively connected to a memory 110, an input device 118, a motion capture device 120, and a display device 108. While the illustrated system 100 is shown using a single computing device, other example systems 100 can include multiple computing devices that perform various aspects of few-shot learning.

[0017] In the system 100 as shown, the processor 102 includes one or more integrated circuits that implement the functionality of a central processing unit (CPU) 104 and a graphics processing unit (GPU) 106. In some examples, the processor 102 is a system on a chip (SoC) that integrates the functionality of the CPU 104 and the GPU 106, as well as optional other components including, for example, a memory 110, a network device, and a positioning system, into a single integrated device. In other examples, the CPU 104 and the GPU 106 are connected to each other via a peripheral connection device, such as a PCI express or another suitable peripheral data connection. In one example, the CPU 104 is a commercially available central processing device that implements an instruction set such as one of the x86, ARM, Power, or MIPS instruction set families.

[0018] The GPU 106 can include hardware and software for displaying at least two-dimensional (2D) and optionally three-dimensional (3D) graphics to a display device 108. The display device 108 can include an electronic display screen, a projector, a printer, or any other suitable device that renders a graphical display. In some examples, the processor 102 uses hardware functionality in the GPU 106 to execute software programs including drivers and other software instructions to accelerate the generation and display of visualizations of graphical depictions and quantitative calculations of human movement models described herein.

[0019] During operation, the CPU 104 and the GPU 106 execute stored program instructions retrieved from the memory 110. The stored program instructions include software that controls the operation of the CPU 104 and the GPU 106 to perform the operations described herein. While the system 100 depicts the processor 102 as including both the CPU 104 and the GPU 106, alternative embodiments can omit the GPU 106, as, for example, the processor 102 can have a server that generates output visualization data using only the CPU 104 and transmits the output visualization data to a remote client computing device that uses the GPU 106 and the display device 108 to display the data. Additionally, alternative embodiments of the processor 102 can include a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or any other suitable digital logic device in addition to or in place of the CPU 104 and the GPU 106.

[0020] In system 100, memory 110 includes both non-volatile and volatile memory devices. Non-volatile memory includes solid-state memory such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when system 100 is deactivated or powered off. Volatile memory includes static and dynamic random access memory (RAM) that stores program instructions and data during operation of system 100, including multi-lens analysis application 112, motion capture data 114, and machine learning model 116. In some embodiments, CPU 104 and GPU 106 each have access to separate RAM devices (e.g., variants of DDR SDRAM for CPU 104 and variants of GDDR, HBM, or other RAM for GPU 106), while in other embodiments, CPU 104 and GPU 106 access a shared memory device.

[0021] Input devices 118 can include any of a variety of devices that enable system 100 to receive input from a user. Examples of suitable input devices also include human interface input such as a keyboard, mouse, touch screen, voice input device, etc. In some examples, system 100 implements input devices 118 as network adapters or peripheral interconnect devices that receive data from another computer or external data storage device, which can be useful for receiving large sets of motion capture data 114 in an efficient manner.

[0022] Motion capture data 114 refers to a plurality of records representative of a position of at least one tracked item or item portion over time. For example, motion capture data 114 can include one or more of the following: position records of reference points on a body part over time or at set time intervals, sensor data taken over time, video streams or video streams that have been processed using computer vision techniques, data indicative of an operating state of a machine over time, etc. In some cases, motion capture data 114 can include data representative of more than one continuous movement. For example, motion capture data 114 can include a combination of multiple sets of combined motion capture data 114.

[0023] Motion capture devices 120 are devices configured to generate motion capture data 114. As some non-limiting examples, motion capture devices 120 can include cameras, visual sensors, infrared sensors, ultrasonic sensors, accelerometers, gyroscopes, pressure sensors, etc. One non-limiting example of motion capture devices 120 is a digital glove worn by a user while performing a periodic motion. The digital glove can include sensors that capture the user’s motion to generate motion capture data 114 stored in memory 110.

[0024] In a machine learning system, model-based inference refers to an inferential method that operates based on a machine learning model 116 of the world view to be analyzed. Generally, a machine learning model 116 is trained to learn a function that provides an accurate correlation between input values and output values. At runtime, a machine learning engine uses the knowledge encoded in the machine learning model 116 to derive conclusions such as a diagnosis or a prediction against observed data. One example machine learning system can include the TensorFlow AI engine available from Alphabet, Inc. of Mountain View, California, although other machine learning systems can additionally or alternatively be used. In the examples described herein, the model 116 is a few-shot learning model.

[0025] Figure 2 An example setup is illustrated that repeats the execution of a human activity. At each assembly station, several cycles of a standard procedure (shown as a minimum one cycle) are recorded as a reference to train the machine learning model 116. For example, a motion capture device 120 can be used to capture motion capture data 114 to use as the reference movements. It should be noted that depending on the assembly procedure, the motion activities at different stations can be completely different, and furthermore, although the activities are repetitive, they can be performed at different rates, and small motion tolerances are affected by variations.

[0026] During the inference phase, the operator repeats the assembly cycle under observation of the motion capture device 120 and the model 116 identifying which phase is being performed at each time step. This is challenging due to certain constraints. First, acquiring a large amount of labeled data to train the model 116 can be prohibitive due to the busy manufacturing schedule in the factory and the labeling effort from human effort. Since minimal supervision ensures the usability of the system, the focus is on the challenging case in which only one assembly procedure is available at each station. Second, the assembly activity is complex and can be segmented at different levels of granularity. For example, some users can divide the cycle into several long, coarse motion phases, while others are interested in short, fine-grained motions. As a result, although the motion phase labels are given by the user during the reference recording, they are actually highly ambiguous. The lack of a uniform definition of the motion patterns makes it difficult to transfer knowledge from related classes, challenging the typical assumption from most few-shot learning methods (see, e.g., Snell, Jake, Kevin Swersky, and Richard Zemel, “Prototypical networks for few-shot learning.” Advances in Neural Information Processing Systems, pp. 4077-4087. 2017; and Finn, Chelsea, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks.” Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pp. 1126-1135. JMLR, 2017). To cope with these challenges, motion phase recognition is formulated under the few-shot learning framework. To train the embedding network, a task construction method based on time series alignment is then provided.

[0027] Figure 3 An example time segmentation of a motion activity using the sliding window approach is illustrated. As shown, the time series corresponds to the motion capture device 120 readings from one cycle of the considered workstation, which consists of three phases. Since only one assembly activity demonstration (cycle) is available, the training data for performing phase recognition is extremely limited. As shown, for a given training cycle of the motion capture data 114 for the workstation, a first sliding window based time segmentation is performed. The cycle length is denoted as l , the window length is denoted as w , and the step size of the moving window is denoted as τ , resulting in a total number of training samples , which is typically low sample regime. The number of samples (windows) per phase is also very low. Note that there are multiple phases in the cycle. Therefore, the number of training examples per class can be even lower, possibly two or three.

[0028] Assembly procedures across workstations are typically significantly different (assembling a power drill versus a washing machine). Therefore, the labeled motion capture data 114 from existing workstations cannot be directly shared with a new workstation, and their specific phases must be considered as different classes. This scenario fits the few-shot setting in the machine learning domain, where a model 116 must learn to generalize from a few examples (see, e.g., Fei-Fei, Li, Rob Fergus, and Pietro Perona. “One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence 28 , no. 4 (2006): 594-611).

[0029] Figure 4 An example of motion phase recognition formulated under the few-shot learning framework is illustrated. The target workstation is where the transfer of the model 116 will take place, and is named the test set (T). The training set (S) consists of the labeled motion capture data 114 from existing workstations. The model 116 is trained on the training set (S) and then transferred to the target workstation (T). The model 116 is evaluated on the test set (T) to determine the performance of the model 116. Figure 4(Workstation 4 in the middle). Note that the test set includes both the reference period and the actual operation period. The reference period is named the support set and is where model 116 can be fine-tuned after the pass-through learning is complete. The operation period corresponds to the query set and is where model 116 identifies the real-time execution phase. From other workstations ( Figure 4 The reference and operating cycles of workstations 1 to 3 in the model are referred to as the training set. As explained in further detail, model 116 is trained on the entire training set and then passed to the test set. This procedure is called meta-learning.

[0030] Directly applying few-shot learning models, such as ProtoNet (see, for example, Snell et al.), to this framework presents problems. First, while the training set is large, containing records from multiple workstations, the labeled portion only contains references to each workstation and is a very small fraction of the entire set. Label sparsity contradicts the typical assumption of few-shot learning, which assumes rich labeling in the training set. Furthermore, motion sequences are inherently ambiguous; therefore, it is difficult to clearly define each stage and its boundaries. Thus, constructing the task from the training set—i.e., defining appropriate support and query sets—is crucial for successful delivery. An efficient task construction method is proposed to address these challenges.

[0031] Figure 5 The illustration shows an example task construction method for the training set. As shown, an example for sampling a three-way, three-camera task is illustrated. The first aspect of the task construction method is to derive pseudo-labels for the unlabeled data. Intuitively, the periodicity of the records indicates the existence of inherent categories indicated by the correspondence between periods. Once the periods are aligned, stage labels can be revealed within these periods.

[0032] However, joint alignment of large amounts of time series is difficult (see, for example, "A global averaging method for dynamic timewarping, with applications to clustering" by Petitjean, Francois, Alain Ketterlin, and Pierre Gancarski). Pattern Recognition 44 (see references , no. 3 (2011): 678-693), and some cycles may even contain anomalous motion. Therefore, a reference cycle is used as an anchor to align with all existing cycles. Specifically, the reference cycle is denoted as... x rej And mark the operation cycle as , using dynamic time warping (DTW) (see, e.g., Keogh, Eamonn, and Chotirat Ann Ratanamahatana. "Exact indexing of dynamic time warping." Knowledge and information systems 7 , no. 3 (2005): 358-386) in x rej and each x n between the reference and each Figure 5

[0033] Next, alignment in motion is used to construct classes. Due to the difficulty in defining motion classes and their boundaries, atomic motions in short sliding windows are used to define a large number of classes. Specifically, first a time segmentation is performed on the reference cycle, similar to as shown in Figure 3 . The stride of the moving window is set to in order to avoid overlap between classes. The first k window is denoted as , and then the time boundaries of the DTW match of the window within each cycle are found. These matching motion windows are denoted as . In the case of multiple matching steps, the step with the lowest matching cost is chosen. For each station, the procedure is followed to construct K s classes, where s denotes the station. Thus, the total number of classes is . The procedure is illustrated in the class construction block of Figure 5 .

[0034] ​Given the obtained classes, meta-training is performed, which uses a similar architecture to ProtoNet in order to learn the model. One obstacle is that ProtoNet utilizes a convolutional neural network (CNN) as the modeling component, and it has two drawbacks for immediate application. First, CNNs are superior in capturing spatial correlations in images, but are deficient in modeling temporal ordering information in sensor time series. Second, the obtained motion windows can vary significantly in length due to rate variations in different operating periods, as discussed above. CNNs only accept fixed-size inputs, and the required padding operation can alter the inherent motion information. To address these concerns, a bidirectional long short-term memory (LSTM) network (see, e.g., Graves, Alex, and “Framewise phoneme classification with bidirectional LSTM and other neural network architectures.” Neural networks 18 , no. 5-6 (2005): 602-610) is utilized to replace the modeling component in ProtoNet.

[0035] Figure 6 A bidirectional LSTM architecture for computing fixed-length embeddings is illustrated. As shown in Figure 6 , the network consists of two LSTM units, both of which model the time series in a recurrent fashion. One unit looks only in the forward direction, while the other looks in the backward direction, and then their corresponding hidden embeddings are summed or concatenated to form a d dimensional fixed-length output embedding. The overall model is denoted as , where the learnable network parameters are denoted as .

[0036] Similar to Snell et al., to train the model, the prototype for each class is computed as the mean of the class embedding vectors as follows:

[0037]

[0038] Then, the predicted class distribution is modeled as a softmax over the Euclidean distances between the query sample and all class prototypes as follows:

[0039]

[0040] Finally, learning is performed by minimizing the negative log-likelihood loss with respect to the true class label k :

[0041] .

[0042] A key procedure in most meta-learning methods is to sample a large number of few-shot tasks from a meta-training set (see, e.g., Snell et al. and Vinyals, Oriol, Charles Blundell, Timothy Lillicrap, and Daan Wierstra, “Matching networks for one shot learning. Advances in neural information processing systems , pp. 3630-3638. 2016). Each sampled task consists of C classes and M samples per class. An example is shown in the meta-learning framework of Figure 4 with C = 3 and M = 3. The sampling procedure is performed as follows: at each training epoch, first K classes are sampled from the total of C classes. Then, for each selected class, M samples are randomly sampled to construct the support set. From the remaining data of the unselected classes, Q random samples are randomly retrieved to construct the query set. Given the support and query sets, can be computed, and the model parameters can be updated by backpropagation.

[0043] As mentioned previously, the network is meta-trained on all existing stations (e.g., stations 1-3 of Figure 5 ), but needs to be deployed on a new station with only references as support sets (e.g., station 4 of Figure 5 ). A station-specific model can be obtained by fine-tuning . Specifically, for the target station, a time split is performed on the reference period, and is further trained with user-defined phase labeling. The training can be stopped early to prevent overfitting, especially when the motion is significantly different from existing stations.

[0044] During runtime, a time split is performed on the operational data stream in real-time, and predictions are made about the motion window according to . The limitation of low training data is addressed by the knowledge transfer from the meta-training phase, and the model generalization towards new operational periods is improved.

[0045] Accordingly, recognition of repetitive human activity is formulated as few-shot model learning under low data regimes. Inherent classes of repetitive human activity are revealed through temporal alignment. Human motion knowledge is transferred from large annotated sets to novel classes through meta-training. Human motion knowledge is also transferred from large annotated sets to novel classes through fine-tuning. The solution further improves temporal transfer in few-shot learning by leveraging a bidirectional LSTM model.

[0046] Figure 7 An example process 700 for performing few-shot learning of repetitive human tasks is illustrated. In examples, the process 700 can be performed by the system 100 using the methods discussed in detail herein.

[0047] At operation 702, the system 100 performs a sliding window based temporal segmentation of sensor data for a plurality of cycles of a repetitive task. The plurality of cycles can include a reference cycle and one or more operational cycles.

[0048] At operation 704, the system 100 performs motion alignment of the plurality of cycles. Motion alignment can map portions of the plurality of cycles to corresponding portions of other cycles in the plurality of cycles. At operation 706, the system 100 constructs classes for each corresponding portion of the plurality of cycles according to the motion alignment.

[0049] At operation 708, the system 100 performs meta-training to teach a model according to data sampled from an annotated set of human motion and the classes of each corresponding portion. The model can leverage a bidirectional long short-term memory (LSTM) network to account for length variations between the plurality of cycles. At operation 710, the system 100 performs temporal segmentation of a data stream of sensor data in real-time using the model for predicting motion windows within the data stream.

[0050] The processes, methods, or algorithms disclosed herein can be deliverable to / implemented by a processing device, controller, or computer that can include any existing programmable electronic control unit or specialized electronic control units. Similarly, the processes, methods, or algorithms can be stored as data and instructions on a non-transitory storage medium that can be readable by a controller or computer. The instructions can be read by the controller or computer into a memory to implement some or all of a method, process, or algorithm. The memory of the controller or computer can be a RAM memory, a flash memory, or any other suitable memory. The memory can be volatile or non-volatile. The instructions can be those specially adapted to a special control unit or those that are more generally used.

[0051] While the example embodiments have been described above, these embodiments are not intended to describe all possible forms of the claims. The words used in the specification are words of description rather than limitation, and it is understood that various changes can be made without departing from the spirit and scope of the disclosure. As previously described, features of various embodiments can be combined to form further embodiments of the present application that can not be expressly described or illustrated. While various embodiments can have been described as providing advantages or being preferred over other embodiments or prior art implementations, this description is not intended to limit the scope of the claims to these particular embodiments. It is intended that the description be considered as illustrative only and not restrictive. It is intended that the embodiments and related applications can be practiced in a variety of forms disclosed herein, and that the disclosure is to be considered as comprising the embodiments as well as the following claims, which are to be accorded the full scope allowable under the patent laws.

Claims

1. A method for few-shot learning of repetitive human tasks, the method comprising: A sliding window-based time segmentation of sensor data is performed for multiple cycles of a repetitive task, the multiple cycles including a reference cycle and one or more operation cycles; Perform motion alignment of the plurality of cycles, wherein the motion alignment maps a portion of a reference cycle in the plurality of cycles to a corresponding portion of other cycles in the plurality of cycles; A category is constructed for each corresponding portion of the plurality of cycles based on motion alignment; Meta-training is performed to teach the model based on data sampled from a labeled set of human motions and the category of each corresponding part, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the multiple cycles; as well as The model is used to perform time segmentation on the sensor data stream in real time to predict motion windows within the data stream.

2. The method of claim 1, wherein dynamic time warp (DTW) is used to perform motion alignment to calculate the optimal match between discrete portions of the plurality of cycles.

3. The method of claim 1, wherein a reference cycle is used as an anchor to perform motion alignment to align the one or more operation cycles.

4. The method of claim 1, wherein the LSTM network includes a first unit viewed in the forward direction and a second unit viewed in the backward direction, wherein hidden embeddings from each of the first unit and the second unit are combined to form a fixed-length output embedding.

5. The method of claim 1, wherein the model is trained by the following operations: Calculate the mean of the class prototype for each category as the category's embedding vector; The predicted class distribution is modeled using a softmax function based on the Euclidean distance between the query sample and the class prototype. as well as We learn by minimizing the negative log-probability loss of the true class label for the category.

6. The method of claim 1, wherein sensor data is received from the digital glove.

7. A system for few-shot learning of repetitive human tasks, comprising: The memory is configured to store few-lens analysis application and motion capture data, including a reference cycle and one or more operating cycles. and The processor, operatively connected to memory, is configured to execute few-lens analysis applications, to A sliding window-based time segmentation of sensor data is performed for multiple cycles of a repetitive task, the multiple cycles including a reference cycle and one or more operation cycles; Perform motion alignment of the plurality of cycles, the motion alignment mapping a portion of a reference cycle in the plurality of cycles to a corresponding portion of other cycles in the plurality of cycles; A category is constructed for each corresponding portion of the plurality of cycles based on motion alignment; Meta-training is performed to teach the model based on data sampled from a labeled set of human motions and the category of each corresponding part, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the multiple cycles; as well as The model is used to perform time segmentation on the sensor data stream in real time to predict motion windows within the data stream.

8. The system of claim 7, wherein dynamic time warp (DTW) is used to perform motion alignment to calculate the optimal match between discrete portions of the plurality of cycles.

9. The system of claim 7, wherein a reference cycle is used as an anchor to perform motion alignment to align the one or more operating cycles.

10. The system of claim 7, wherein the LSTM network includes a first unit viewed in the forward direction and a second unit viewed in the backward direction, wherein hidden embeddings from each of the first unit and the second unit are combined to form a fixed-length output embedding.

11. The system of claim 7, wherein the processor is further configured to perform a few-shot analysis application to train the model by operations including: Calculate the mean of the class prototype for each category as the category's embedding vector; The predicted class distribution is modeled using a softmax function based on the Euclidean distance between the query sample and the class prototype. as well as We learn by minimizing the negative log-probability loss of the true class label for the category.

12. The system of claim 7, wherein sensor data is received from the digital glove.

13. A non-transitory computer-readable medium comprising instructions for a few-lens analysis application, the instructions, when executed by one or more processors, causing the one or more processors to: A sliding window-based time segmentation of sensor data is performed for multiple cycles of a repetitive task, the multiple cycles including a reference cycle and one or more operation cycles; Perform motion alignment of the plurality of cycles, the motion alignment mapping a portion of a reference cycle in the plurality of cycles to a corresponding portion of other cycles in the plurality of cycles; A category is constructed for each corresponding portion of the plurality of cycles based on motion alignment; Meta-training is performed to teach the model based on data sampled from a labeled set of human motions and the category of each corresponding part, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variations between the multiple cycles; as well as The model is used to perform time segmentation on the sensor data stream in real time to predict motion windows within the data stream.

14. The non-transitory computer-readable medium of claim 13, wherein dynamic time warp (DTW) is used to perform motion alignment to compute the best match between discrete portions of the plurality of cycles.

15. The non-transitory computer-readable medium of claim 13, wherein a reference period is used as an anchor to perform motion alignment to align the one or more operating periods.

16. The non-transitory computer-readable medium of claim 13, wherein the LSTM network comprises a first unit viewed in the forward direction and a second unit viewed in the backward direction, wherein hidden embeddings from each of the first and second units are combined to form a fixed-length output embedding.

17. The non-transitory computer-readable medium of claim 13, wherein the medium further comprises instructions for a few-shot analysis application, the instructions, when executed by the one or more processors, causing the one or more processors to: Calculate the mean of the class prototype for each category as the category's embedding vector; The predicted class distribution is modeled using a softmax function based on the Euclidean distance between the query sample and the class prototype. as well as We learn by minimizing the negative log-probability loss of the true class label for the category.

18. The non-transitory computer-readable medium of claim 13, wherein sensor data is received from the digital glove.

Images