System and method for standardized evaluation of activity sequences

Abstract

A system and method for standardized evaluation of an activity sequence. Motion windows are generated based on a query activity sequence. For each of the motion windows in the query activity sequence, a corresponding motion window is found in a reference activity sequence. One or more difference calculations are performed between the motion windows of the query activity sequence and the corresponding motion windows in the reference activity sequence based on at least one criterion associated with physical meaning. Anomalies of the motion windows are determined based on the one or more difference calculations. Based on the detected abnormal motion windows in the query activity sequence, a standardized evaluation result of the query activity sequence is output.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for standardized evaluation of activity sequences by using criteria associated with physical meanings. Background Art

[0002] Various aspects of human movement can have a large impact on, for example, the way tools are designed, the way workspaces are laid out, or the way tasks are performed. Understanding how the human body can move and interact with objects and environments can result in more ergonomic tools, more efficient workspaces to navigate, and more intuitive tasks to perform. However, the range of possible human motions and postures is extensive, and simple tasks such as lifting a cup, pointing in a direction, or turning a screw often result from a complex set of biomechanical interactions. This relationship of simple results from complex movements can make human motion and posture extremely difficult to quantify or understand in a meaningful or practical way. Summary of the invention

[0003] In one or more illustrative examples, a system for detecting abnormal motion in an activity sequence includes: a display device; a memory configured to store a motion analysis application and motion capture data, the motion capture data including a reference activity sequence and a query activity sequence; and a processor operably connected to the memory and the display device. The processor is configured to generate a motion window according to the query activity sequence; for each of the motion windows in the query activity sequence, find a corresponding motion window in the reference activity sequence; perform one or more difference calculations between the motion window of the query activity sequence and the corresponding motion window in the reference activity sequence based on at least one criterion associated with physical meaning; determine the abnormality of the motion window according to the one or more difference calculations; and output a standardized evaluation result of the query activity sequence to the display device, the standardized evaluation result indicating a measure of the abnormality of the motion window in the query activity sequence.

[0004] In one or more illustrative examples, a method for detecting abnormal motion in an activity sequence includes: generating motion windows based on a query activity sequence; for each of the motion windows in the query activity sequence, finding a corresponding motion window in a reference activity sequence; performing one or more difference calculations between the motion windows of the query activity sequence and the corresponding motion windows in the reference activity sequence based on at least one criterion associated with physical meaning; determining the abnormality of the motion windows based on the one or more difference calculations; and outputting a standardized evaluation result of the query activity sequence, the standardized evaluation result indicating a measure of the abnormality of the motion windows in the query activity sequence.

[0005] In one or more illustrative examples, a non-transitory computer-readable medium includes instructions for a motion analysis application, which when executed by one or more processors cause the one or more processors to generate motion windows based on a query activity sequence; for each of the motion windows in the query activity sequence, find a corresponding motion window in a reference activity sequence; based on multiple criteria associated with physical meanings, perform multiple difference calculations between the motion windows of the query activity sequence and the corresponding motion windows in the reference activity sequence, the difference calculations including orientation difference calculations and motion time difference calculations; perform a fusion of the multiple difference calculations to determine anomalies of the motion windows; and output a standardized evaluation result of the query activity sequence, the standardized evaluation result indicating a measure of anomalies of the motion windows in the query activity sequence, the standardized evaluation result including an indication of motion windows identified as having anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An example system for performing standardized evaluation of activity sequences according to the present disclosure is illustrated;

[0007] Figure 2 illustrates an example process for evaluating a level of normalization of a query activity sequence compared to a reference activity sequence;

[0008] Figure 3 illustrates an example representation of a motion window for querying an activity sequence;

[0009] Figure 4 illustrates an example calculation of anomalies for a motion window based on orientation differences; and

[0010] Figure 5 An example calculation of anomalies for a motion window based on motion temporal differences is illustrated. DETAILED DESCRIPTION

[0011] Embodiments of the present disclosure are described herein. However, it will be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The drawings are not necessarily to scale; some features may be enlarged or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as being restrictive, but merely as a representative basis for teaching those skilled in the art to adopt the embodiments in various ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the accompanying drawings may be combined with the features illustrated in one or more other drawings to produce embodiments that are not explicitly illustrated or described. The combination of the illustrated features provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable for specific applications or implementations.

[0012] Understanding human activities based on inertial measurement unit (IMU) sensing data is an important and challenging problem in industry. As a fundamental topic for activity understanding, it is necessary to evaluate the standardized level of human activities in many scenarios. For example, within a manufacturing plant, clearly understanding and evaluating the assembly actions of operators can lead to improved product quality management. However, manual inspections to perform such evaluations are labor-intensive, and therefore it is necessary to develop a solution that automatically evaluates the standardized level. Some systems can pre-record standardized operations and then calculate the difference between each query activity sequence and the reference. Such a scheme only implies an abstract value as the overall difference between the two activities. In addition, in order to give meaning to the results, it is necessary to map the values ​​to standardized levels based on each human input according to the operating scenario.

[0013] The present disclosure proposes a scheme for evaluating the standardized level of human physical activity using time series data from an IMU. The scheme compares a query activity sequence with a given reference activity sequence and fuses the evaluation results that take into account multiple characteristics of physical motion. Compared with other systems that evaluate the activity as a whole and return a single number as a standardized level, the proposed scheme detects abnormal motion in the activity sequence in order to provide local and fine-grained evaluation results. Another significant advantage of the scheme is that the threshold variables can be associated with physical meanings and can be generally defined for a wide range of activities, while in previous schemes, the parameters were manually selected for each case.

[0014] Figure 1 1 is a schematic diagram of an exemplary embodiment of a system 100 for performing standardized evaluation of activity sequences. The system 100 can quantitatively calculate the accuracy of movement, such as deviation from a prescribed motion or posture and deviation from a target time period completed. The system 100 includes a processor 102, which is operably connected to a memory 110, an input device 118, a motion capture device 120, and a display device 108. As described in more detail below, during operation, the system 100 (i) receives motion capture data 114 including a reference activity sequence and a query activity sequence, such as from the memory 110, the input device 118, or another source, (ii) processes the query activity sequence per motion window for comparison with the reference activity sequence, (iii) performs multiple analyses and data fusion of the analyses to determine anomalies of the motion windows, and (iv) generates a calculated standardized evaluation result 116 that identifies the standardized evaluation of the query activity sequence and the abnormal motion windows.

[0015] In system 100, processor 102 includes one or more integrated circuits that implement the functionality of a central processing unit (CPU) 104 and a graphics processing unit (GPU) 106. In some examples, processor 102 is a system on a chip (SoC) that integrates the functionality of CPU 104 and GPU 106 and optionally other components including, for example, memory 110, network devices, and positioning systems into a single integrated device. In other examples, CPU 104 and GPU 106 are connected to each other via peripheral connection devices such as PCI express or another suitable peripheral data connection. In one example, CPU 104 is a commercially available central processing device that implements an instruction set such as one of the x86, ARM, Power, or MIPS instruction set families.

[0016] GPU 106 may include hardware and software for displaying at least two-dimensional (2D) and optionally three-dimensional (3D) graphics to display device 108. Display device 108 may include an electronic display screen, a projector, a printer, or any other suitable device for reproducing a graphical display. In some examples, processor 102 uses the hardware functionality in GPU 106 to execute a software program including drivers and other software instructions to accelerate the generation and display of graphical depictions of human movement models and visualization of quantitative calculations described herein.

[0017] During operation, CPU 104 and GPU 106 execute stored program instructions retrieved from memory 110. The stored program instructions include software that controls the operation of CPU 104 and GPU 106 to perform the operations described herein.

[0018] Although Figure 1 Processor 102 is depicted as including both CPU 104 and GPU 106, but alternative embodiments may omit GPU 106, for example, processor 102 may be a server that generates output visualization data using only CPU 104 and transmits the output visualization data to a remote client computing device that displays the data using GPU 106 and display device 108. Additionally, alternative embodiments of processor 102 may include a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or any other suitable digital logic device in addition to or as a replacement for CPU 104 and GPU 106.

[0019] In the system 100, the memory 110 includes both non-volatile memory and volatile memory devices. Non-volatile memory includes solid-state memory such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the system 100 is deactivated or loses power. Volatile memory includes static and dynamic random access memory (RAM) that stores program instructions and data during operation of the system 100, including motion analysis applications 112, motion capture data 114, and standardized evaluation results 116. In some embodiments, the CPU 104 and the GPU 106 each have access to separate RAM devices (e.g., a variant of DDR SDRAM for the CPU 104 and a variant of GDDR, HBM, or other RAM for the GPU 106), while in other embodiments, the CPU 104 and the GPU 106 access a shared memory device. The memory 110 can store the motion capture data 114, the motion analysis application 112, and the standardized evaluation results 116 for maintenance and retrieval.

[0020] The input device 118 may include any of a variety of devices that enable the system 100 to receive the motion capture data 114, the motion analysis application 112, and the standardized evaluation results 116. Examples of suitable input devices also include human interface inputs, such as a keyboard, a mouse, a touch screen, a voice input device, etc. In some examples, the system 100 implements the input device 118 as a network adapter or peripheral interconnect device that receives data from another computer or an external data storage device, which may be useful for receiving large sets of motion capture data 114 in an efficient manner.

[0021] The display device 108 may include an electronic display screen, a projector, a printer, or any other suitable device that reproduces a graphical display of the standardized assessment results 116 generated by the system 100 based on the motion capture data 114 .

[0022] The motion analysis application 112 includes instructions that, when executed by the processor 102 of the system 100, cause the system 100 to perform the processes and operations described herein. These processes and operations include, for example, (i) receiving motion capture data 114 including a reference activity sequence and a query activity sequence from the memory 110, the input device 118, or another source, (ii) processing the query activity sequence per motion window for comparison with the reference activity sequence, (iii) performing multiple analyses and data fusion of the analyses to determine anomalies of the motion windows, and (iv) generating a calculated normalized assessment result 116 that identifies a normalized assessment of the query activity sequence and an abnormal motion window.

[0023] Motion capture data 114 refers to a plurality of records representing the position of at least one tracked item or portion of that item over time. For example, motion capture data 114 may include one or more of the following: a record of the position of a reference point on a body part over time or at set time intervals, sensor data acquired over time, a video stream or a video stream that has been processed using computer vision techniques, data indicating the operating state of a machine over time, etc. In some cases, motion capture data 114 may include data representing more than one continuous movement. For example, motion capture data 114 may include a combination of multiple combined sets of motion capture data 114.

[0024] The motion capture device 120 is a device configured to generate motion capture data 114. As some non-limiting examples, the motion capture device 120 may include: a camera, a visual sensor, an infrared sensor, an ultrasonic sensor, an accelerometer, a pressure sensor, etc. A non-limiting example of the motion capture device 120 is one or a pair of digital gloves worn by a user while performing cyclical motions. The digital gloves may include sensors that capture user motion to generate motion capture data 114 that is stored in the memory 110.

[0025] A move is an action performed by an operator. A reference move is a baseline or canonical version of a move. A reference move can be used as a comparison standard for other moves, allowing identification of how closely the other moves match the reference move.

[0026] For the purpose of calculating accuracy of human motion, the motion capture data 114 can generally be classified into one of two categories: reference activity sequences 122, which include data representing reference or baseline movements; and query activity sequences 124, which include data representing test movements, i.e., movements that are compared and quantitatively evaluated for accuracy relative to baseline movements.

[0027] The reference activity sequence 122 may include motion capture data 114 received from the motion capture device 120. The data may also include processed movement data, such as, for example, frames, steps, cycles, and time information collected from the raw movement data. In one example, the reference movement is represented as a reference activity sequence 122 having an ordered set of frames, each frame in the ordered set of frames including motion capture data 114 corresponding to a respective time interval of the reference movement.

[0028] The query activity sequence 124 may also include motion capture data 114 received from the motion capture device 120. In some examples, one or more movements in the motion capture data 114 include a label or labels that classify the movement as a reference movement or a test movement. The motion analysis application 112 may be programmed to receive instructions, such as from a user via an input device 118, or from another source, for classifying one or more movements as a reference movement or a test movement.

[0029] The motion analysis application 112 may be programmed to separate the motion capture data 114 into individual movements. In some examples, the motion analysis application 112 calculates different possible separations of the motion capture data 114 into individual movements and selects a separation based on the accuracy calculated by the motion analysis application 112.

[0030] The motion analysis application 112 can also be programmed to separate the received movement into frames, whereby a "frame" corresponds to a discrete time interval. In other words, each frame of the movement includes the following portion of the motion capture data 114: the portion corresponds to the portion of the movement that occurs during each interval of the timeline of the movement. In some examples, the duration of the interval corresponding to the individual frame is preset. In some examples, the duration of the interval corresponding to the individual frame is set based on instructions received from a user or from another source via an input device 118, for example. In some examples, the duration of the interval corresponding to the individual frame is set with reference to one or more characteristics of the motion capture data 114. For example, in some embodiments, the duration of the interval corresponding to the individual frame is set with reference to one or more of the following: the duration of the reference movement, the total travel distance of the movement, multiple individual movements or gestures within the movement, the movement speed, etc. In general, the same interval of the frame duration is used for both the reference movement and the test movement that will be evaluated relative to the reference movement.

[0031] In some examples, the motion capture data 114 may be received from a data storage device as a file of stored motion capture data. In such examples, the motion analysis application 112 may separate one or more movements in the motion capture data 114 into frames for further processing.

[0032] The motion analysis application 112 may also map frames of the test move to corresponding frames of the reference move. In some examples, the test move and the reference move are synchronized so that frames of the test move are mapped to frames of the reference move that correspond in time, and in some embodiments, the test move and the reference move are aligned in terms of gestures and motions within the move so that frames of the test move are mapped to frames of the reference move that correspond with respect to a sequence of motions and / or gestures performed in the move.

[0033] The motion analysis application 112 may further calculate the accuracy of the test move or moves relative to the reference move. Based on the analysis, the motion analysis application 112 may generate a standardized assessment result 116. The standardized assessment result 116 may include a standardized level score indicating the amount of objective difference between the test move and the reference move. The standardized assessment result 116 may further include information indicating abnormal motion within the test move compared to the reference move. Additional aspects of the generation of the standardized assessment result 116 are described in detail below.

[0034] Although the illustrated system 100 is shown using a single computing device that incorporates a display device 108, other example systems 100 may include multiple computing devices. As an example, the processor 102 generates the standardized assessment results 116 as one or more data files, and the processor 102 transmits the standardized assessment results 116 to a remote computing device via a data network. The remote computing device can then display the output standardized assessment results 116. In a non-limiting example, the processor 102 is implemented in a server computing device, which executes the motion analysis application 112 to implement a web server, which transmits the standardized assessment results 116 to a web browser in a remote client computing device via a data network. The client computing device implements a web browser or other suitable image display software to display the standardized assessment results 116 received from the server using the display device 108 of the client computing device.

[0035] Figure 2 An example process 200 for evaluating the standardized level of a query activity sequence 124 compared to a reference activity sequence 122 is illustrated. As used herein, a motion window is a time window having a motion defined in the time window. A motion window can be generated based on the query activity sequence 124. Then, for each iteration, a motion window in the query activity sequence 124 finds a corresponding motion window in the reference activity sequence 122. As the motion windows are identified, the difference between the two motion windows can be calculated based on one or more criteria. In an illustrative implementation, the difference between the two motion windows is calculated based on two criteria, namely orientation and motion time. The final anomaly of the motion window can be determined by fusing the results from the criteria. Iterations continue until all motion windows are processed. A standardized evaluation of the query activity sequence is calculated based on the detected abnormal motion windows. As discussed in more detail below, process 200 can be performed by the processor 102 of the system 100.

[0036] More specifically, at operation 202, the processor 102 receives a query activity sequence 124 and a reference activity sequence 122. In some examples, motion capture data 114 of the query activity sequence 124 and the reference activity sequence 122 may be received from a motion capture device 120. For example, the motion capture device 120 may be a set of gloves (not shown) that, when worn by a user, is configured to transmit motion capture data 114 representing the orientation of the user's hands given by the direction in which the palm of each hand faces, the posture of each hand given by the joint angles of each hand joint, and the movement of each hand given by the linear acceleration in three dimensions. In some examples, the motion capture device 120 may be configured to connect to the system 100 via a wireless connection protocol such as, for example, Bluetooth, Wi-Fi, radio transmission, etc. In some examples, the motion capture device 120 may include tracking points that are trackable using an optical tracking system such as a camera or an infrared tracking system. In some examples, the motion capture device 120 may include one or more controls, such as buttons or switches, configured to one or more of: cause the system 100 to begin capturing motion capture data 114 transmitted by the motion capture device 120, and cause the system 100 to stop capturing motion capture data 114. In some examples, the processor 102 may be configured to store the captured motion capture data 114 in a data file on a data storage device, such as in response to an instruction from a user received via the input device 118.

[0037] In some examples, motion capture data 114 may be received as a data file from a data storage device. For example, processor 102 may be configured to receive a user instruction via input device 118 to load motion capture data 114 from a data file on a data storage device. In some examples, portions of motion capture data 114 are received from different sources. For example, reference activity sequence 122 may be loaded from a data file on a data storage device such as memory 110, while query activity sequence 124 may be captured using motion capture device 120.

[0038] Regardless of the source, the inputs to process 200 are query activity sequence 124 and reference activity sequence 122. In one or more implementations, the data in query activity sequence 124 includes orientation data from an IMU sensor. Quaternions are used to represent orientation. Thus, activity sequence 124, which includes a sequence of quaternions, can be represented as shown in equation (1):

[0039] (1)

[0040] in tQuaternion data representing orientation information is the basis for determining the dissimilarity of the query activity sequence 124 compared to the reference activity sequence 122 .

[0041] At operation 204 , the processor 102 generates a motion window according to the query activity sequence 124 . Figure 3 An example 300 representation of a motion window for querying an activity sequence 124 is shown. The motion window may be referred to as , which indicates a window centered at index i of the query activity sequence. For each frame of the query activity sequence 124, the processor 102 generates a motion window based on a time window centered at the frame and having a duration configured by a parameter (e.g., a time amount, a number of frames, etc.). For frames at indices lacking duration in both directions of the range of the configured parameters, the motion window may be clamped at the boundary of the query activity sequence 124. In general, if the query activity sequence 124 includes n frames, the processor 102 generates n motion windows based on the query activity sequence 124.

[0042] As shown at operation 206, the processor 102 finds corresponding motion windows in the query activity sequence 124 and the reference activity sequence 122. Due to the high level of flexibility in the movement of the query activity sequence 124 and the reference activity sequence 122, the query activity sequence 124 and the reference activity sequence 122 can be synchronized so that frame-level matching can be achieved. Various algorithms can be used for this task, including dynamic time warping (DTW), which is a process by which a warping path is determined to align desynchronized portions of the query activity sequence 124 and the reference activity sequence 122. Matching algorithms are typically based on pairwise distances between sequence frames, which in this particular case are quaternions. In one example, the following quaternion distance is used: , as shown in equation (2):

[0043] (2)

[0044] in Denotes the inner product of quaternions.

[0045] Given the set of pairwise distances, processor 102 then identifies the best match such that the overall distance is minimized. As a result, processor 102 obtains a match between the frames in query activity sequence 124 and the frames in reference activity sequence 122. Each frame of the motion window in query activity sequence 124 is correspondingly associated with its corresponding frame in reference activity sequence 122. The set of corresponding frames defines a corresponding motion window in reference activity sequence 122, which is denoted as .

[0046] Regarding difference calculation, the difference between two motion windows can be calculated based on one or more criteria. In one implementation, the difference between two motion windows is calculated based on two criteria, namely orientation and motion time.

[0047] As shown at operation 208 , the processor 102 calculates the orientation difference. Figure 4 An example 400 calculation of anomalies of a moving window based on orientation differences is illustrated. The quaternion distance defined above can be used The orientation difference between two frames is defined as q1, q2, where q1, q2 are quaternion values ​​contained in the frame data. Similarly, the orientation difference between two motion windows can be calculated based on the difference between all frames in the two motion windows. However, even for the same motion, movements can often exhibit large orientation variations. To accommodate such variability, the median of the pairwise distances is chosen to represent the distance between the two motion windows. If the median is above a predefined threshold, then the motion window from the query activity sequence is considered to be the same based on the orientation difference. Determined to be abnormal.

[0048] As shown at operation 210 , the processor 102 calculates motion time differences. Figure 5 An example 500 calculation of anomalies of motion windows based on motion time differences is illustrated. The duration of a motion window is another powerful indicator of the level of normalization. The present invention takes into account the motion time differences of two motion windows. The length of a motion window is defined as the number of frames in the motion window. If the difference in length between two motion windows, i.e., a motion window in a query activity sequence and its counterpart in a reference activity sequence, is greater than a certain threshold, then the motion window is considered abnormal.

[0049] At operation 212, fusion of the differences to determine the abnormality of the motion window is performed. In an example, the abnormality of the motion window is determined by fusing the results from multiple criteria. In some implementations, continuing with the illustrated example, the difference between the two motion windows is calculated based on two criteria, namely, orientation and motion time. As one possibility, if any of the criteria are met, the motion window is identified as abnormal.

[0050] At operation 214, the processor 102 determines whether there are any additional motion windows to be processed. If so, control returns to operation 206. If all motion windows have been processed, control passes to operation 216.

[0051] At 216, once all motion windows in the query activity sequence are evaluated, the processor 102 calculates a normalized evaluation result 116 for the sequence based on the abnormal motion windows detected in the sequence. One approach is to give a normalized level score for the query activity sequence 124 by calculating the ratio of abnormal motion windows to the total number of motion windows. The score can be calculated as shown in equation (3):

[0052] (3)

[0053] in:

[0054] ,and

[0055] is the number of motion windows in the query activity sequence 124 .

[0056] Thus, the disclosed method provides a solution for automatically evaluating the normalization level of the query activity sequence 124 by detecting abnormal motions within the query activity sequence 124. This solution may be useful for any system that utilizes an IMU for abnormal detection and normalization evaluation of human activities, including but not limited to specific hand motions.

[0057] The process, method or algorithm disclosed herein may be delivered to / implemented by a processing device, a controller or a computer, which may include any existing programmable electronic control unit or a dedicated electronic control unit. Similarly, the process, method or algorithm may be stored in many forms as data and instructions executable by a controller or a computer, including but not limited to: information permanently stored on a non-writable storage medium (such as a ROM device) and information changably stored on a writable storage medium (such as a floppy disk, a tape, a CD, a RAM device, and other magnetic and optical media). The process, method or algorithm may also be implemented in a software executable object. Alternatively, the process, method or algorithm may be embodied in whole or in part using suitable hardware components such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a state machine, a controller or other hardware component or device, or a combination of hardware, software and firmware components.

[0058] Although exemplary embodiments are described above, these embodiments are not intended to describe all possible forms included by the claims. The words used in the specification are descriptive words, not limiting words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of the various embodiments can be combined to form other embodiments of the present invention that may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages or being preferred to other embodiments or prior art implementations in terms of one or more desired characteristics, it is recognized by those of ordinary skill in the art that one or more features or characteristics may be compromised depending on the specific application and implementation to achieve the desired overall system properties. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, etc. As such, to the extent that any embodiment is described as not as desirable as other embodiments or prior art implementations in terms of one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desirable for specific applications.

Claims

1. A system for detecting abnormal motion in an activity sequence, comprising: Display devices; a memory configured to store a motion analysis application and motion capture data, the motion capture data comprising a reference motion sequence and a query motion sequence; as well as A processor operably connected to the memory and the display device and configured to execute a motion analysis application to: Generate motion windows based on the query activity sequence; For each of the motion windows in the query activity sequence, find a corresponding motion window in the reference activity sequence; performing one or more difference calculations between motion windows of the query activity sequence and corresponding motion windows in a reference activity sequence based on at least one criterion associated with physical meaning; determining anomalies of the motion window based on the one or more difference calculations; as well as A normalized evaluation result of the query activity sequence is output to a display device, the normalized evaluation result indicating a measure of abnormality of the motion window in the query activity sequence.

2. The system according to claim 1, wherein: The one or more difference calculations include an orientation difference calculation and a motion time difference calculation, and determining an abnormality of the motion window based on the one or more difference calculations includes performing a fusion of results of the orientation difference calculation and the motion time difference calculation.

3. The system according to claim 1, wherein: If any criteria of the one or more difference calculations are met, the normalized evaluation result indicates an abnormality of the motion window.

4. The system according to claim 1, wherein: The processor is further configured to execute the motion analysis application to output to a display device any motion windows identified as having anomalies.

5. The system according to claim 1, wherein: The processor is further configured to execute the motion analysis application to: calculating a ratio of abnormal motion windows to the total number of motion windows; and The ratios are included in the standardized assessment results.

6. The system according to claim 1, wherein: The reference activity sequence and the query activity sequence include quaternion values ​​representing orientation information, and the processor is further configured to execute a motion analysis application to: Obtaining the orientation difference between the motion windows of the reference activity sequence and the query activity sequence according to the quaternion distance; and The motion windows of the query activity sequence are identified as having anomalies based on orientation differences exceeding a predefined threshold.

7. The system according to claim 1, wherein: The processor is further configured to execute the motion analysis application to: identifying a motion time difference based on a difference in length between motion windows of a reference motion sequence and a query motion sequence; and Motion windows of the query activity sequence are identified as having anomalies based on differences in length exceeding a predefined threshold.

8. The system of claim 1, wherein the query activity sequence comprises a plurality of frames, and the processor is further configured to execute a motion analysis application to generate, for each frame of the query activity sequence, a motion window as a time window centered at the frame and having a duration of predefined parameters.

9. A method for detecting abnormal motion in an activity sequence, comprising: generating a motion window according to the query activity sequence; For each of the motion windows in the query activity sequence, find a corresponding motion window in the reference activity sequence; performing one or more difference calculations between motion windows of the query activity sequence and corresponding motion windows in a reference activity sequence based on at least one criterion associated with physical meaning; determining anomalies of the motion window based on the one or more difference calculations; as well as A normalized evaluation result of the query activity sequence is output, the normalized evaluation result indicating a measure of abnormality of the motion window in the query activity sequence.

10. The method according to claim 9, wherein: The one or more difference calculations include an orientation difference calculation and a motion time difference calculation, and further include determining an abnormality of the motion window based on the one or more difference calculations by performing a fusion of results of the orientation difference calculation and the motion time difference calculation.

11. The method according to claim 9, wherein: If any criteria of the one or more difference calculations are met, the normalized evaluation result indicates an abnormality of the motion window. 12 . The method of claim 9 , further comprising outputting any motion windows identified as having anomalies to a display device.

13. The method according to claim 9, further comprising: Calculate the ratio of abnormal motion windows to the total number of motion windows; as well as The ratios are included in the standardized assessment results.

14. The method according to claim 9, wherein the reference activity sequence and the query activity sequence include quaternion values ​​representing orientation information, further comprising: Obtaining the orientation difference between the motion windows of the reference activity sequence and the query activity sequence according to the quaternion distance; as well as The motion windows of the query activity sequence are identified as having anomalies based on orientation differences exceeding a predefined threshold.

15. The method according to claim 9, further comprising: identifying a motion time difference based on a difference in length between motion windows of a reference motion sequence and a query motion sequence; as well as Motion windows of the query activity sequence are identified as having anomalies based on differences in length exceeding a predefined threshold.

16. The method according to claim 9, wherein: The query activity sequence includes a plurality of frames, and further includes, for each frame of the query activity sequence, generating a motion window as a time window centered at the frame and having a duration of predefined parameters.

17. A non-transitory computer readable medium comprising instructions of a motion analysis application, the instructions, when executed by one or more processors, causing the one or more processors to: generating a motion window according to the query activity sequence; For each of the motion windows in the query activity sequence, find a corresponding motion window in the reference activity sequence; performing a plurality of difference calculations between the motion windows of the query activity sequence and corresponding motion windows in the reference activity sequence based on a plurality of criteria associated with physical meaning, the difference calculations comprising orientation difference calculations and motion time difference calculations; performing a fusion of the plurality of difference calculations to determine anomalies of the motion window; as well as A normalized evaluation result of the query activity sequence is output, the normalized evaluation result indicating a measure of abnormality of motion windows in the query activity sequence, the normalized evaluation result including an indication of motion windows identified as having abnormality.

18. The medium of claim 17, further comprising instructions of a motion analysis application that, when executed by one or more processors, cause the one or more processors to: calculating a ratio of abnormal motion windows to the total number of motion windows; and The ratios are included in the standardized assessment results.

19. The medium of claim 17, wherein the reference activity sequence and the query activity sequence include quaternion values ​​representing orientation information, and further include instructions of a motion analysis application that, when executed by one or more processors, cause the one or more processors to: Obtaining the orientation difference between the motion windows of the reference activity sequence and the query activity sequence according to the quaternion distance; and The motion windows of the query activity sequence are identified as having anomalies based on orientation differences exceeding a predefined threshold.

20. The medium of claim 17, further comprising instructions of a motion analysis application that, when executed by one or more processors, cause the one or more processors to: identifying a motion time difference based on a difference in length between motion windows of a reference motion sequence and a query motion sequence; and Motion windows of the query activity sequence are identified as having anomalies based on differences in length exceeding a predefined threshold.

21. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 9 to 16.

Images