A method and system for capturing motion anomalies based on behavioral signatures

Abstract

This invention discloses a method and system for capturing abnormal movements based on behavioral feature recognition, belonging to the field of behavioral feature recognition technology. The method includes acquiring continuous image sequences and performing pose estimation and multi-dimensional behavioral feature extraction. The system combines personalized historical training records of rehabilitation personnel to construct individualized behavioral standards. Through a behavioral temporal variation clustering mechanism, the system segments the temporal data and aligns it through dynamic time warping to ensure accurate comparison of behavioral features. Downsampling further simplifies the data, helping the system observe movements at multiple time scales and improving the ability to identify abnormal behaviors. Finally, a pre-trained convolutional neural network is used to detect anomalies in the temporal features. This method can accurately capture abnormal movements during the rehabilitation process, ensuring the safety and effectiveness of training and providing precise support for personalized rehabilitation training.

Description

Technical Field

[0001] This invention relates to the field of behavioral feature recognition technology, and in particular to a method and system for capturing abnormal actions based on behavioral feature recognition. Background Technology

[0002] In current rehabilitation training, behavioral monitoring of patients is crucial. With the development of technology, many methods have begun to be applied to monitor patients' motor status, such as using sensors or cameras to analyze their posture and movements. However, these traditional methods often rely on manual assessment, lacking real-time performance and high accuracy, and are prone to misjudgment and missed detection, especially when faced with complex or irregular movements.

[0003] Existing technologies mostly rely on static, standard models for assessment, but these methods fail to consider the individual differences of each patient and fail to effectively integrate diverse behavioral data. This makes it difficult to accurately capture each patient's specific behavioral patterns, especially when personalized analysis is required during the treatment process, thus failing to provide sufficient support. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for capturing motion anomalies based on behavioral feature recognition, which solves the problems of poor accuracy, lack of personalized evaluation, and inability to handle complex actions in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for capturing abnormal actions based on behavioral feature recognition, which includes acquiring a continuous image sequence of a person undergoing treatment, and performing pose estimation and multi-dimensional behavioral feature extraction on the image sequence;

[0008] A behavioral profile is constructed based on the extracted behavioral features, and personalized behavioral standards are formed by combining the historical rehabilitation training records and behavioral experience samples of the convalescent patients.

[0009] The deviation between the current behavior profile and personalized behavior standards is analyzed, and anomaly assessment results are generated based on the behavior time-series variation clustering mechanism;

[0010] The behavioral temporal variation clustering mechanism includes dividing the continuous behavioral spectrum of the convalescents into multiple fixed-length temporal behavioral subgraphs, and constructing a temporal behavioral feature sequence based on the graph structure features of each subgraph; applying dynamic time warping to the behavioral feature sequence to calculate the behavioral deviation sequence between it and the personalized behavioral standard, and inputting the deviation sequence into a density clusterer with a sliding window; evaluating the association of anomalies in each subgraph according to the density clustering of different deviations, and finally outputting the action anomaly segment identification result.

[0011] As a preferred embodiment of the action anomaly capture method based on behavior feature recognition described in this invention, the pose estimation and multi-dimensional behavior feature extraction include: performing pose estimation on each frame of a continuous image sequence using a deep learning model, identifying the key joint positions of the human skeleton and evaluating the overall pose.

[0012] The overall posture is generated by the deep learning model using the relationship between actions between consecutive frames to create the movement state of the convalescent patient.

[0013] The key joints are pre-identified joints in the human skeleton. Specifically, the joints in the human skeleton are pre-identified so that the identified joints can be recognized during the learning process of the deep learning model, thereby ignoring unidentified joints.

[0014] Based on the position of the key joint in consecutive frames of images, the center of gravity trajectory, joint angle change, and inter-joint velocity covariance of the key joint in each image are calculated to obtain the multi-dimensional behavioral features.

[0015] As a preferred embodiment of the action anomaly capture method based on behavior feature recognition according to the present invention, the behavior graph includes the key joints as nodes of the graph and the multi-dimensional behavior features as node attributes.

[0016] The edge between any two nodes in the figure is used to represent the body mechanism coupling relationship between the corresponding two key joints in the rehabilitation training action, and the edge calculates the mechanism coupling parameter between the joints based on the node attributes of the two nodes;

[0017] The mechanism coupling parameters are a multidimensional set of vectors, each vector in the set representing a contribution to a function of the organism; wherein, the contribution to each function of the organism is obtained by mapping the node attributes of the two key joints to the corresponding organism function evaluation space through a pre-trained mapping model.

[0018] As a preferred embodiment of the action anomaly capture method based on behavioral feature recognition described in this invention, the personalized behavioral standard includes: after medical staff selects a supervised training option, based on the historical standard recuperation training record and corresponding behavioral map sequence of the designated recuperative under the training option, an individual physical function benchmark model is generated to describe the physical function state of the designated recuperative under each recuperation training action.

[0019] The historical standard recuperation training records were conducted under the supervision of medical staff.

[0020] The individual bodily function benchmark model includes generating functional benchmark intervals for each bodily function in each frame of the behavior map sequence by statistically analyzing the distribution of the mechanistic coupling parameter vectors of each node pair in the historical behavior map in the bodily function evaluation space, thereby defining the normal range of change of each bodily function of the designated convalescent personnel during the convalescent training process.

[0021] As a preferred embodiment of the action anomaly capture method based on behavior feature recognition described in this invention, wherein: the behavior graph of each frame of the image in the temporal behavior subgraph is characterized, and the node attributes and edges in the behavior graph are used as the graph structure features;

[0022] After standardizing the graph structure features of each frame image, the graph structure features of consecutive frames are smoothed, and all the processed sub-graphs are spliced ​​together in temporal order to obtain the temporal behavior feature sequence.

[0023] The standardization process eliminates noise and scale differences between different frames; the smoothing process removes fluctuations caused by motion noise, ensuring the stability of temporal behavior features.

[0024] As a preferred embodiment of the action anomaly capture method based on behavior feature recognition described in this invention, the deviation sequence includes: using the midpoint of each functional benchmark interval in each frame of the image sequence of the personalized behavior standard as a comparison benchmark; aligning the comparison benchmark with the temporal behavior feature sequence using dynamic time warping; and obtaining a behavior deviation measurement sequence by calculating the difference between the aligned temporal behavior feature sequence and the corresponding comparison benchmark.

[0025] Simultaneously, based on the alignment relationship between the temporal behavior feature sequence and each comparison benchmark, the system analyzes whether each edge feature of each frame image in the temporal behavior feature sequence exceeds the corresponding functional benchmark range; if it does, an anomaly marker is generated; otherwise, no marker is generated.

[0026] As a preferred embodiment of the action anomaly capture method based on behavior feature recognition described in this invention, the density clusterer includes: in each frame, clustering adjacent frames according to the distribution of the sequence based on the ratio of the deviation of each function to the length of the function reference interval as a metric, and segmenting according to the clustering results, and treating each segment of continuous frame image as an observation individual for behavior feature recognition.

[0027] In each frame of the clustered temporal segment, n downsampling results are generated by downsampling n times, and the deep learning model is used to identify the multi-dimensional behavioral features in each downsampling result; where n is the preset number of downsampling times.

[0028] By constructing the behavior map, the temporal behavior feature sequence of each downsampling result is obtained;

[0029] According to the segmentation method of the observed individuals, the temporal behavioral feature sequence of each downsampling result is segmented, and each segment contains n downsampled temporal behavioral feature sequences and one unsampled temporal behavioral feature sequence;

[0030] Using n+1 states as all uncertain states for each observation individual, and employing the aforementioned deviation sequence calculation method, dynamic time warping is performed again. During the alignment process, each observation individual can switch between its own n+1 states.

[0031] With the goal of minimizing the total deviation, the state of each observed individual is confirmed, and the temporal behavioral feature sequence corresponding to each observed individual is obtained.

[0032] Based on the temporal behavioral feature sequence corresponding to each observed individual and the anomaly marker, a pre-trained convolutional neural network is used to generate the capture result of the action anomaly.

[0033] Secondly, the present invention provides a motion anomaly capture system based on behavioral feature recognition, comprising: a data acquisition unit for acquiring a continuous image sequence of a person undergoing rehabilitation, and performing pose estimation and multi-dimensional behavioral feature extraction on the image sequence;

[0034] The comparison unit constructs a behavior map based on the extracted behavioral features, and combines it with the historical rehabilitation training records and behavioral experience samples of the convalescent patients to form personalized behavior standards.

[0035] The evaluation unit analyzes the deviation between the current behavior profile and personalized behavior standards, and generates anomaly evaluation results based on the behavior time-series variation clustering mechanism.

[0036] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the action anomaly capture method based on behavioral feature recognition as described in the first aspect of the present invention.

[0037] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the action anomaly capture method based on behavioral feature recognition as described in the first aspect of the present invention.

[0038] The beneficial effects of this invention are as follows: it enables real-time monitoring of the behavior of convalescent patients, accurately capturing and analyzing any anomalies during the training process. By developing personalized behavioral standards, taking into account each patient's historical records and behavioral experience, it ensures that anomaly detection can be adjusted according to individual circumstances, avoiding misjudgments that may arise from uniform standards in traditional methods. Furthermore, by utilizing a temporal variation clustering mechanism to divide behavioral features into multiple segments and performing multi-level analysis, the detection effect is further improved. Through techniques such as dynamic time warping and density clustering, it can accurately capture subtle changes in behavior, enhancing the system's stability. Finally, with the aid of a pre-trained convolutional neural network, abnormal behavior can be identified more accurately, improving the efficiency and real-time performance of motion anomaly capture. This method provides strong support for personalized and precise convalescent training. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a method for capturing action anomalies based on behavioral feature recognition.

[0041] Figure 2 This is a diagram of a computer device used for an action anomaly capture method based on behavioral feature recognition. Detailed Implementation

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0044] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0045] Reference Figures 1-2 This is one embodiment of the present invention, which provides a method for capturing action anomalies based on behavioral feature recognition, including the following steps:

[0046] S1: Collect a continuous image sequence of the convalescents, and perform pose estimation and multi-dimensional behavioral feature extraction on the image sequence.

[0047] Furthermore, the pose estimation and multi-dimensional behavioral feature extraction include performing pose estimation on each frame of a continuous image sequence using a deep learning model, identifying the key joint positions of the human skeleton, and evaluating the overall pose.

[0048] The overall posture is generated by the deep learning model using the relationship between actions between consecutive frames to determine the movement state of the person in the convalescence; it includes macroscopic movement states such as raising the leg and lowering the arm.

[0049] The key joints are pre-identified joints in the human skeleton. Specifically, the joints in the human skeleton are pre-identified, enabling the deep learning model to recognize these joints and ignore unidentified joints. By estimating the pose of each frame using the deep learning model, the model can accurately identify the positions of key joints when patients perform specific movements, thus capturing their overall movement state. By focusing on important joints, computational complexity is reduced, and excessive redundant details are avoided.

[0050] Based on the position of the key joint in consecutive frames of images, the center of gravity trajectory, joint angle change, and inter-joint velocity covariance of the key joint in each image are calculated to obtain the multi-dimensional behavioral features.

[0051] Specifically, the system includes: **Center of Gravity Trajectory:** By calculating changes in the center of gravity position in each frame, the patient's balance and gait are assessed. For example, drastic changes in the center of gravity may indicate that the patient is out of balance and at risk of falling. **Joint Angle Changes:** By analyzing joint angle changes in each frame, the system assesses the patient's joint flexibility and range of motion, revealing joint coordination and range of motion, and helping to identify uncoordinated or restricted movements. **Inter-Joint Velocity Covariance:** By calculating the velocity changes of each joint in each frame, the system assesses the coordination between joints. Large differences in velocity changes between joints may indicate poor coordination and a risk of behavioral abnormalities. These features provide quantitative evidence for analyzing the movement performance and deviations of patients during training, thus providing precise data support for identifying abnormal behaviors.

[0052] S2: Construct a behavioral map based on the extracted behavioral features, and combine it with the historical rehabilitation training records and behavioral experience samples of the convalescent patients to form personalized behavioral standards.

[0053] The behavioral graph includes key joints as nodes and multi-dimensional behavioral features as node attributes. An edge between any two nodes in the graph represents the mechanistic coupling relationship between the corresponding two key joints in the therapeutic training movement, and the mechanistic coupling parameters between the joints are calculated based on the node attributes of the two nodes.

[0054] By transforming the movements of patients into behavioral maps, and utilizing the structured representation of these maps to capture the motion of key joints during training, we can analyze their contribution to functional recovery. Specifically, by treating each key joint as a node in the graph and constructing the graph structure in conjunction with the joint's motion characteristics, we can gain a clearer understanding of the interactions between joints and the coordination in movements. During training, the coordination between joints is crucial for functional recovery; therefore, representing the mechanistic coupling relationships between joints through the edges of the graph helps analyze whether movements meet recovery standards.

[0055] The mechanism coupling parameters are a multidimensional set of vectors, each vector in the set representing a contribution to a function of the organism (because the purpose of training is to restore the organism's function, the functional expression of each frame of the image can be mapped through the attributes of the two nodes, thus enabling effective observation of the training process); wherein, the contribution result of each organism function is obtained by mapping the node attributes of the two key joints to the corresponding organism function evaluation space through a pre-trained mapping model.

[0056] "Contribution to a bodily function" refers to the impact of each joint's movement during training on the recovery of bodily functions (such as flexibility, strength, and coordination). By calculating the coupling relationship between two key joints, their contribution to a particular bodily function can be assessed. For example, in gait training, the knee joint may help recover lower limb strength, while the hip joint may contribute to upper limb coordination. Such calculations allow for the assessment of whether training movements effectively promote the recovery of bodily functions and the identification of any uncoordinated or abnormal movements.

[0057] A pre-trained mapping model refers to a model that has been trained on a large amount of data. Its purpose is to map input behavioral features (e.g., motion data of key joints, angle changes, velocity, etc.) to a specific functional evaluation space and evaluate results related to bodily function based on these input data. This model has learned the relationship between behavioral features and bodily function, and can map new input behavioral features to a predefined functional space to evaluate recovery effects.

[0058] Specifically, the pre-trained mapping model, based on historical data and expertise, can generate a mechanistic coupling parameter for each pair of key joints after training. These parameters reflect the contribution of that joint's movement to a specific function. For example, knee joint movement may affect lower limb strength recovery, while hip joint movement may affect upper limb coordination recovery. Through the pre-trained model, joint motion data is transformed into a quantifiable metric representing its contribution to specific bodily functions (such as strength, flexibility, and coordination).

[0059] In this embodiment, the mapping model is a pre-trained deep neural network. This network learns the relationship between behavioral features and bodily functions from historical data, enabling it to map the behavioral features of each joint to a bodily function score. The model optimizes the network weights during training to ensure accurate assessment of the impact of each movement on bodily function recovery. Specifically, the deep neural network automatically extracts and maps the degree of bodily function recovery by learning multi-dimensional behavioral features (such as joint angles and speed changes).

[0060] In other alternative embodiments, the mapping model can also be a Support Vector Machine (SVM), which learns nonlinear relationships in the training data to map the behavioral features of each pair of joints to the organism's functional evaluation space. SVMs can construct hyperplanes in high-dimensional space to distinguish between normal and abnormal behavioral features, further improving the accuracy of anomaly detection. Furthermore, the mapping model can also employ a decision tree model, classifying behavioral features through a tree structure to identify abnormal behaviors that do not meet the recovery criteria.

[0061] The personalized behavioral standards include an individual functional baseline model generated by healthcare professionals after they select supervised training options. This model is based on the historical standard rehabilitation training records and corresponding behavioral atlas sequences of the designated patient under each training option. This model describes the patient's functional state under each rehabilitation training movement. "Healthcare professionals select supervised training options" means that they choose appropriate training programs or movements based on the patient's health condition and recovery needs, and provide guidance and supervision during training. For example, they might select suitable movements (such as leg raises or stretches) based on the patient's specific situation, and determine the intensity and frequency of training to ensure it is challenging but not excessively burdensome. In short, healthcare professionals select appropriate training content based on each patient's individual circumstances and ensure the training is conducted in a scientific manner.

[0062] The historical standard recuperation training records were obtained through training conducted under the supervision of medical staff.

[0063] The individual bodily function benchmark model includes generating functional benchmark intervals for each bodily function in each frame of the behavior map sequence by statistically analyzing the distribution of the mechanistic coupling parameter vectors of each node pair in the historical behavior map in the bodily function evaluation space, thereby defining the normal range of change of each bodily function of the designated convalescent personnel during the convalescent training process.

[0064] It's important to note that each person's recovery progresses at a different pace, so a uniform standard cannot be used to assess the training effectiveness for everyone. By analyzing each individual's past behavioral data, their normal behavioral range during rehabilitation can be determined. Specifically, this involves calculating the joint movements and other behavioral characteristics observed during past training to determine the normal fluctuation range of their functional recovery. If behavior deviates from this range, abnormalities can be identified promptly, allowing for effective monitoring of their recovery progress and helping medical staff make adjustments.

[0065] S3: Analyze the deviation between the current behavior profile and personalized behavior standards, and generate anomaly assessment results based on the behavior time-series variation clustering mechanism.

[0066] Furthermore, the behavioral graph of each frame in the temporal behavioral subgraph is characterized, and the node attributes and edges in the behavioral graph are used as the graph structure features. After standardization of the graph structure features of each frame, the graph structure features of consecutive frames are smoothed, and all processed subgraphs are concatenated temporally to obtain the temporal behavioral feature sequence.

[0067] The standardization process eliminates noise and scale differences between different frames; the smoothing process removes fluctuations caused by motion noise, ensuring the stability of temporal behavior features.

[0068] It's important to note that the behavioral atlas is constructed using the key joint positions in each image frame. However, there may be noise or scale differences between different image frames, affecting data consistency. Standardization is used to eliminate these differences, making the features of each image frame uniform and facilitating subsequent analysis. Next, smoothing is used to remove random fluctuations caused by motion, which may not reflect actual actions but are merely irrelevant noise. Smoothing makes the motion features more stable and realistic. Finally, all behavioral features are concatenated in chronological order to form a complete temporal behavioral feature sequence, ensuring a coherent description of the participants' movement changes throughout the training process.

[0069] The behavioral temporal variation clustering mechanism includes dividing the continuous behavioral spectrum of the convalescents into multiple temporal behavioral subgraphs of fixed length (in other optional embodiments, it can be divided according to actions to generate multiple sub-action segments), and constructing a temporal behavioral feature sequence based on the graph structure features of each subgraph; applying dynamic time warping to the behavioral feature sequence to calculate the behavioral deviation sequence between it and the personalized behavioral standard, and inputting the deviation sequence into a density clusterer with a sliding window; evaluating the correlation of anomalies in each subgraph according to the density clustering of different deviations, and finally outputting the action anomaly segment identification result.

[0070] Furthermore, in the image sequence of the personalized behavior standard, the midpoint of the benchmark interval for each function in each frame is used as the comparison benchmark (each function is compared separately); the comparison benchmark and the temporal behavior feature sequence are aligned using dynamic time warping, and the behavior deviation measurement sequence is obtained by calculating the difference between the aligned temporal behavior feature sequence and the corresponding comparison benchmark.

[0071] In each training exercise, a midpoint is established for the baseline interval of each function. This midpoint serves as a comparison benchmark, used to compare against each frame in the temporal behavioral feature sequence. This ensures that each function (e.g., strength, coordination) can be compared individually, guaranteeing that changes in behavioral features accurately match the standard. Dynamic time warping effectively handles misalignments in the time series, such as differences in action duration or speed, ensuring that the behavioral features of each frame accurately match the benchmark. Once the behavioral feature sequence and personalized standard are aligned using DTW, the behavioral features at each time step are compared to the benchmark value. This alignment allows for a more precise measurement of the difference between the behavioral features and the standard, resulting in a behavioral deviation metric sequence that reflects the deviation of each frame from the benchmark.

[0072] Simultaneously, based on the alignment relationship between the temporal behavior feature sequence and each comparison benchmark, the system analyzes whether each edge feature of each frame image in the temporal behavior feature sequence exceeds the corresponding functional benchmark range; if it does, an anomaly marker is generated; otherwise, no marker is generated.

[0073] The density clusterer includes clustering adjacent frames according to the distribution of the sequence based on the ratio of the deviation of each function to the length of the functional baseline interval in each frame, and segmenting the images according to the clustering results. Each segment of continuous frame images is treated as an observation individual for behavioral feature identification. It is important to note that this is to ensure a unified and standardized measurement of deviations across different functions. During the rehabilitation process, each individual may experience varying progress and magnitudes of functional changes. By calculating the ratio of deviation to the length of the functional baseline interval, scale differences arising from varying ranges of change in different functions can be eliminated, allowing for comparison of changes in all functions under a unified standard. This not only improves the accuracy of abnormal behavior detection but also ensures the comparability of deviations between different functions, avoiding misjudgments due to excessively large or small baseline intervals for certain functions. Through this method, the system can more accurately identify behaviors deviating from the normal range and ensure that abnormal detection adapts to the actual situation of each individual during various functional recovery processes.

[0074] In each frame of the clustered time-series image segment, n downsampling operations are performed to generate n downsampling results. The deep learning model is then used to identify multi-dimensional behavioral features in each downsampling result; where n is the preset number of downsampling operations. The purpose of downsampling is to simplify the data and reduce redundant information, enabling the system to analyze the same action at different time scales. Each downsampling result represents a different level of observation of behavioral features, helping to capture detailed changes in action at multiple scales. This not only reduces the amount of data and improves processing efficiency but also eliminates the influence of motion instability or noise, making feature extraction more stable. Furthermore, downsampling enhances the model's robustness to irregular actions, helping the system identify abnormal behaviors that only appear at certain scales. Through this method, the system can more accurately identify abnormal actions, ensuring effective monitoring of the training process for rehabilitation personnel.

[0075] By constructing the behavioral map, the temporal behavioral feature sequence of each downsampling result is obtained.

[0076] According to the segmentation method of the observed individuals, the temporal behavioral feature sequence of each downsampling result is segmented, and each segment contains n downsampled temporal behavioral feature sequences and one unsampled temporal behavioral feature sequence.

[0077] Using the n+1 states as all uncertain states for each observation individual, and employing the aforementioned deviation sequence calculation method, dynamic time warping is performed again. During the alignment process, each observation individual can switch between its own n+1 states.

[0078] With the goal of minimizing the total deviation, the state of each observed individual is confirmed, resulting in a temporal behavioral feature sequence (either after downsampling or before downsampling) corresponding to each individual. Based on the temporal behavioral feature sequence and the anomaly markers for each observed individual, a pre-trained convolutional neural network is used to generate capture results of action anomalies.

[0079] It's worth noting that through multi-level, multi-scale processing, the system can comprehensively analyze each movement from different angles, ensuring accurate identification of both macroscopic performance and subtle changes in detail. By combining downsampling and raw data, the system can observe the same movement at different scales. This allows the system to flexibly adapt to different training modes, such as differences in the rhythm, speed, or duration of movements of patients, all of which can affect the performance of normal behavior. Downsampled data helps the system capture the overall trend of movements at a macroscopic level while maintaining attention to details, thereby improving the robustness of anomaly detection. Dynamic Time Warping (DTW) technology helps the system align differences between different behavioral states, ensuring accurate comparison even for movements that are not synchronized in time. This ensures that regardless of changes in movement speed during training, the system can effectively compare movements to determine which deviate from personalized standards, ultimately helping to identify abnormal behavior.

[0080] It's important to understand that the alignment process of temporal behavioral features transforms the behavioral features of each frame of an image from a "quantum state" to a final state determined according to the target. This is similar to how the system continuously adjusts and optimizes the action state at each moment by aligning behavioral features at different time scales until the final state best matches the personalized behavioral standards.

[0081] During rehabilitation training, the behavioral characteristics represented by each frame of an image are often initially uncertain, potentially containing noise or bias. This is similar to a quantum state, existing in a fuzzy and uncertain state. This state does not directly conform to personalized training standards but requires adjustment and comparison to approach the ideal state.

[0082] Using Dynamic Time Warping (DTW) technology, the system progressively adjusts these initial uncertain states (“quantum states”) to align them with preset, personalized target standards. This target state is defined based on the rehabilitation goals of the patients and is an ideal, definite state. During the alignment process, the system calculates the deviation between each time segment and the target standard and progressively corrects these deviations until each action feature matches the target state as closely as possible.

[0083] This process ensures that regardless of how the patient's movements change, the system can accurately compare them across multiple scales. Deviations and differences in each temporal behavioral feature are effectively identified, allowing the system to determine which movements deviate from the normal range and which are expected. This transition from a quantum state to a target state is essentially a gradual alignment and adjustment process that transforms ambiguous and uncertain behavioral features into standardized, target-compliant movement states. Ultimately, the system can accurately identify abnormal behaviors, thereby optimizing the training process and ensuring that patients train according to predetermined goals during rehabilitation.

[0084] This embodiment also provides a motion anomaly capture system based on behavioral feature recognition, including: a data acquisition unit, which acquires a continuous image sequence of convalescents, and performs pose estimation and multi-dimensional behavioral feature extraction on the image sequence.

[0085] The comparison unit constructs a behavior map based on the extracted behavioral features, and combines it with the historical rehabilitation training records and behavioral experience samples of the convalescent patients to form personalized behavior standards.

[0086] The evaluation unit analyzes the deviation between the current behavior profile and personalized behavior standards, and generates anomaly evaluation results based on the behavior time-series variation clustering mechanism.

[0087] This embodiment also provides a computer device applicable to the action anomaly capture method based on behavioral feature recognition, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the action anomaly capture method based on behavioral feature recognition as proposed in the above embodiment.

[0088] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0089] This embodiment also provides a storage medium on which a computer program is stored. When executed by a processor, the program implements the action anomaly capture method based on behavioral feature recognition as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0090] In summary, this invention establishes personalized behavioral standards for each individual undergoing rehabilitation by acquiring continuous image sequences, performing pose estimation and multi-dimensional behavioral feature extraction, and combining this with personalized historical training records. Based on this, the system segments the data using a behavioral temporal variation clustering mechanism and aligns it through dynamic time warping to ensure that the behavioral features of each temporal segment are aligned with the standards, avoiding errors caused by time asynchrony. Through downsampling and multi-scale analysis, the system observes actions at multiple levels, capturing subtle abnormal changes from different angles, and ultimately identifying abnormal behaviors through a convolutional neural network. This not only improves the accuracy of anomaly detection but also provides precise monitoring based on each individual's rehabilitation progress, ensuring the safety and effectiveness of the rehabilitation training process.

[0091] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for capturing action anomalies based on behavioral feature recognition, characterized in that: This includes collecting a continuous sequence of images of people undergoing treatment, and performing pose estimation and multi-dimensional behavioral feature extraction on the image sequences; A behavioral profile is constructed based on the extracted behavioral features, and personalized behavioral standards are formed by combining the historical rehabilitation training records and behavioral experience samples of the convalescent patients. The deviation between the current behavior profile and personalized behavior standards is analyzed, and anomaly assessment results are generated based on the behavior time-series variation clustering mechanism; The behavioral temporal variation clustering mechanism includes dividing the continuous behavioral spectrum of the convalescents into multiple fixed-length temporal behavioral subgraphs, and constructing a temporal behavioral feature sequence based on the graph structure features of each subgraph; applying dynamic time warping to the behavioral feature sequence to calculate the behavioral deviation sequence between it and the personalized behavioral standard, and inputting the deviation sequence into a density clusterer with a sliding window; evaluating the association of anomalies in each subgraph according to the density clustering of different deviations, and finally outputting the action anomaly segment identification result.

2. The action anomaly capture method based on behavioral feature recognition as described in claim 1, characterized in that: The pose estimation and multi-dimensional behavioral feature extraction include performing pose estimation on each frame of a continuous image sequence using a deep learning model, identifying the key joint positions of the human skeleton and evaluating the overall pose. The overall posture is generated by the deep learning model using the relationship between actions between consecutive frames to create the movement state of the convalescent patient. The key joints are pre-identified joints in the human skeleton. Specifically, the joints in the human skeleton are pre-identified so that the identified joints can be recognized during the learning process of the deep learning model, thereby ignoring unidentified joints. Based on the position of the key joint in consecutive frames of images, the center of gravity trajectory, joint angle change, and inter-joint velocity covariance of the key joint in each image are calculated to obtain the multi-dimensional behavioral features.

3. The action anomaly capture method based on behavioral feature recognition as described in claim 2, characterized in that: The behavior graph includes the key joints as nodes of the graph and the multi-dimensional behavioral features as node attributes. The edge between any two nodes in the figure is used to represent the body mechanism coupling relationship between the corresponding two key joints in the rehabilitation training action, and the edge calculates the mechanism coupling parameter between the joints based on the node attributes of the two nodes; The mechanism coupling parameters are a multidimensional set of vectors, each vector in the set representing a contribution to a function of the organism; wherein, the contribution to each function of the organism is obtained by mapping the node attributes of the two key joints to the corresponding organism function evaluation space through a pre-trained mapping model.

4. The action anomaly capture method based on behavioral feature recognition as described in claim 3, characterized in that: The personalized behavioral standards include, after medical staff select the supervised training options, an individual physical function benchmark model is generated based on the historical standard recuperation training records and corresponding behavioral map sequences of the designated recuperation personnel under the training options, which is used to describe the physical function status of the designated recuperation personnel under each recuperation training action; The historical standard recuperation training records were conducted under the supervision of medical staff; The individual bodily function benchmark model includes generating functional benchmark intervals for each bodily function in each frame of the behavior map sequence by statistically analyzing the distribution of the mechanistic coupling parameter vectors of each node pair in the historical behavior map in the bodily function evaluation space, thereby defining the normal range of change of each bodily function of the designated convalescent personnel during the convalescent training process.

5. The action anomaly capture method based on behavioral feature recognition as described in claim 4, characterized in that: In the temporal behavior subgraph, the behavior graph of each frame image is characterized, and the node attributes and edges in the behavior graph are used as the graph structure features. After standardizing the graph structure features of each frame image, the graph structure features of consecutive frames are smoothed, and all the processed sub-graphs are spliced ​​together in temporal order to obtain the temporal behavior feature sequence. In this process, standardization eliminates noise and scale differences between different frames; and smoothing removes fluctuations caused by motion noise, ensuring the stability of temporal behavior features.

6. The action anomaly capture method based on behavioral feature recognition as described in claim 5, characterized in that: The deviation sequence includes: using the midpoint of each functional baseline interval in each frame of the image sequence of the personalized behavior standard as a comparison baseline; aligning the comparison baseline with the temporal behavior feature sequence using dynamic time warping; and obtaining a behavior deviation measurement sequence by calculating the difference between the aligned temporal behavior feature sequence and the corresponding comparison baseline. Meanwhile, based on the alignment relationship between the temporal behavior feature sequence and each comparison benchmark, it is analyzed whether each edge feature of each frame image in the temporal behavior feature sequence exceeds the corresponding functional benchmark range. If the error exceeds the limit, an exception flag will be generated. Conversely, no marking is performed.

7. The action anomaly capture method based on behavioral feature recognition as described in claim 6, characterized in that: The density clusterer includes, in each frame, using the ratio of the deviation of each function to the length of the function reference interval as a metric, clustering adjacent frames according to the distribution of the metric in the sequence, and segmenting according to the clustering results, and treating each segment of continuous frame image as an observation individual for behavioral feature recognition. In each frame of the clustered temporal segment, n downsampling results are generated by downsampling n times, and the deep learning model is used to identify the multi-dimensional behavioral features in each downsampling result; where n is the preset number of downsampling times. By constructing the behavior map, the temporal behavior feature sequence of each downsampling result is obtained; According to the segmentation method of the observed individuals, the temporal behavioral feature sequence of each downsampling result is segmented, and each segment contains n downsampled temporal behavioral feature sequences and one unsampled temporal behavioral feature sequence; Using n+1 states as all uncertain states for each observation individual, and employing the aforementioned deviation sequence calculation method, dynamic time warping is performed again. During the alignment process, each observation individual can switch between its own n+1 states. With the goal of minimizing the total deviation, the state of each observed individual is confirmed, and the temporal behavioral feature sequence corresponding to each observed individual is obtained. Based on the temporal behavioral feature sequence corresponding to each observed individual and the anomaly marker, a pre-trained convolutional neural network is used to generate capture results of action anomalies.

8. A motion anomaly capture system based on behavioral feature recognition, based on the motion anomaly capture method based on behavioral feature recognition according to any one of claims 1 to 7, characterized in that: Includes a data acquisition unit that acquires a continuous sequence of images of the patients and performs pose estimation and multi-dimensional behavioral feature extraction on the image sequence; The comparison unit constructs a behavior map based on the extracted behavioral features, and combines it with the historical rehabilitation training records and behavioral experience samples of the convalescent patients to form personalized behavior standards. The evaluation unit analyzes the deviation between the current behavior profile and the personalized behavior standard, and generates anomaly evaluation results based on the behavior time-series variation clustering mechanism.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the action anomaly capture method based on behavioral feature recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the action anomaly capture method based on behavioral feature recognition as described in any one of claims 1 to 7.

Images