Mobile joint activity monitoring method and system

Abstract

The invention relates to the technical field of medical health monitoring, and discloses a mobile joint movement monitoring method, which comprises the following steps of: setting a tracking network of joint movement, and collecting a real-time image of the joint movement; identifying a key joint part of the monitored object, analyzing a joint movement rule of the key joint part, and extracting joint movement characteristics of the monitored object; identifying the movement track of the key joint part in the joint movement mode, and analyzing the abnormal mode of the joint movement; abnormal activity characteristics of the key joint part in the abnormal mode are extracted, and an abnormal judgment mechanism and a visual feedback interface of joint activity are set; recognizing normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring switching mechanism of the joint movement; in combination with a tracking network, a visual feedback interface and a dynamic monitoring switching mechanism, the joint movement is monitored in real time, and a real-time monitoring result is obtained. According to the invention, the comprehensiveness and accuracy of joint movement monitoring can be improved.

Description

Technical Field

[0001] The present invention relates to the field of medical health monitoring technology, and in particular to a mobile joint activity monitoring method and system. Background Art

[0002] Mobile joint motion monitoring refers to the use of portable devices to monitor and evaluate an individual's joint mobility in real time, and to improve the efficiency and quality of joint motion through intelligent monitoring and feedback mechanisms. With the acceleration of global aging, the incidence of joint diseases is increasing, which not only affects the quality of life of patients, but also increases the medical burden. Therefore, real-time monitoring and evaluation of joint motion has become increasingly important.

[0003] Traditional joint movement monitoring methods mainly use special mechanical devices, such as protractors, to measure the range of motion of joints. Although this method is simple and intuitive, it can only provide static measurement results and cannot reflect dynamic joint movement, affecting the comprehensiveness and accuracy of joint movement monitoring results. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a mobile joint activity monitoring method and system, which can improve the comprehensiveness and accuracy of joint activity monitoring.

[0005] In a first aspect, the present invention provides a mobile joint activity monitoring method, comprising: Acquire the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect a real-time image of the joint activity based on the tracking network; Based on the real-time image, identify the key joint parts of the monitored object, analyze the joint movement rules of the key joint parts, and extract the joint movement characteristics of the monitored object according to the joint movement rules; Analyzing the joint movement pattern of the monitored object according to the joint movement characteristics, identifying the movement trajectory of the key joint parts under the joint movement pattern, and analyzing the abnormal pattern of the joint movement based on the movement trajectory; Extracting abnormal activity features of the key joint parts in the abnormal mode, setting an abnormality judgment mechanism for the joint activity based on the abnormal activity features, and setting a visual feedback interface for the joint activity according to the abnormality judgment mechanism; Identifying normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring and switching mechanism for the joint movement according to the normal joint movement characteristics; In combination with the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism, the joint activity is monitored in real time to obtain real-time monitoring results.

[0006] In a possible implementation of the first aspect, setting the tracking network of the joint movement includes: Identify the joint part corresponding to the joint movement, and configure the sensor device of the joint movement based on the joint part; According to the sensing device, a data acquisition unit for the joint activity is provided; defining a communication method between the sensor device and the data acquisition unit; extracting the activity data of the joint movement based on the sensing device and the data acquisition unit, and configuring a data processor of the activity data; Identifying data processing results of the data processor and constructing a visualization interface of the data processing results; In combination with the sensor device, the data acquisition unit, the communication method, the data processor and the visualization interface, a tracking network for the joint activity is set up.

[0007] In a possible implementation of the first aspect, collecting the real-time image of the joint movement based on the tracking network includes: Based on the tracking network, collecting dynamic data of the joint movement; Extracting characteristic parameters of the joint movement according to the dynamic data; Based on the characteristic parameters, construct a 3D joint model corresponding to the joint movement; According to the 3D joint model, simulating the skeletal animation of the joint movement; Identify the 3D joint model and the 2D image sequence of the skeletal animation; Based on the two-dimensional image sequence, real-time images of the joint movement are acquired.

[0008] In a possible implementation of the first aspect, analyzing the joint movement pattern of the key joint part includes: Collecting monitoring data corresponding to the key joint parts, and extracting time series data of the monitoring data; Analyzing the movement rhythm of the key joint parts according to the time series data; Based on the movement rhythm, identifying the range of motion of the key joint parts; Determine the conventional movement standard of the key joint parts by combining the movement rhythm and the movement range; According to the conventional movement standards, identifying specific movements of the key joints; Analyzing the specific movement pattern of the key joint parts in the specific movement; Combine the movement rhythm, the movement range and the specific movement pattern to analyze the joint movement patterns of the key joint parts.

[0009] In a possible implementation of the first aspect, analyzing the joint movement pattern of the monitored object according to the joint movement feature includes: Performing dimensionality reduction processing on the joint activity features to obtain dimensionality reduction features; Extracting characteristic keywords of the joint movement characteristics; Based on the characteristic keywords, identifying the derived characteristics of the monitored object; Performing feature fusion processing on the dimension reduction feature and the derived feature to obtain a fused feature; identifying the joint movement type of the monitored object according to the fusion feature; Based on the joint movement type, the joint movement pattern of the monitored object is analyzed.

[0010] In a second aspect, the present invention provides a mobile joint activity monitoring system, characterized in that the system comprises: A visual tracking module, used to obtain the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect a real-time image of the joint activity based on the tracking network; A feature extraction module, for identifying key joints of the monitored object based on the real-time image, analyzing joint movement patterns of the key joints, and extracting joint movement features of the monitored object according to the joint movement patterns; an abnormality identification module, used to analyze the joint movement pattern of the monitored object according to the joint movement characteristics, identify the movement trajectory of the key joint part under the joint movement pattern, and analyze the abnormal pattern of the joint movement based on the movement trajectory; A visual feedback module, used for extracting abnormal activity characteristics of the key joint parts in the abnormal mode, setting an abnormal judgment mechanism for the joint activity based on the abnormal activity characteristics, and setting a visual feedback interface for the joint activity according to the abnormal judgment mechanism; A monitoring mode setting module, used for identifying normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring switching mechanism for the joint movement according to the normal joint movement characteristics; The mobile monitoring module is used to combine the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint movement in real time and obtain real-time monitoring results.

[0011] Compared with the prior art, the technical principle and beneficial effects of this solution are: The embodiment of the present invention sets up the tracking network of the joint movement, which can track and record the joint movement in real time, improve the accuracy and reliability of the collected data, and also increase the flexibility and portability of the monitoring system; further, the embodiment of the present invention analyzes the joint movement rules of the key joint parts, can understand the movement mechanism of the human joints, extract the joint movement characteristics of the monitored object, identify and distinguish different movement patterns, and thus find abnormal or atypical movement patterns; secondly, the embodiment of the present invention identifies the movement trajectory of the key joint parts under the joint movement mode, which can help automatically identify and classify different movements, so as to better understand the behavior of the joints in a specific movement mode, help medical professionals identify potential movement disorders or diseases, and help evaluate the recovery process and treatment effect of joint patients, and also help to find the movement habits or risk factors that may lead to joint injuries, so as to take preventive measures; thirdly, the embodiment of the present invention sets up the abnormal judgment mechanism and visual feedback interface of the joint movement based on the abnormal activity characteristics, which can realize joint The automatic monitoring of activities helps users to immediately understand the abnormal situation of joint activities and take timely measures to reduce the intervention of medical staff; the embodiment of the present invention can identify the normal mode and abnormal mode of joint activities by constructing the dynamic monitoring switching mechanism of joint activities according to the characteristics of normal joint activities, and select the corresponding monitoring mode according to the identification results, which is helpful to improve the efficiency of monitoring and ensure that appropriate monitoring and intervention measures can be taken in time when abnormalities are detected; finally, the embodiment of the present invention combines the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint activities in real time to obtain real-time monitoring results, which can accurately capture the slight changes in joint activities and improve the accuracy and reliability of monitoring, and present the real-time monitoring results of joint activities to users in an intuitive and easy-to-understand form through the visual feedback interface, which helps users to find and correct wrong behaviors in time, and at the same time, the dynamic monitoring switching mechanism can respond to the changes of joint activities in real time according to the normal mode and abnormal mode of joint activities, so as to improve the monitoring efficiency and effect of joint activities. Therefore, the mobile joint activity monitoring method proposed by the present invention can improve the comprehensiveness and accuracy of joint activity monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0013] Figure 1 A schematic diagram of a flow chart of a mobile joint activity monitoring method provided by an embodiment of the present invention; Figure 2 A schematic diagram of modules of a mobile joint activity monitoring system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0015] The embodiment of the present invention provides a mobile joint activity monitoring method, and the execution subject of the mobile joint activity monitoring method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the mobile joint activity monitoring method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0016] See also Figure 1 FIG. 1 is a flow chart of a mobile joint activity monitoring method provided by an embodiment of the present invention. Figure 1 The mobile joint motion monitoring method described in the invention includes: S1. Acquire the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect real-time images of the joint activity based on the tracking network.

[0017] The embodiment of the present invention can clarify the monitoring target and improve the accuracy of subsequent joint activity characteristic analysis by acquiring the joint activity to be monitored and its corresponding monitoring object. The joint activity refers to the activity of the joint within the normal functional range, including various movements such as rotation, flexion and extension, adduction, and abduction. The monitoring object refers to the person participating in the joint activity monitoring, such as patients and athletes.

[0018] Furthermore, the embodiment of the present invention can track and record joint movements in real time by setting up the tracking network of joint movements, thereby improving the accuracy and reliability of collected data, while also increasing the flexibility and portability of the monitoring system. The tracking network refers to a portable monitoring device that tracks and monitors human joint movements.

[0019] As an embodiment of the present invention, the setting of the tracking network for the joint movement includes: identifying the joint parts corresponding to the joint movement, and configuring the sensing device for the joint movement based on the joint parts; setting the data acquisition unit for the joint movement according to the sensing device; defining the communication method between the sensing device and the data acquisition unit; extracting the activity data of the joint movement based on the sensing device and the data acquisition unit, and configuring a data processor for the activity data; identifying the data processing results of the data processor, and constructing a visualization interface for the data processing results; and setting the tracking network for the joint movement in combination with the sensing device, the data acquisition unit, the communication method, the data processor and the visualization interface.

[0020] Among them, the joint part refers to the joint that specifically performs activities in the human body, such as the knee joint, shoulder joint, elbow joint, etc., the sensing device refers to the device used to capture the activities of the joint part, such as accelerometers, gyroscopes, pressure sensors, etc., the data acquisition unit refers to the device responsible for collecting data from the sensor, usually including an analog-to-digital converter, a signal amplifier and a microprocessor, the communication method refers to the method of transmitting data between the sensor and the data acquisition unit, such as Bluetooth, Wi-Fi, wired connection, etc., the activity data refers to the original data about joint activities extracted from the sensor, including timestamp, position, speed, acceleration, etc., the data processor refers to the software or hardware system used to process and analyze activity data, the data processing result refers to the conclusion or information obtained by the data processor after analyzing and processing the original activity data, and the visualization interface refers to the user interface that displays the data processing results in a graphical manner.

[0021] Optionally, the configuration of the sensing device for the joint movement based on the joint position can be implemented using a sensor, such as a pressure sensor, and the construction of a visualization interface for the data processing results can be displayed through a data visualization tool, such as a Tableau tool.

[0022] The embodiment of the present invention can provide instant visual information of joint movement by collecting real-time images of the joint movement based on the tracking network, and help identify specific movement patterns or behaviors. The real-time image refers to image data captured in real time by a camera or other visual sensor.

[0023] As an embodiment of the present invention, the real-time image of the joint movement is collected based on the tracking network, including: collecting dynamic data of the joint movement based on the tracking network; extracting characteristic parameters of the joint movement based on the dynamic data; constructing a 3D joint model corresponding to the joint movement based on the characteristic parameters; simulating the skeletal animation of the joint movement based on the 3D joint model; identifying a two-dimensional image sequence of the 3D joint model and the skeletal animation; and collecting the real-time image of the joint movement based on the two-dimensional image sequence.

[0024] Among them, the dynamic data refers to data about joint movement collected in real time through a tracking network, the characteristic parameters refer to key information extracted from the dynamic data for describing the characteristics of joint movement, such as the range of change of joint angles, the 3D joint model refers to a digital model representing the joint and limb structure, the skeletal animation refers to an animation that controls and simulates the dynamic movement of joints through bones, and the two-dimensional image sequence refers to a series of 2D images rendered from the 3D joint model and skeletal animation.

[0025] Optionally, the extraction of characteristic parameters of the joint movement based on the dynamic data can be obtained through a convolutional neural network, the construction of a 3D joint model corresponding to the joint movement based on the characteristic parameters can be implemented using Blender modeling software, the skeletal animation simulation of the joint movement based on the 3D joint model can be determined by 3ds Max animation software, and the two-dimensional image sequence recognition of the 3D joint model and the skeletal animation can be implemented using a 3D rendering engine.

[0026] S2. Based on the real-time image, identify the key joint parts of the monitored object, analyze the joint movement patterns of the key joint parts, and extract the joint movement characteristics of the monitored object according to the joint movement patterns.

[0027] The embodiment of the present invention can accurately track and record the movement of the joints by identifying the key joint parts of the monitored object based on the real-time image to analyze the motion range, movement pattern and movement coordination of the joints. The key joint parts refer to the connection points of human joints or the movement centers of joints, such as ankle joints, knee joints, spinal joints, etc.

[0028] Optionally, the recognition of key joint parts of the monitored object based on the real-time image can be achieved by using a human posture estimation model, such as an OpenPose model.

[0029] Furthermore, the embodiments of the present invention can understand the movement mechanism of human joints and identify abnormal movement patterns during joint movement by analyzing the joint movement patterns of the key joint parts. The joint movement patterns refer to the rules followed by the joints when performing various movements, such as the range of motion.

[0030] As an embodiment of the present invention, the analysis of the joint movement pattern of the key joint part includes: collecting monitoring data corresponding to the key joint part, and extracting time series data of the monitoring data; analyzing the movement rhythm of the key joint part according to the time series data; identifying the movement range of the key joint part based on the movement rhythm; determining the conventional movement standard of the key joint part in combination with the movement rhythm and the movement range; identifying the specific movement of the key joint part according to the conventional movement standard; analyzing the specific movement pattern of the key joint part in the specific movement; analyzing the joint movement pattern of the key joint part in combination with the movement rhythm, the movement range and the specific movement pattern.

[0031] Among them, the monitoring data refers to the original data collected from key joints, the time series data refers to the sequence formed by arranging the monitoring data in chronological order, the movement rhythm refers to the periodic and rhythmic characteristics of the movement of the joints, such as step frequency, swing cycle, etc., the range of motion refers to the maximum angle or distance that the joint can reach during movement, the conventional motion standard refers to the benchmark that describes the range of motion and rhythm that the joint should follow in standard or typical movements, the specific movement refers to a movement with special or specific characteristics compared with the conventional motion standard, and the specific movement pattern refers to the unique movement sequence and characteristics exhibited in a specific movement.

[0032] Optionally, the analysis of the movement rhythm of the key joint parts based on the time series data can be achieved using a periodic detection method, the identification of the movement range of the key joint parts based on the movement rhythm can be determined by analyzing the joint angle changes or position changes in the time series data, and the analysis of the specific movement pattern of the key joint parts in the specific movement can be achieved using a deep learning model.

[0033] The embodiment of the present invention can identify and distinguish different movement patterns by extracting the joint movement characteristics of the monitored object according to the joint movement rules, thereby discovering abnormal or atypical movement patterns. The joint movement characteristics refer to a series of key attributes of joint movement that can be observed and measured, such as the speed change and acceleration of the joint during movement.

[0034] Optionally, the extraction of joint movement features of the monitored object according to the joint movement rules can be implemented using the PyTorch machine learning framework.

[0035] S3. Analyze the joint movement pattern of the monitored object according to the joint movement characteristics, identify the movement trajectory of the key joint parts under the joint movement pattern, and analyze the abnormal pattern of the joint movement based on the movement trajectory.

[0036] The embodiment of the present invention can deeply understand the individual's movement behavior in specific sports or daily activities by analyzing the joint movement pattern of the monitored object according to the joint movement characteristics, help identify exercise habits or risk factors that may lead to injuries, and thus take preventive measures. The joint movement pattern refers to the specific way or regularity of joint movement.

[0037] As an embodiment of the present invention, analyzing the joint movement pattern of the monitored object according to the joint movement characteristics includes: performing dimensionality reduction processing on the joint movement characteristics to obtain dimensionality reduction features; extracting feature keywords of the joint movement characteristics; identifying derived features of the monitored object based on the feature keywords; performing feature fusion processing on the dimensionality reduction features and the derived features to obtain fusion features; identifying the joint movement type of the monitored object according to the fusion features; and analyzing the joint movement pattern of the monitored object based on the joint movement type.

[0038] Among them, the dimensionality reduction feature refers to the feature obtained after processing the original joint movement feature through the dimensionality reduction technology, the feature keyword refers to the word that best represents the feature characteristics extracted from the joint movement feature, the derived feature refers to the new feature calculated or derived based on the original joint movement feature, the fusion feature refers to the feature set obtained by combining the dimensionality reduction feature and the derived feature, and the joint movement type refers to the different categories or types of joint movement identified according to the fusion feature.

[0039] Optionally, the dimensionality reduction processing of the joint movement characteristics can be obtained through dimensionality reduction technology, such as linear discriminant analysis, and the feature keyword extraction of the joint movement characteristics can be implemented using a keyword extraction algorithm, such as the TF-IDF algorithm. Based on the fusion characteristics, the joint movement type identification of the monitored object can be determined by a classification algorithm, such as a decision forest algorithm.

[0040] Furthermore, the embodiments of the present invention can help automatically identify and classify different movements by identifying the movement trajectory of the key joint parts in the joint movement mode, so as to better understand the behavior of the joints in a specific movement mode. The movement trajectory refers to the path of the joint parts as they move over a period of time.

[0041] Optionally, the identification of the movement trajectory of the key joint parts in the joint movement mode can be determined by an optical motion capture system, such as a Vicon system.

[0042] The embodiments of the present invention can help medical professionals identify potential movement disorders or diseases, and help evaluate the recovery process and treatment effects of joint patients by analyzing the abnormal pattern of joint movement based on the activity trajectory. It can also help to discover movement habits or risk factors that may lead to joint injuries, so as to take preventive measures. The abnormal pattern refers to a movement pattern that is significantly different from a normal or expected joint movement pattern.

[0043] As an embodiment of the present invention, the analyzing the abnormal pattern of the joint movement based on the activity trajectory includes: analyzing the normal activity range of the joint movement based on the activity trajectory; identifying the trajectory deviation of the activity trajectory according to the normal activity range; analyzing the triggering effect of the trajectory deviation on the joint movement; and analyzing the abnormal pattern of the joint movement based on the triggering effect.

[0044] Among them, the normal range of motion refers to the maximum range of motion that a joint can achieve under normal, pain-free and unrestricted conditions; the trajectory deviation refers to the difference between the actual joint movement trajectory and the expected or normal movement trajectory; the triggering effect refers to the direct impact of the trajectory deviation on joint movement, including possible pain, dysfunction, reduced movement efficiency or other negative consequences.

[0045] Optionally, the analysis of the normal range of motion of the joint activity based on the activity trajectory can be determined by joint type and individual differences, and the analysis of the triggering effect of the trajectory deviation on the joint activity can be achieved by analyzing the kinematic characteristics and dynamic characteristics of the joint activity.

[0046] S4. Extract the abnormal activity characteristics of the key joint parts in the abnormal mode, set an abnormal judgment mechanism for the joint activity based on the abnormal activity characteristics, and set a visual feedback interface for the joint activity according to the abnormal judgment mechanism.

[0047] The embodiment of the present invention can monitor the abnormal activity characteristics of the key joint parts by extracting the abnormal activity characteristics of the key joint parts under the abnormal mode, thereby detecting signs of joint pathology or injury early and reducing the risk of re-injury. The abnormal activity characteristics refer to characteristics in the joint activity that are significantly different from the normal or expected pattern, such as trembling of the joint during movement.

[0048] Optionally, the abnormal activity feature extraction of the key joint parts in the abnormal mode can be implemented using a recurrent neural network.

[0049] Furthermore, the embodiment of the present invention can realize automatic monitoring of joint movement, reduce manual intervention, and improve monitoring efficiency by setting up the abnormal judgment mechanism of joint movement based on the abnormal activity characteristics. The abnormal judgment mechanism refers to a method for identifying, evaluating and judging whether the joint movement deviates from the normal mode.

[0050] As an embodiment of the present invention, the abnormal judgment mechanism for the joint activity is set based on the abnormal activity characteristics, including: defining the abnormal action type of the joint activity based on the abnormal activity characteristics; identifying the action type corresponding to the joint activity; calculating the degree of fit between the action type and the abnormal action type; setting the abnormal judgment standard for the joint activity according to the degree of fit; identifying the abnormal cause of the joint activity based on the abnormal action type and the abnormal activity characteristics; setting the interpretation of the abnormal judgment result of the joint activity according to the abnormal cause and the abnormal action type; and setting the abnormal judgment mechanism for the joint activity in combination with the abnormal judgment standard and the interpretation of the abnormal judgment result.

[0051] Among them, the abnormal movement type refers to a movement category that is significantly different from the normal or expected movement pattern, the movement type refers to the different movement patterns exhibited by the joints in specific activities, such as walking, running, jumping, etc., the degree of fit refers to the similarity between the actual joint movement and the abnormal movement type, the abnormal judgment standard refers to the specific rules or thresholds used to judge whether the joint movement is abnormal, the abnormal cause refers to the specific factors that cause abnormal joint movement, such as muscle strain, arthritis or technical errors, and the abnormal judgment result interpretation refers to the explanation and description of the abnormal detection results, including the possible causes, impacts and recommended follow-up actions of the abnormality.

[0052] Optionally, the identification of the action type corresponding to the joint movement can be achieved by using video analysis, and according to the abnormal cause and the abnormal action type, the interpretation setting of the abnormal judgment result of the joint movement can be formulated by combining the abnormal activity characteristics, abnormal cause and medical knowledge.

[0053] In an optional embodiment of the present invention, the following formula is used to calculate the degree of fit between the action type and the abnormal action type: ; in, Indicates the degree of fit between the action type and the abnormal action type. A represents the vector representation of the action type, and B represents the vector representation of the abnormal action type. represents the value of vector A in the e-th dimension, represents the value of vector B in the e-th dimension, Represents the dimension index number corresponding to vector A and vector B, and m represents the number of dimensions corresponding to vector A and vector B.

[0054] The embodiment of the present invention can help users immediately understand abnormal situations of joint movements and take timely measures by setting up the visual feedback interface of the joint movement according to the abnormal judgment mechanism. The visual feedback interface refers to a user interface that provides real-time feedback information to users through visual elements.

[0055] As an embodiment of the present invention, setting the visual feedback interface of the joint activity according to the abnormal judgment mechanism includes: collecting real-time activity images of the joint activity; identifying abnormal activities of the joint activity according to the abnormal judgment mechanism, and setting abnormal prompts for the abnormal activities; extracting an image sequence of the abnormal activities in the real-time activity image; setting a screen reproduction mechanism for the abnormal activities based on the image sequence; setting a visual display type of the joint activity in combination with the abnormal prompt and the screen reproduction mechanism; setting the visual feedback interface of the joint activity according to the abnormal prompt, the screen reproduction mechanism and the visual display type.

[0056] Among them, the real-time activity image refers to the image data obtained by the visual sensor in the process of capturing joint movement in real time, the abnormal activity refers to the activity that is significantly different from the normal or expected joint movement pattern, the abnormal prompt refers to the warning or prompt provided by the system when the abnormal activity is identified, the image sequence refers to a series of image frames that capture the process of abnormal activity, the picture reproduction mechanism refers to the technology used to reproduce the abnormal activity image sequence, and the visual display type refers to the way of displaying joint movement and abnormal prompts, such as 2D charts.

[0057] Optionally, the real-time activity image acquisition of the joint movement can be achieved using a visual sensor, such as a camera, the abnormal prompt setting of the abnormal activity can be determined by an auditory notification method, such as a prompt sound, and the picture reproduction mechanism setting of the abnormal activity based on the image sequence can be achieved using video processing technology.

[0058] S5. Identify normal joint movement characteristics of the joint movement, and construct a dynamic monitoring and switching mechanism for the joint movement based on the normal joint movement characteristics.

[0059] By identifying the normal joint movement characteristics of the joint movement, the embodiments of the present invention can provide a theoretical basis and practical needs for the subsequent design of a dynamic monitoring and switching mechanism, ensure that the dynamic monitoring and switching mechanism can effectively identify key issues, and help the dynamic monitoring and switching mechanism maintain adaptability and generalization capabilities when facing different situations and individual differences. The normal joint movement characteristics refer to the typical and healthy characteristics exhibited by the joints when performing various activities in the absence of pain, injury or other pathological conditions.

[0060] As an embodiment of the present invention, the identifying of normal joint movement characteristics of the joint movement includes: collecting an overall joint movement data set of the joint movement; identifying a monitoring group corresponding to the overall joint movement data set; extracting group joint movement characteristics of the monitoring group; querying the standard action corresponding to the joint movement; based on the standard action, calculating the movement deviation degree of the joint movement; according to the movement deviation degree, setting a normal feature standard of the group joint movement characteristics; based on the normal feature standard, identifying the normal joint movement characteristics of the joint movement.

[0061] Among them, the overall joint movement data set refers to the collection of all joint movement data collected from the monitoring group, including various measurements and characteristics of the joint movement of the group in a specific action or a series of actions. The monitoring group refers to a specific group of people selected for joint movement monitoring. The group joint movement characteristics refer to the key features extracted from the overall joint movement data set of the monitoring group that can represent the characteristics of the joint movement of the group, such as average joint angle, range of motion, speed, etc. The standard action refers to the action mode that is considered to be correct or ideal in a specific field. The degree of movement deviation refers to the degree of difference between the joint movement characteristics of the monitoring group and the standard action. The normal feature standard refers to a series of reference standards or thresholds determined based on standard actions and group joint movement characteristics for evaluating whether the joint movement is normal.

[0062] Optionally, the group joint activity feature extraction of the monitored group can be obtained by performing statistical analysis and feature extraction on the overall joint activity data set.

[0063] In an optional embodiment of the present invention, based on the standard action, the movement deviation degree of the joint activity is calculated using the following formula: ; Among them, r represents the degree of deviation of joint movement, represents the feature vector of the i-th joint action in the joint activity, represents the feature vector of the standard action, n represents the total number of joint actions corresponding to the standard action, and i represents the joint action index number in the joint activity.

[0064] Furthermore, the embodiment of the present invention constructs a dynamic monitoring and switching mechanism for the joint movement according to the normal joint movement characteristics, thereby identifying the normal mode and abnormal mode of the joint movement and selecting the corresponding monitoring mode according to the identification result, which helps to improve the efficiency of monitoring and ensure that appropriate monitoring and intervention measures can be taken in time when an abnormality is detected. The dynamic monitoring and switching mechanism refers to a system that can dynamically adjust the monitoring strategy according to whether the monitored joint movement is abnormal.

[0065] As an embodiment of the present invention, the dynamic monitoring and switching mechanism of the joint movement is constructed according to the normal joint movement characteristics, including: collecting real-time activity data of the joint movement; extracting real-time joint movement characteristics of the real-time activity data; calculating the matching coefficient between the normal joint movement characteristics and the real-time joint movement characteristics; identifying the normal joint movement and the abnormal joint movement of the joint movement according to the normal joint movement characteristics; setting a dual-modal monitoring mechanism of the joint movement based on the normal joint movement and the abnormal joint movement; setting a mode switching trigger condition of the dual-modal monitoring mechanism according to the matching coefficient; and constructing the dynamic monitoring and switching mechanism of the joint movement in combination with the dual-modal monitoring mechanism and the mode switching trigger condition.

[0066] Among them, the real-time activity data refers to data about joint movement collected in real time from the monitoring equipment, the real-time joint movement characteristics refer to characteristics extracted from the real-time activity data that can describe the dynamic characteristics of joint movement, such as motion range, speed, rhythm, etc., the matching coefficient refers to a numerical value measuring the similarity between the real-time joint movement characteristics and the normal joint movement characteristics, the normal joint movement refers to the joint movement that conforms to the expected movement pattern of a healthy individual, the abnormal joint movement refers to the joint movement that exhibits abnormal or pathological characteristics, the dual-modal monitoring mechanism refers to a mechanism that can switch between two monitoring modes according to the monitoring results, one mode is used for monitoring normal joint movement, and the other mode is used for monitoring abnormal joint movement, and the mode switching trigger condition refers to setting the switching condition for the dual-modal monitoring mechanism to switch from one mode to another.

[0067] Optionally, the dual-modal monitoring mechanism setting of the joint activity based on the normal joint movement and the abnormal joint movement can be implemented using a support vector machine, and the mode switching trigger condition setting of the dual-modal monitoring mechanism according to the matching coefficient can be determined by the numerical value of the matching coefficient. If the matching coefficient value is less than 1, the abnormal monitoring mode in the dual-modal monitoring mechanism is triggered.

[0068] In an optional embodiment of the present invention, the matching coefficient between the normal joint movement feature and the real-time joint movement feature is calculated using the following formula: ; Among them, p represents the matching coefficient between normal joint movement characteristics and real-time joint movement characteristics, represents the g-th normal joint activity feature value, g represents the sequence number of the normal joint activity feature, represents the real-time joint activity feature value corresponding to the g-th normal joint activity feature, represents the maximum value of the g-th normal joint movement feature value and its corresponding real-time joint movement feature value, and r represents the total number of features of the normal joint movement feature.

[0069] S6. Combine the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint movement in real time to obtain real-time monitoring results.

[0070] The embodiment of the present invention combines the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint movement in real time and obtain real-time monitoring results. It can accurately capture slight changes in joint movement, improve the accuracy and reliability of monitoring, and present the real-time monitoring results of joint movement to the user in an intuitive and easy-to-understand form through the visual feedback interface, which helps the user to promptly discover and correct erroneous behaviors. At the same time, the dynamic monitoring switching mechanism can respond to changes in joint movement in real time according to the normal mode and abnormal mode of joint movement, thereby improving the monitoring efficiency and effect of joint movement. The real-time monitoring results refer to the data or information obtained after continuous and real-time monitoring of joint movement by combining the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism, such as the degree of bending, extension speed, movement trajectory, etc. of the joint.

[0071] It can be seen that the embodiment of the present invention can track and record joint movements in real time by setting up the tracking network of the joint movements, thereby improving the accuracy and reliability of the collected data, and also increasing the flexibility and portability of the monitoring system; further, the embodiment of the present invention can understand the movement mechanism of human joints by analyzing the joint movement rules of the key joint parts, extract the joint movement characteristics of the monitored object, identify and distinguish different movement patterns, and thus discover abnormal or atypical movement patterns; secondly, the embodiment of the present invention can help automatically identify and classify different movements by identifying the movement trajectory of the key joint parts under the joint movement mode, so as to better understand the behavior of the joints in a specific movement mode, help medical professionals identify potential movement disorders or diseases, and help evaluate the recovery process and treatment effect of joint patients, and also help discover movement habits or risk factors that may lead to joint injuries, so as to take preventive measures; thirdly, the embodiment of the present invention can realize by setting up the abnormal judgment mechanism and visual feedback interface of the joint movement based on the abnormal activity characteristics. The automatic monitoring of joint activities can help users to immediately understand the abnormal situation of joint activities and take timely measures to reduce the intervention of medical staff; the embodiment of the present invention can identify the normal mode and abnormal mode of joint activities by constructing the dynamic monitoring switching mechanism of joint activities according to the characteristics of normal joint activities, and select the corresponding monitoring mode according to the identification results, which is helpful to improve the efficiency of monitoring and ensure that appropriate monitoring and intervention measures can be taken in time when abnormalities are detected; finally, the embodiment of the present invention combines the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint activities in real time to obtain real-time monitoring results, which can accurately capture the slight changes in joint activities and improve the accuracy and reliability of monitoring, and present the real-time monitoring results of joint activities to users in an intuitive and easy-to-understand form through the visual feedback interface, which helps users to find and correct wrong behaviors in time, and at the same time, the dynamic monitoring switching mechanism can respond to the changes of joint activities in real time according to the normal mode and abnormal mode of joint activities, so as to improve the monitoring efficiency and effect of joint activities. Therefore, the mobile joint activity monitoring method proposed by the present invention can improve the comprehensiveness and accuracy of joint activity monitoring.

[0072] like Figure 2 Shown is a system functional module diagram of the mobile joint activity monitoring system of the present invention.

[0073] The mobile joint activity monitoring system 200 of the present invention can be installed in an electronic device. According to the functions to be implemented, the mobile joint activity monitoring system can include a visual tracking module 201, a feature extraction module 202, an abnormality identification module 203, a visual feedback module 204, a monitoring mode setting module 205 and a mobile monitoring module 206. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, which are stored in the memory of the electronic device.

[0074] In the embodiment of the present invention, the functions of each module / unit are as follows: The visual tracking module 201 is used to obtain the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect a real-time image of the joint activity based on the tracking network; A feature extraction module 202 is used to identify the key joint parts of the monitored object based on the real-time image, analyze the joint movement rules of the key joint parts, and extract the joint movement features of the monitored object according to the joint movement rules; An abnormality identification module 203 is used to analyze the joint activity pattern of the monitored object according to the joint activity characteristics, identify the activity trajectory of the key joint part under the joint activity pattern, and analyze the abnormal pattern of the joint activity based on the activity trajectory; A visual feedback module 204 is used to extract abnormal activity characteristics of the key joint parts in the abnormal mode, set an abnormal judgment mechanism for the joint activity based on the abnormal activity characteristics, and set a visual feedback interface for the joint activity according to the abnormal judgment mechanism; A monitoring mode setting module 205, for identifying normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring switching mechanism for the joint movement according to the normal joint movement characteristics; The movement monitoring module 206 is used to combine the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint movement in real time and obtain real-time monitoring results.

[0075] In detail, each module in the mobile joint activity monitoring system 200 of the embodiment of the present invention is used in the same manner as described above. Figure 1 The mobile joint activity monitoring method described in the invention has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0076] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

Claims

1. A mobile joint activity monitoring method, characterized in that: The method comprises: Acquire the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect a real-time image of the joint activity based on the tracking network; Based on the real-time image, identify the key joint parts of the monitored object, analyze the joint movement rules of the key joint parts, and extract the joint movement characteristics of the monitored object according to the joint movement rules; Analyzing the joint movement pattern of the monitored object according to the joint movement characteristics, identifying the movement trajectory of the key joint parts under the joint movement pattern, and analyzing the abnormal pattern of the joint movement based on the movement trajectory; Extracting abnormal activity features of the key joint parts in the abnormal mode, setting an abnormality judgment mechanism for the joint activity based on the abnormal activity features, and setting a visual feedback interface for the joint activity according to the abnormality judgment mechanism; Identifying normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring and switching mechanism for the joint movement according to the normal joint movement characteristics; In combination with the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism, the joint activity is monitored in real time to obtain real-time monitoring results.

2. The method according to claim 1, characterized in that The step of setting up the tracking network for the joint activity comprises: Identify the joint part corresponding to the joint movement, and configure the sensor device of the joint movement based on the joint part; According to the sensing device, a data acquisition unit for the joint activity is provided; Define the communication mode between the sensor device and the data acquisition unit.

3. The method according to claim 2, characterized in that The step of setting up the tracking network for the joint activity comprises: extracting the activity data of the joint movement based on the sensing device and the data acquisition unit, and configuring a data processor of the activity data; Identifying data processing results of the data processor and constructing a visualization interface of the data processing results; In combination with the sensor device, the data acquisition unit, the communication method, the data processor and the visualization interface, a tracking network for the joint activity is set up.

4. The method according to claim 3, characterized in that The collecting of the real-time image of the joint movement based on the tracking network includes: Based on the tracking network, collecting dynamic data of the joint movement; Extracting characteristic parameters of the joint movement according to the dynamic data; Based on the characteristic parameters, a 3D joint model corresponding to the joint movement is constructed.

5. The method according to claim 4, characterized in that The collecting of the real-time image of the joint movement based on the tracking network includes: According to the 3D joint model, simulating the skeletal animation of the joint movement; Identify the 3D joint model and the 2D image sequence of the skeletal animation; Based on the two-dimensional image sequence, real-time images of the joint movement are acquired.

6. The method according to claim 1, characterized in that The analyzing the joint movement rules of the key joint parts includes: Collecting monitoring data corresponding to the key joint parts, and extracting time series data of the monitoring data; Analyzing the movement rhythm of the key joint parts according to the time series data; Based on the movement rhythm, the range of motion of the key joint parts is identified.

7. The method according to claim 6, characterized in that The analyzing the joint movement rules of the key joint parts includes: Determine the conventional movement standard of the key joint parts by combining the movement rhythm and the movement range; According to the conventional movement standards, identifying specific movements of the key joints; Analyzing the specific movement pattern of the key joint parts in the specific movement; Combine the movement rhythm, the movement range and the specific movement pattern to analyze the joint movement patterns of the key joint parts.

8. The method according to claim 1, characterized in that Analyzing the joint movement pattern of the monitored object according to the joint movement characteristics includes: Performing dimensionality reduction processing on the joint activity features to obtain dimensionality reduction features; Extracting characteristic keywords of the joint movement characteristics; Based on the feature keywords, derived features of the monitored object are identified.

9. The method according to claim 8, characterized in that Analyzing the joint movement pattern of the monitored object according to the joint movement characteristics includes: Performing feature fusion processing on the dimension reduction feature and the derived feature to obtain a fused feature; identifying the joint movement type of the monitored object according to the fusion feature; Based on the joint movement type, the joint movement pattern of the monitored object is analyzed.

10. A mobile joint activity monitoring system, the system implementing the method according to claim 1, characterized in that: The system comprises: A visual tracking module, used to obtain the joint activity to be monitored and its corresponding monitoring object, set up a tracking network for the joint activity, and collect a real-time image of the joint activity based on the tracking network; A feature extraction module, for identifying key joints of the monitored object based on the real-time image, analyzing joint movement patterns of the key joints, and extracting joint movement features of the monitored object according to the joint movement patterns; an abnormality identification module, used to analyze the joint movement pattern of the monitored object according to the joint movement characteristics, identify the movement trajectory of the key joint part under the joint movement pattern, and analyze the abnormal pattern of the joint movement based on the movement trajectory; A visual feedback module, used for extracting abnormal activity characteristics of the key joint parts in the abnormal mode, setting an abnormal judgment mechanism for the joint activity based on the abnormal activity characteristics, and setting a visual feedback interface for the joint activity according to the abnormal judgment mechanism; A monitoring mode setting module, used for identifying normal joint movement characteristics of the joint movement, and constructing a dynamic monitoring switching mechanism for the joint movement according to the normal joint movement characteristics; The mobile monitoring module is used to combine the tracking network, the visual feedback interface and the dynamic monitoring switching mechanism to monitor the joint movement in real time and obtain real-time monitoring results.

Images