Adaptive digital rehabilitation system and method based on machine vision and finite state machine

Abstract

The application discloses an adaptive digital rehabilitation system and method based on machine vision and a finite state machine, and belongs to the technical field of medical digital rehabilitation and computer vision. The system acquires a video stream and extracts bone key points and confidence; unreliable frames are filtered based on confidence locking, and a dynamic weighted smoothing filtering algorithm of a historical time sequence is used for data denoising; a virtual bed surface model is constructed, a real clinical angle is calculated by using double zero-return calibration, and compensation cheating is intercepted through normal distance verification; a finite state machine triggered by an edge and a dynamic tolerance time window are combined to prevent motion misjudgment, and a forced rest lock is used to prevent fatigue; redundant operations are avoided by using a minimum extreme value capacity matrix to cut off invalid counting; and non-blocking feedback is executed through concurrent instruction execution decoupled from visual rendering. The application effectively improves the medical effectiveness and reliability of digital rehabilitation data.

Description

Technical Field

[0001] This invention relates to the field of medical digital rehabilitation and computer vision interdisciplinary technology, and in particular to an adaptive digital rehabilitation system and method based on machine vision and finite state machine. Background Technology

[0002] With the popularization of sports medicine and orthopedic surgery, the demand for postoperative rehabilitation training is growing exponentially. Traditional postoperative rehabilitation training relies heavily on physical therapists (PTs) providing in-person guidance and manual goniometer measurements in fixed locations (such as hospital rehabilitation departments or rehabilitation institutions). However, the time and space limitations of medical resources result in a large number of patients lacking scientific supervision and guidance during the home rehabilitation phase, making them highly susceptible to muscle compensation, joint stiffness, and even secondary physical injuries due to improper movements, overtraining, or poor compliance.

[0003] In recent years, fueled by the wave of artificial intelligence, computer vision (CV)-based assisted rehabilitation systems have gradually entered the market. These systems mostly acquire human body videos via cameras and use 2D or 3D pose estimation algorithms to analyze key skeletal points, aiming to replace the visual observation of physical therapists. However, when these existing technologies are applied to complex home scenarios, they generally reveal the following serious underlying technological shortcomings: Firstly, there are misjudgments caused by visual drift and miscalculation of neural network coordinates due to missing features. In real-world home environments, patients often face situations where furniture obstructs their view, clothing is loose, or limbs extend beyond the camera's field of view due to distance adjustments. Traditional pose estimation algorithms, in order to ensure the continuity of the output sequence, often force coordinate estimation in their underlying neural networks when features are missing, leading to distortion and generating a large amount of "dirty" data that deviates from the actual physical position of the human body. This directly results in serious errors in angle calculations in medical scales.

[0004] Secondly, there is a lack of clinical-grade fault tolerance mechanisms and anti-compensation logic. Most existing systems use the simplest frame-level absolute threshold judgment logic (for example, a knee bend greater than 90 degrees is immediately considered a completed repetition). In actual implementation, on the one hand, inherent noise of the sensors or changes in lighting can cause high-frequency flashing at key points, leading the system to trigger false counts excessively; on the other hand, the system cannot recognize the "compensatory movements" unconsciously made by patients when fatigued (such as using trunk leaning back and pelvic compensation instead of thigh force when performing straight leg raises, or using a half-squat to cheat and increase the number of repetitions when squatting against a wall), which greatly reduces the medical effectiveness of the system.

[0005] Finally, there are the risks of concurrent blocking and hardware deadlock. Traditional visual interaction systems, when processing real-time video streams (typically requiring 30 frames per second or higher), are prone to resource contention in Python environments (due to the Global Interpreter Lock, GIL) or on low-power edge devices if they simply employ traditional multithreading methods to add voice guidance and front-end UI re-rendering. This can lead to severe video stuttering and audio-visual desynchronization. More seriously, when the system exits abnormally due to patient error, the driver handles of the underlying hardware (such as camera device nodes) often fail to be released correctly, resulting in hardware deadlock and even privacy breaches.

[0006] In summary, how to provide a computer vision rehabilitation system that can overcome environmental occlusion interference, has high fault tolerance and strictly prevents muscle compensation behavior, and can completely solve the system lag caused by multi-threaded concurrency and high-frequency invalid redundant motion counting is a fundamental problem that urgently needs to be solved in the current field of digital medical rehabilitation. Summary of the Invention

[0007] The purpose of this invention is to provide an adaptive digital rehabilitation system and method based on machine vision and finite state machines, aiming to solve the technical problems of existing visual rehabilitation systems in complex scenarios, such as data distortion caused by the misestimation and distortion of neural network coordinates due to the lack of features, ineffective rehabilitation actions due to the lack of high fault-tolerant state machines and anti-compensation control, and system blockage and lag caused by the concurrency of video streams and audio feedback.

[0008] In a first aspect, the present invention provides an adaptive digital rehabilitation system based on machine vision and finite state machines, comprising the following modules: The video acquisition and skeleton extraction module is used to acquire the target video stream of the patient during the rehabilitation training process, extract the initial set of skeletal key points of continuous frames from the target video stream through the computer vision skeleton extraction network, and obtain the model confidence corresponding to each key point in the initial set of skeletal key points. The confidence feature cleaning module is used to clean the initial set of skeletal key points based on the confidence lock, and generate cleaned skeletal coordinate data. Specifically, when the model confidence of a key point corresponding to a core force-bearing joint is lower than a preset confidence threshold, the confidence lock is triggered to discard the current frame data and freeze the state machine timing logic. The module also performs dynamic weighted smoothing filtering based on historical time series on the current frame data that has not been discarded, and outputs the cleaned skeletal coordinate data. The spatial coordinate remapping module is used to remap the spatial coordinates of the cleaned bone coordinate data using a dual zeroing calibration algorithm and a virtual bed surface model, generating individualized real clinical angle data. The motion assessment and state machine control module is used to input the individualized real clinical perspective data into a preset edge-triggered finite state machine, and output the motion assessment state results based on a dynamic tolerance time window and a forced rest lock mechanism; the edge-triggered finite state machine is configured with rehabilitation motion state machine nodes at different levels; The compliance tracking and concurrent scheduling module is used to establish a capacity matrix based on a single fact source mechanism and extract front-end user interface instructions. It combines the action evaluation status results to perform closed-loop tracking of medical compliance and outputs the final rehabilitation training report. At the same time, when the state of the edge-triggered finite state machine flips, it executes asynchronous voice broadcasting without being blocked by the main visual rendering by calling the underlying system silent process of the operating system.

[0009] As an optional implementation of the first aspect of this application, in the confidence feature cleaning module, feature cleaning is performed on the initial set of skeletal key points based on a confidence lock to generate cleaned skeletal coordinate data. Specifically, this includes: determining whether the model confidence corresponding to each key point in the initial set of skeletal key points is lower than a preset confidence threshold; if the model confidence of a key point corresponding to a core force-bearing joint is lower than the preset confidence threshold, it is determined that limb occlusion or screen overload has occurred, and a global confidence lock signal is immediately generated, discarding all skeletal data contained in the current frame, and moving towards the edge-triggered... The finite state machine sends a clock freeze command; if the model confidence of the key point corresponding to the core force joint is higher than or equal to the preset confidence threshold, the smoothed coordinate data of the current frame is calculated based on the initial skeletal key point coordinate data of the current frame and the historical smoothed coordinate data of the previous frame, eliminating high-frequency noise from the sensor and generating the cleaned skeletal coordinate data; the smoothed coordinate data of the current frame is obtained by multiplying the initial coordinate value of the current frame by a smoothing weight coefficient, and then adding the product obtained by multiplying the historical smoothed coordinate value of the previous frame by a coefficient complementary to the smoothing weight coefficient.

[0010] As an optional implementation of the first aspect of this application, the spatial coordinate remapping module specifically includes: acquiring the initial absolute joint angles when the patient is in a static reference position during the initial training phase, calculating the difference between the initial absolute joint angles and the standard absolute zero position, and generating an individualized offset angle, wherein the standard absolute zero position refers to the ideal angle of the human body in a fully supine and straight state. In subsequent real-time training, the real-time dynamic joint angles calculated based on the cleaned skeletal coordinate data are superimposed with the individualized offset angles to generate the individualized real clinical angle data; the pixel coordinates of the hip joint and heel on the same side are extracted from the cleaned skeletal coordinate data, and a virtual bed surface straight line equation is constructed with the hip joint pixel coordinates and heel pixel coordinates as endpoints; the pixel coordinates of the knee joint are extracted from the cleaned skeletal coordinate data, and the normal distance from the knee joint pixel coordinates to the straight line corresponding to the virtual bed surface straight line equation is calculated. The normal distance is output as the anti-compensation normal distance data, which is used to trigger an out-of-bounds alarm signal when the normal distance is greater than a preset distance threshold.

[0011] As an optional implementation of the first aspect of this application, in the action assessment and state machine control module, the individualized real clinical angle data is input into a preset edge-triggered finite state machine, and the action assessment state result is output based on a dynamic tolerance time window and a forced rest lock mechanism. Specifically, this includes: comparing the individualized real clinical angle data with the passing range required by the clinical prescription; when the individualized real clinical angle data is within the passing range, controlling the edge-triggered finite state machine to flip to the target holding state and starting an effective timer for cumulative timing; when the target holding state is in place and the individualized real clinical angle data is within the passing range, controlling the edge-triggered finite state machine to flip to the target holding state and starting an effective timer for cumulative timing; when the target holding state is in place and the individualized real clinical angle data is within the passing range, controlling the edge-triggered finite state machine to flip to the target holding state and starting an effective timer for cumulative timing; when the target holding state is in place and the individualized real clinical angle data is within the target holding state, the individualized real clinical angle data is calculated to be ... When the data slips out of the passing range, the edge-triggered finite state machine suspends the effective timer to pause accumulation but keeps the value from being zero, and simultaneously starts a preset dynamic tolerance time window for countdown. If the individualized real clinical perspective data returns to the passing range before the countdown of the dynamic tolerance time window ends, the dynamic tolerance time window is closed and the accumulation of the effective timer is resumed. If the countdown of the dynamic tolerance time window ends and the individualized real clinical perspective data still has not returned to the passing range, the effective timer is reset and the action evaluation status result of the single action failure is output.

[0012] As an optional implementation of the first aspect of this application, in the action assessment and state machine control module, the individualized real clinical perspective data is input into a preset edge-triggered finite state machine, and the action assessment state result is output based on a dynamic tolerance time window and a forced rest lock mechanism. Specifically, it also includes: real-time monitoring of the accumulated time value of the effective timer or the number of action completions; when the accumulated time value or the number of action completions of a single action reaches a preset prescription target value, controlling the edge-triggered finite state machine to forcibly switch to a forced rest lock state with a preset hardware-level lock time length; during the period of the forced rest lock state, intercepting and refusing to respond to any repetitive action request signals to the underlying data engine until the hardware-level lock time length ends.

[0013] As an optional implementation of the first aspect of this application, in the compliance tracking and concurrent scheduling module, when the state of the edge-triggered finite state machine flips, asynchronous voice broadcasting is executed without being blocked by the main visual rendering by calling the underlying system silent process of the operating system. Specifically, this includes: monitoring the state flip edge signal of the edge-triggered finite state machine; when the state flip edge signal is captured, bypassing the foreground visual rendering thread of the main program, directly sending a scheduling instruction to the kernel of the operating system that is decoupled from the main visual rendering thread and runs concurrently; receiving the scheduling instruction to call the underlying speech synthesis and playback interface of the operating system to execute the voice broadcasting, and simultaneously starting an audio concurrent mutex based on a global cooldown time window; during the preset cooldown period of the audio concurrent mutex's activation cycle, blocking all subsequently triggered voice broadcasting requests to avoid multiple speech overlaps and prevent concurrent blocking of the main video stream.

[0014] As an optional implementation of the first aspect of this application, in the compliance tracking and concurrent scheduling module, a capacity matrix based on a single fact source mechanism is established and front-end user interface instructions are extracted. Medical compliance closed-loop tracking is performed in conjunction with the action assessment status results. Specifically, this includes: parsing clinical prescription parameters to construct a three-dimensional clinical capacity matrix containing the number of target groups, the number of times per group, and the duration of retention; running a minimum extreme value truncation algorithm in the underlying data engine; obtaining the physical actual completed capacity data corresponding to the action assessment status results; comparing the physical actual completed capacity data with the prescription target capacity data configured in the three-dimensional clinical capacity matrix; extracting the minimum extreme value between the physical actual completed capacity data and the prescription target capacity data, and writing the minimum extreme value as the current compliant physical capacity data into the final database; when the physical actual completed capacity data is greater than the prescription target capacity data, generating a physical state lock to freeze the input requests of the front-end user interface and truncate invalid repetitive actions that exceed expectations, and generating the final rehabilitation training report.

[0015] As an optional implementation of the first aspect of this application, it also includes a port resource conflict resolution module, used to perform a pre-port probing and a mechanism for forcibly terminating residual processes and releasing hardware ports based on the highest system privileges. Specifically, it includes: using a pre-port sniffing mechanism to scan the occupancy status of system physical ports required for the target service to run; when it is detected that the system physical port is occupied by a historical residual old process, extracting the process identifier corresponding to the old process; sending a forced termination signal with the highest privileges to the operating system to destroy the process identifier corresponding to the old process, and automatically releasing the occupancy rights of the system physical port, so as to ensure the uniqueness and security of the current rehabilitation service instance in device hardware scheduling.

[0016] Secondly, embodiments of this application provide an adaptive digital rehabilitation method based on machine vision and finite state machines, comprising the following steps: The target video stream of the patient during rehabilitation training is acquired, and an initial set of skeletal key points for consecutive frames is extracted from the target video stream through a computer vision skeleton extraction network. The model confidence corresponding to each key point in the initial set of skeletal key points is then obtained. The initial set of skeletal key points is cleaned based on a confidence lock to generate cleaned skeletal coordinate data. Specifically, when the model confidence of a key point corresponding to a core stress joint is lower than a preset confidence threshold, the confidence lock is triggered to discard the current frame data and freeze the state machine timing logic. The remaining current frame data is then subjected to dynamic weighted smoothing filtering based on historical time series, outputting the cleaned skeletal coordinate data. A dual zero-calibration algorithm and a virtual bed surface model are used to remap the cleaned skeletal coordinate data to generate personalized, real-world clinical angle data. The individualized real-world clinical data is input into a preset edge-triggered finite state machine, and the action assessment state results are output based on a dynamic tolerance time window and a forced rest lock mechanism; the edge-triggered finite state machine is configured with rehabilitation action state machine nodes at different levels; Establish a capacity matrix based on a single fact source mechanism and extract front-end user interface instructions. Combine the action assessment status results to perform closed-loop tracking of medical compliance and output the final rehabilitation training report. When the state of the edge-triggered finite state machine flips, asynchronous voice broadcasting is executed by calling the underlying silent process of the operating system, without being blocked by the main visual rendering.

[0017] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0018] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects: 1. Avoiding coordinate misestimation and distortion caused by missing features in neural networks, achieving medical-grade high-precision measurement: This invention introduces a global confidence lock and clock freeze mechanism. When the joint confidence is too low, the current frame is immediately discarded and the timing is frozen. Combined with dynamic weighted smoothing filtering based on historical time series, it effectively cuts off the coordinate misestimation and distortion caused by missing features in the forced prediction of neural networks in complex environments, filters dirty data and eliminates high-frequency noise, thereby achieving extremely high measurement accuracy and data reliability.

[0020] 2. Constructing Two-Dimensional Spatial Constraints for In-Depth Anti-Compensation and Injury Prevention: This invention eliminates multi-dimensional baseline errors through a dual zero-calibration algorithm and extracts joint coordinates to construct a virtual bed surface straight line equation for real-time calculation of the normal distance. This feature can sensitively quantify and intercept minute "compensatory movements" that are difficult to detect with the naked eye, such as pelvic tilt. An alarm is triggered when these movements exceed the limits, effectively preventing secondary injuries caused by incorrect force application and ensuring the clinical effectiveness of rehabilitation.

[0021] 3. Dynamic time logic improves fault tolerance, and forced rest lock avoids overwork: This invention introduces a dynamic tolerance time window mechanism into the state machine. When muscle tremors cause the angle to briefly exceed the limit, the timer is only suspended and not cleared. This solves the patient frustration caused by frequent resets in traditional systems and greatly improves the fault tolerance rate. In conjunction with the forced rest lock mechanism, repeated actions are intercepted after a single set of targets are met. This physically blocks dangerous high-frequency ineffective redundant actions and ensures that patients safely pass through the lactate metabolism period.

[0022] 4. Bypassing thread blocking and overcoming concurrency performance bottlenecks in lightweight frameworks: This invention invokes voice playback by directly sending independent concurrent scheduling instructions, decoupled from the main visual rendering thread, to the system kernel, and supplements this with an audio concurrency mutex based on a global cooldown time window, completely decoupling high-density voice feedback from the main program's video inference. This feature not only shields against multiple voice overlaps caused by concurrency but also reduces the blocking of video frame rate by voice to zero.

[0023] 5. A robust overflow prevention mechanism provides tamper-proof, medical-grade data tracking: This invention runs a minimum extremum truncation algorithm at the underlying level, extracting the minimum extremum between the actual and target capacity as the legally written data, and combining this with a physical state lock on the front-end interface to freeze excessive input. This feature completely patches the "progress bar overflow" and "state redundancy re-entry" vulnerabilities caused by sensor mis-triggers or malicious clicks from the underlying layer. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the structure of an adaptive digital rehabilitation system based on machine vision and finite state machine provided in an embodiment of the present invention; Figure 2A flowchart illustrating an adaptive digital rehabilitation method based on machine vision and finite state machine, provided as an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the anti-compensation spatial coordinate remapping and calculation principle based on virtual bed surface construction and dual zeroing calibration, provided in an embodiment of the present invention. Figure 4 The workflow diagram of an edge-triggered finite state machine (FSM) with clock freeze, grace period and rest lock provided in the embodiments of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] Example 1 Please see Figure 1 The diagram shown is a schematic representation of an adaptive digital rehabilitation system based on machine vision and finite state machine according to the first embodiment of this application. The system includes the following key modules: The video acquisition and skeleton extraction module 100 is used to acquire the target video stream of the patient during the rehabilitation training process, extract the initial set of skeletal key points of continuous frames from the target video stream through the computer vision skeleton extraction network, and obtain the model confidence corresponding to each key point in the initial set of skeletal key points. The confidence feature cleaning module 200 is used to clean the initial set of skeletal key points based on a confidence lock, and generate cleaned skeletal coordinate data. Specifically, when the model confidence of a key point corresponding to a core force-bearing joint is lower than a preset confidence threshold, the confidence lock is triggered to discard the current frame data and freeze the state machine timing logic. The module also performs dynamic weighted smoothing filtering based on historical time series on the current frame data that has not been discarded, and outputs the cleaned skeletal coordinate data. The spatial coordinate remapping module 300 is used to remap the spatial coordinates of the cleaned bone coordinate data using a dual zeroing calibration algorithm and a virtual bed surface construction model, thereby generating individualized real clinical angle data. The motion assessment and state machine control module 400 is used to input the individualized real clinical perspective data into a preset edge-triggered finite state machine, and output the motion assessment state results based on a dynamic tolerance time window and a forced rest lock mechanism; the edge-triggered finite state machine is configured with rehabilitation motion state machine nodes at different levels; The compliance tracking and concurrent scheduling module 500 is used to establish a capacity matrix based on a single fact source mechanism and extract front-end user interface instructions. It combines the action evaluation status results to perform closed-loop tracking of medical compliance and outputs the final rehabilitation training report. At the same time, when the state of the edge-triggered finite state machine flips, it executes asynchronous voice broadcasting without being blocked by the main visual rendering by calling the underlying system silent process of the operating system.

[0028] Example 2 This embodiment takes the classic lower limb rehabilitation exercises "wall squat" and "straight leg raise" as examples to illustrate in detail the specific computation steps and data execution flow logic of this adaptive digital rehabilitation system based on machine vision and finite state machine.

[0029] Please see Figure 2 This is a flowchart illustrating an adaptive digital rehabilitation method based on machine vision and finite state machines, provided by an embodiment of the present invention. The method may include the following steps: Step S1: Pre-acquisition of video stream and feature cleaning based on global confidence lock.

[0030] After initialization, the system captures a video stream of the patient undergoing rehabilitation training using a physical device on the user terminal (such as an RGB camera or webcam). The captured sequence of continuous video frames is then input into a pre-built computer vision skeleton extraction network at a specific resolution and color space. For the set of N 3D skeletal keypoints output by the network, the coordinate data of each keypoint in space (or pixel plane) is extracted. Simultaneously, the model prediction confidence level assigned to this key point by the network layer is obtained. This typically represents the probability that the keypoint is not occluded and can be accurately observed within the current frame.

[0031] Before delving into specific angle calculations, this system innovatively introduces a "global confidence lock" mechanism at the beginning of the frame processing pipeline. The system extracts the set of core force-bearing joints involved in the current rehabilitation movement (e.g., the hip, knee, and ankle joints that must be monitored during straight leg raises). For each node in this set, its... Perform logical verification: determine whether any core node exists. (in, To pre-set the reliability threshold and balance strictness and usability, this embodiment sets it to 0.5.

[0032] If the verification result is true (i.e., key points are not visible or have low confidence, which usually means the patient's limbs are obscured by the sofa or the movement is too large and they have moved out of the frame), the system immediately discards the neural network coordinate misestimation and distorted offset data caused by feature loss, and triggers the confidence lock mechanism. This mechanism issues two instructions: first, completely discard all dirty coordinate data generated in the current frame; second, send a "clock freeze" instruction to the backend finite state machine to suspend the accumulation of all valid time.

[0033] If the verification result is false (i.e., all core key points are observed well), in order to further eliminate high-frequency flicker noise from the sensor caused by low-pixel home cameras or light flicker, the system performs an exponential moving average (EMA) spatial filtering algorithm on the set of key point coordinates that were not discarded. Taking the coordinate dimension as an example, the smoothed coordinate values ​​of the current frame's t-th frame... The calculation method is as follows: ;in, These are the original coordinate observations extracted from the current frame. For the previous frame t Historical smoothed coordinate values ​​retained at time 1 The weighting coefficients for determining the sensitivity of smooth tracking are used. The filtered and cleaned skeletal coordinate data will be output to the next module for spatial coordinate remapping.

[0034] Step S2: A spatial anti-compensation mechanism based on dual zeroing calibration and virtual bed surface construction.

[0035] Due to the significant differences in body size among patients, and the inability to standardize the placement height and tilt angle of the camera, the system introduces a static zeroing and dynamic anti-compensation calculation model.

[0036] Before the patient begins formal training, the system guides the patient into a specific "static reference position" for the movement via voice (for example, before performing straight leg raises, the patient needs to lie flat on a mat and keep the affected leg fully extended and relaxed).

[0037] First, the baseline calibration calculation is performed: the system uses the coordinate data cleaned in step S1 to calculate the initial absolute physical angle of the patient's knee joint at the current moment. (That is, the angle between the thigh vector and the calf vector in space). The system extracts the standard absolute zero point as defined in medical guidelines (usually considered the ideal angle when the human body is fully supine and straight). The difference between the two values ​​yields an individualized offset angle that reflects the patient's skeletal characteristics and the current camera tilt angle: .

[0038] In subsequent dynamic real-time training calculations, all the original dynamic knee joint angles calculated in real time from the images are... All of these must be superimposed with this individualized bias angle to generate individualized real clinical angle data for comparing prescriptions: This operation achieves baseline uniformity across body types and viewpoints.

[0039] Next, spatial anti-compensation modeling is performed: such as... Figure 3 As shown, many patients, due to insufficient quadriceps strength, subconsciously employ compensatory movements such as raising their hips and arching their backs to gain leverage when performing leg raises. In the baseline position, the system extracts the horizontal and vertical coordinates of the hip joint pixels on the same side of the patient from the two-dimensional pixel image. and the lateral and longitudinal coordinates of the heel. Using the principle of plane analytic geometry that two points determine a line, the equation of a straight line with a "virtual bed surface" between the two points is constructed: The coefficients A, B, and C are derived entirely from the reference points obtained from the static data collection mentioned above (e.g., ...). wait).

[0040] During the patient's subsequent training phase, the system extracts the node pixel coordinates corresponding to the groin and the area below the knee joint in real time. The normal distance d from the node to the aforementioned virtual bed surface is calculated in real time. The mathematical logic is to substitute the point's coordinates into the line equation to obtain the absolute value, then divide by the square root of the sum of the squares of the horizontal and vertical coordinate coefficients. This normal distance is... The calculation formula is as follows: If the system calculates that the real-time d value is greater than the preset distance threshold If the action is detected as cheating, such as "lifting the thigh off the bed or excessively rotating and lifting the pelvis", an out-of-bounds alarm will be triggered immediately, and the action will be determined to be non-compliant in the state machine.

[0041] Step S3: Perform finite state machine operations with dynamic tolerance time window and edge triggering logic.

[0042] like Figure 4 As shown, the system breaks down complex rehabilitation movements into multi-level states with strict sequential transition logic (e.g., initial preparation -> movement phase -> static holding phase -> retreat and reset phase -> forced rest phase). State transitions are controlled by "edge-triggered" logic, meaning that state reversal is triggered only the instant the data crosses a threshold boundary.

[0043] The system will generate personalized real-world clinical data from step S2. The passing grade range tailored for this movement in the rehabilitation prescription. Perform high-frequency comparisons. For example, for "wall squat", the passing range may be set as [90°, 150°].

[0044] when Upon first entering this interval, the finite state machine (FSM) is edge-triggered, the state flips to the target hold state (HOLD), and the effective timer in memory is activated to start accumulating seconds.

[0045] Traditional systems face a major challenge here: when patients are close to exhaustion, muscle tremors are inevitable, causing the joint angle to jump frequently at the critical point (e.g., 106°). As a result, traditional systems will constantly reset the timer, causing great frustration to patients.

[0046] To address this issue, this system introduces a "dynamic tolerance time window" on the exit path of the HOLD state. When in the target-holding state, if a frame measures... If the clock slips out of the acceptable range, the FSM will not immediately determine failure and reset. Instead, the FSM will only "suspend" the valid timer (pause the injection of clock frequency but never clear the accumulated value) and immediately start a countdown window with a preset duration, for example, T. grace =3s.

[0047] In the next T grace If the patient readjusts their force application pattern and brings their limb angle back to the acceptable range within seconds, the system immediately closes the tolerance window countdown and seamlessly resumes the accumulation of the valid timer; however, if the patient still has not returned to the correct posture by the end of the countdown, the system will execute the penalty logic, reset the valid timer, determine the current action phase as a failure, and issue an intervention prompt.

[0048] In addition, to prevent secondary injury, once the effective timer reaches its target and a single action or set of actions is completed, the FSM will perform a hard cutoff, forcibly jumping into a hardware-level rest lock state (REST) ​​for a preset time (e.g., 30 seconds). This lock is a highest priority blocking lock. During this period, the system's underlying layer refuses to respond to any trigger requests generated by the patient's voluntary early execution of actions, ensuring lactate metabolism and forcibly preventing high-frequency invalid and redundant actions caused by dangerous continuous overexertion.

[0049] Step S4: Capacity matrix deduction based on a single source of fact and physical locking of interface state.

[0050] Digital healthcare for rehabilitation demands extremely stringent requirements for data authenticity and adherence tracking. This system completely abandons the loose structure of traditional apps that rely solely on incremental variables from front-end UI components, and instead constructs a Single Source of Truth (SSOT) data processing engine.

[0051] The system retrieves prescriptions from the cloud and parses them in local memory to construct a three-dimensional clinical capacity matrix, which includes three orthogonal dimensions: the number of target groups, the number of times per group, and the duration of each treatment.

[0052] Upon receiving the action evaluation status results and actual completion capacity data, such as the number of completions, from the aforementioned FSM transmission, the engine's core processor executes the "minimum truncation algorithm": the system sets the actual completion capacity data C... actual The current dimension of the prescription target capacity data C is specified in the capacity matrix. target At the same time, the extreme value comparison function is passed in.

[0053] Through calculation formula Extract the minimum value between the two. This minimum value represents the final, legal, and compliant physical capacity data. This data is then archived and written to the database record table. This algorithm, from a mathematical foundation, prevents redundant re-entry errors caused by repeated false triggering of sensors, such as "the patient requests to do it 10 times, the database records 15 times, causing the progress bar UI rendering component to crash and exceed 100%".

[0054] Furthermore, when C actual Greater than or equal to C target In an instant, the underlying data engine uses a callback mechanism to send a forced command with a physical state lock to the presentation layer (front-end UI), freezing and graying out all submit buttons and step count progress bars. This completely eliminates the possibility of high-frequency invalid and redundant action counting caused by network latency or malicious high-frequency clicks by patients, ultimately generating a rehabilitation training report with medical-grade rigor and no possibility of tampering.

[0055] Step S5: Cross-platform asynchronous concurrent audio scheduling and a mechanism for forcibly terminating residual processes based on the highest system privileges.

[0056] Visual inference engines typically require significant CPU / GPU computing power. When running this system (especially in UI environments built using lightweight frameworks with a Global Interpreter Lock (GIL) like Python's Streamlit), if each action state transition requires a voice prompt such as "The curvature is good, please maintain it," traditional solutions use a foreground visual rendering thread, i.e., the same process for text-to-speech parsing and sound card calls. This forces the main program to suspend for 1-2 seconds to wait for the audio to finish playing, resulting in catastrophic stuttering of the video, like a frozen slideshow.

[0057] To address this bottleneck, this invention implements a system-level asynchronous concurrency strategy that bypasses the main program. The system employs an independent listener specifically designed to detect "state toggle edge signals" output by the FSM. Once a toggle event is detected (e.g., the instant from MOVING to HOLD), the listener no longer requests the foreground visual rendering thread at the application layer. Instead, it directly sends a scheduling instruction to the operating system kernel, decoupled from the main visual rendering thread and operating independently. Leveraging operating system features (e.g., the '&' background operator in Unix / Mac systems, or silent execution by stripping the standard input / output pipes using subprocess.Popen in Windows), the underlying speech synthesis (broadcast instructions and speech text) is directly fed to the TTS playback interface at the operating system kernel layer for speech playback. The main program returns to the video stream reading loop immediately after sending the instruction, consuming less than 1 millisecond, thus achieving complete decoupling from the main visual rendering thread.

[0058] To complement this asynchronous approach, the system also incorporates an independent time-cooler called an audio concurrent mutex based on a global cooldown time window. The moment an asynchronous call is emitted, this audio concurrent mutex based on the global cooldown time window is activated and begins a preset cooldown period (e.g., 3 seconds, roughly the same as the duration of the audio broadcast). During this period, even if the patient's unstable movements cause frequent state machine jumps, triggering numerous audio call requests, all requests will be relentlessly discarded by the active cooldown mutex, ensuring that regardless of the unexpected event, the patient will not hear overlapping garbled noise, and no stack overflow will occur in the foreground.

[0059] Furthermore, because the system frequently calls underlying hardware and starts local web port services (such as port 8501), hardware deadlocks can easily occur when the program shuts down abnormally. Therefore, this embodiment also configures a daemon process sequence containing a pre-detection mechanism. Before the system initializes and starts the main service, it uses a network socket sniffing mechanism to scan the status of target physical ports. When it finds that the port is being held by a remnant old zombie process, the system extracts the PID (Process Identifier) ​​of the old process, requests the highest privilege from the operating system, and forcibly sends a termination signal, forcibly destroying the zombie process along with its hardware handles from the physical memory level. This completely releases the hardware and ensures the unique security and data isolation of each digital therapy service, preventing data leakage.

[0060] The adaptive digital rehabilitation system and method described in this invention do not rely on high-cost wearable devices or large-scale specialized testing instruments. They can be widely and safely applied to remote digital physiotherapy scenarios such as postoperative home rehabilitation tracking in orthopedic / sports medicine centers, prevention and management of cruciate ligament injuries in sports medicine, gait retraining for hemiplegic patients in neurology, and fall prevention and monitoring for the elderly in communities.

[0061] An adaptive digital rehabilitation system based on machine vision and finite state machines, as described in this application, can be a device, a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application does not impose specific limitations on these devices.

[0062] The adaptive digital rehabilitation system based on machine vision and finite state machine in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0063] The adaptive digital rehabilitation system based on machine vision and finite state machine provided in this application embodiment can achieve... Figure 2The various processes implemented in the method embodiment of an adaptive digital rehabilitation method based on machine vision and finite state machine will not be described again here to avoid repetition.

[0064] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of an adaptive digital rehabilitation method based on machine vision and finite state machine, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0065] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of an adaptive digital rehabilitation method based on machine vision and finite state machine, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0066] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0067] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0069] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the scope of protection of this application.

Claims

1. An adaptive digital rehabilitation system based on machine vision and finite state machines, characterized in that, include: The video acquisition and skeleton extraction module is used to acquire the target video stream of the patient during the rehabilitation training process, extract the initial set of skeletal key points of continuous frames from the target video stream through the computer vision skeleton extraction network, and obtain the model confidence corresponding to each key point in the initial set of skeletal key points. The confidence feature cleaning module is used to clean the initial set of skeletal key points based on the confidence lock, and generate cleaned skeletal coordinate data. Specifically, when the model confidence of a key point corresponding to a core force-bearing joint is lower than a preset confidence threshold, the confidence lock is triggered to discard the current frame data and freeze the state machine timing logic. The module also performs dynamic weighted smoothing filtering based on historical time series on the current frame data that has not been discarded, and outputs the cleaned skeletal coordinate data. The spatial coordinate remapping module is used to remap the spatial coordinates of the cleaned bone coordinate data using a dual zeroing calibration algorithm and a virtual bed surface model, generating individualized real clinical angle data. The motion assessment and state machine control module is used to input the individualized real clinical perspective data into a preset edge-triggered finite state machine, and output the motion assessment state results based on a dynamic tolerance time window and a forced rest lock mechanism; the edge-triggered finite state machine is configured with rehabilitation motion state machine nodes at different levels; The compliance tracking and concurrent scheduling module is used to establish a capacity matrix based on a single fact source mechanism and extract front-end user interface instructions. It combines the action evaluation status results to perform closed-loop tracking of medical compliance and outputs the final rehabilitation training report. At the same time, when the state of the edge-triggered finite state machine flips, it executes asynchronous voice broadcasting without being blocked by the main visual rendering by calling the underlying system silent process of the operating system.

2. The system according to claim 1, characterized in that, In the confidence feature cleaning module, feature cleaning is performed on the initial set of skeletal keypoints based on confidence locking to generate cleaned skeletal coordinate data, specifically including: Determine whether the model confidence level corresponding to each key point in the initial skeletal key point set is lower than the preset confidence threshold. If the model confidence of a key point corresponding to a core force-bearing joint is lower than the preset confidence threshold, it is determined that limb occlusion or screen over-limit has occurred. A global confidence lock signal is immediately generated, all bone data contained in the current frame is discarded, and a clock freeze command is sent to the edge-triggered finite state machine. If the model confidence of the key point corresponding to the core force-bearing joint is higher than or equal to the preset confidence threshold, then the smoothed coordinate data of the current frame is calculated based on the initial bone key point coordinate data of the current frame and the historical smoothed coordinate data of the previous frame, eliminating high-frequency noise from the sensor and generating the cleaned bone coordinate data. The smoothed coordinate data of the current frame is obtained by multiplying the initial coordinate value of the current frame by a smoothing weight coefficient, and then adding the product of the historical smoothed coordinate value of the previous frame by a coefficient complementary to the smoothing weight coefficient.

3. The system according to claim 1, characterized in that, The spatial coordinate remapping module specifically includes: In the initial training phase, the initial absolute joint angles of the patient in a static baseline position are obtained. The difference between the initial absolute joint angles and the standard absolute zero position is calculated to generate an individualized offset angle. The standard absolute zero position refers to the ideal angle when the human body is fully supine and straight. ; In subsequent real-time training, the individualized offset angle will be superimposed on the real-time dynamic joint angle calculated based on the cleaned bone coordinate data to generate the individualized real clinical angle data. Extract the pixel coordinates of the hip joint and the heel on the same side from the cleaned bone coordinate data, and construct a virtual bed surface straight line equation with the hip joint pixel coordinates and the heel pixel coordinates as endpoints; Extract the knee joint pixel coordinates from the cleaned bone coordinate data, calculate the normal distance from the knee joint pixel coordinates to the straight line corresponding to the virtual bed surface straight line equation, and output the normal distance as the anti-compensation normal distance data, which is used to trigger an over-boundary alarm signal when the normal distance is greater than a preset distance threshold.

4. The system according to claim 1, characterized in that, In the motion assessment and state machine control module, the individualized real-world clinical data is input into a preset edge-triggered finite state machine. Based on a dynamic tolerance time window and a forced rest lock mechanism, the motion assessment state results are output, specifically including: The individualized real-world clinical data is compared with the acceptable range of clinical prescription requirements; When the individualized real clinical data is within the passing range, the edge-triggered finite state machine is controlled to flip to the target holding state, and an effective timer is started to accumulate the time. When the target is held and the individualized real clinical perspective data slips out of the passing range, the edge-triggered finite state machine is controlled to suspend the effective timer to pause the accumulation but keep the value from being cleared, and at the same time, a dynamic tolerance time window of preset duration is started to count down. If the individualized real-clinical data returns to the passing range before the countdown of the dynamic tolerance time window ends, then the dynamic tolerance time window is closed and the cumulative countdown of the effective timer is restored. If the countdown of the dynamic tolerance time window ends and the individualized real clinical perspective data has not yet returned to the passing range, then the effective timer is reset and the action evaluation status result of the single action failure is output.

5. The system according to claim 4, characterized in that, In the motion assessment and state machine control module, the individualized real-world clinical data is input into a preset edge-triggered finite state machine, and the motion assessment state result is output based on a dynamic tolerance time window and a forced rest lock mechanism. Specifically, it also includes: Real-time monitoring of the accumulated time value of the effective timer or the number of times the action is completed; When the accumulated time value or the number of times the action is completed in a single action reaches the preset prescription target value, the edge-triggered finite state machine is controlled to forcibly switch to a forced rest lock state with a preset hardware-level lock time length. During the period of the forced rest lock state, any repetitive action request signals for the underlying data engine are intercepted and refused until the hardware-level lock time expires.

6. The system according to claim 1, characterized in that, In the compliance tracking and concurrency scheduling module, when the state of the edge-triggered finite state machine flips, asynchronous voice broadcasting, unaffected by the main visual rendering, is executed by calling the underlying silent process of the operating system. Specifically, this includes: Monitor the state transition edge signal of the edge-triggered finite state machine; When the state toggle edge signal is captured, bypass the foreground visual rendering thread of the main program and send a scheduling instruction to the kernel of the operating system that is decoupled from the main visual rendering thread and runs independently concurrently. The system receives the scheduling instruction to call the underlying speech synthesis and playback interface of the operating system to execute speech broadcasting, and simultaneously starts an audio concurrent mutex based on a global cooldown time window; during the preset cooldown period of the audio concurrent mutex activation cycle, it blocks all subsequently triggered speech broadcasting requests to avoid multiple speech overlaps and prevent concurrent blocking of the main video stream.

7. The system according to claim 1, characterized in that, In the compliance tracking and concurrent scheduling module, a capacity matrix based on a single fact source mechanism is established and front-end user interface instructions are extracted. Combined with the action evaluation status results, closed-loop tracking of medical compliance is performed, specifically including: Analyze clinical prescription parameters and construct a three-dimensional clinical capacity matrix that includes the number of target groups, the number of times per group, and the duration of retention. Run the minimum extreme value truncation algorithm in the underlying data engine; obtain the actual physical completed capacity data corresponding to the action evaluation state result; The actual physical completed capacity data is compared with the prescription target capacity data configured in the three-dimensional clinical capacity matrix. Extract the minimum extreme value between the actual completed physical capacity data and the prescription target capacity data, and write the minimum extreme value as the current compliant physical capacity data into the final database; When the actual physical completion volume data exceeds the prescription target volume data, a physical state lock is generated to freeze the input requests of the front-end user interface and truncate invalid repetitive actions that exceed expectations, and the final rehabilitation training report is generated.

8. The system according to any one of claims 1 to 7, characterized in that, It also includes a port resource conflict resolution module, used to perform pre-port probing and forced termination of residual processes based on the highest system privileges, as well as hardware port release mechanisms, specifically including: Use a front-end port sniffing mechanism to scan the occupancy status of the system physical ports required for the target service to run; When it is detected that the physical port of the system is occupied by a legacy old process, the process identifier corresponding to the old process is extracted; A forced termination signal with the highest privileges is sent to the operating system to destroy the process identifier corresponding to the old process and automatically release the occupied permissions of the system physical port to ensure the uniqueness and security of the current rehabilitation service instance in device hardware scheduling.

9. An adaptive digital rehabilitation method based on machine vision and finite state machines, characterized in that, Includes the following steps: The target video stream of the patient during rehabilitation training is acquired, and an initial set of skeletal key points for consecutive frames is extracted from the target video stream through a computer vision skeleton extraction network. The model confidence corresponding to each key point in the initial set of skeletal key points is then obtained. The initial set of skeletal key points is cleaned based on a confidence lock to generate cleaned skeletal coordinate data. Specifically, when the model confidence of a key point corresponding to a core stress joint is lower than a preset confidence threshold, the confidence lock is triggered to discard the current frame data and freeze the state machine timing logic. Dynamic weighted smoothing filtering based on historical time series is performed on the current frame data that has not been discarded, and the cleaned skeletal coordinate data is output. The spatial coordinates of the cleaned bone coordinate data are remapped using a dual zeroing calibration algorithm and a virtual bed surface model to generate individualized real clinical angle data. The individualized real-world clinical data is input into a preset edge-triggered finite state machine, and the action assessment state results are output based on a dynamic tolerance time window and a forced rest lock mechanism; the edge-triggered finite state machine is configured with rehabilitation action state machine nodes at different levels; Establish a capacity matrix based on a single fact source mechanism and extract front-end user interface instructions. Combine the action assessment status results to perform closed-loop tracking of medical compliance and output the final rehabilitation training report. When the state of the edge-triggered finite state machine flips, asynchronous voice broadcasting is executed by calling the underlying silent process of the operating system, which is not blocked by the main visual rendering.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the adaptive digital rehabilitation method based on machine vision and finite state machine as described in claim 9.

Images