Man-machine interaction-based intelligent follow-up visit management system and method for lower limb fracture operation

Abstract

The invention relates to the technical field of intelligent medical treatment and rehabilitation assistance, and discloses a human-computer interaction-based intelligent follow-up management system and method for a lower limb fracture operation, and the system comprises a cloud server, a medical care terminal and a patient terminal. The patient end collects a rehabilitation action video; the cloud server performs attitude estimation and kinematics calculation through an intelligent analysis engine, and outputs actually measured joint motion range and stride; the feedback and excitation module executes forward excitation for updating the progress of the virtual scene or adaptive feedback for pushing guidance on the basis of a comparison result of the actually measured data and the rehabilitation plan; the rehabilitation diary module monitors the negative deviation rate and triggers graded early warning and doctor-patient real-time interaction. According to the method, traditional qualitative evaluation which depends on visual inspection of medical staff or protractor measurement and is dominated by subjective experience is converted into automatic numerical calculation based on computer vision, manual measurement errors are effectively eliminated, and standard uniformity and objective accuracy of rehabilitation data in the aspect of time series are ensured.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare and rehabilitation assistance technology, specifically to a human-computer interaction-based intelligent follow-up management system and method for lower limb fracture surgery. Background Technology

[0002] Lower limb fractures are common traumas in orthopedic clinics. Surgical reduction and fixation are only the first step in treatment; scientific and systematic postoperative rehabilitation training is crucial for restoring joint range of motion, muscle strength, and gait function. With the popularization of the Enhanced Recovery After Surgery (ERAS) concept, patients are encouraged to perform home rehabilitation training early after surgery. However, in the home rehabilitation setting, away from the professional environment of the hospital, traditional follow-up management models face many insurmountable challenges.

[0003] In the existing rehabilitation assessment system, the evaluation of a patient's recovery mainly relies on regular outpatient follow-up visits. During these visits, medical staff typically use physical tools such as goniometers or rely solely on visual experience to measure the patient's joint range of motion. This method is limited by the subjective judgment and technique of the measurer, lacks standardized objective data, and has a low measurement frequency, failing to continuously and accurately reflect the patient's rehabilitation progress at home. Furthermore, traditional home rehabilitation training often relies on paper instruction manuals or simple video tutorials, making the process tedious and lacking interactivity. Due to the lack of professional, real-time guidance, patients are prone to developing a fear of difficulty when facing postoperative pain and the fear of re-injury, leading to decreased training adherence. More importantly, without an immediate feedback mechanism, patients often cannot determine whether their movements are up to standard. This long-term feedback lag not only easily leads to missing the optimal window for rehabilitation but may even result in poor rehabilitation outcomes due to incorrect training postures.

[0004] With the increasing number of fracture patients, the imbalance between the supply and demand of medical resources is becoming increasingly prominent. Medical staff find it difficult to provide 24 / 7 remote monitoring of every discharged patient. Existing follow-up models mostly rely on passive methods such as telephone follow-ups or inquiries via social media. This extensive management approach makes it difficult for doctors to promptly identify high-risk patients with rehabilitation stagnation or training deviations from the vast follow-up population. This results in some patients experiencing serious complications such as joint stiffness or internal fixation failure due to undetected deviations in their rehabilitation plan implementation.

[0005] Therefore, there is an urgent need for an intelligent follow-up management technology that can provide objective quantitative assessment, enhance patients' willingness to actively train, and provide timely graded early warning of abnormal conditions. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an intelligent follow-up management system and method for lower limb fracture surgery based on human-computer interaction. This system solves the problems of lacking objective and accurate quantitative assessment methods, low patient compliance with active rehabilitation training, and difficulty in timely detection and intervention of abnormal risks in the traditional home rehabilitation process after lower limb fracture surgery.

[0007] To achieve the above objectives, the present invention provides the following technical solution: This invention constructs an intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction. In terms of data flow architecture, the system first establishes quantitative rehabilitation standards based on a time axis through the medical staff, forming a phased rehabilitation plan data package containing target joint range of motion and target stride length. This data package is then distributed to the patient to drive their data collection behavior.

[0008] At the core quantitative analysis level, this invention abandons traditional contact-based measurement or subjective visual estimation methods, and adopts non-contact visual measurement technology based on deep learning. The system receives video streams collected from the patient through an intelligent analysis engine, uses a convolutional neural network to estimate the pose of each frame, and extracts the two-dimensional coordinate sequences of key points in the lower limbs such as the hip, knee, and ankle. To achieve the conversion from pixel coordinates to clinical rehabilitation indicators, the system introduces vector geometry solution logic: by constructing thigh and calf vectors, the angle between the two vectors is calculated using the vector dot product formula, thereby obtaining the knee joint angle for each frame, and the range of motion (ROM) for that movement is calculated through extreme value difference. Simultaneously, the system identifies the maximum pixel Euclidean distance between the left and right ankle joints during the walking cycle and combines it with a scaling factor to calculate the physical stride length. This calculation method based on key point vectorization ensures the objectivity and repeatability of rehabilitation indicators.

[0009] At the interactive control and feedback level, this invention establishes an automatic feedback state machine based on data comparison. The system defines an evaluation function that compares the measured rehabilitation quantitative data obtained from the above calculation with the thresholds in the rehabilitation plan in real time. Based on the comparison results, the system drives a virtual scene state update unit to make logical judgments: when the measured data meets the threshold requirements, the system generates a positive achievement signal, controls the user's virtual rehabilitation path progress variable to increase by a preset step size, and generates a virtual reward certificate, providing the user with immediate feedback through this visualized progress increment; when the measured data does not meet the threshold requirements, the system generates a deviation status signal, freezes the progress variable, calculates the difference vector (such as angle difference), and calls the corresponding error correction guidance information based on the difference vector. This mechanism transforms the tedious rehabilitation data into intuitive path progress control, realizing closed-loop incentives for rehabilitation training.

[0010] At the level of safety monitoring and tiered intervention, the system introduces a linkage mechanism between the rehabilitation diary module and the doctor-patient interaction module. The rehabilitation diary module not only stores quantitative data in a time-series format to generate trend charts, but also has built-in anomaly detection logic. When the negative deviation of the measured data from the target threshold exceeds a preset safety threshold, the system automatically marks the data point as an anomaly. The doctor-patient interaction module monitors this label in real time, and once an anomaly is detected, it immediately triggers an alert to the medical staff and activates the real-time audio and video channel. This changes the traditional passive waiting mode for follow-up visits, realizing proactive remote intervention based on data anomaly triggers.

[0011] This invention provides an intelligent follow-up management system and method for lower limb fracture surgery based on human-computer interaction. It has the following beneficial effects: 1. This invention uses a convolutional neural network to extract the coordinates of key points in the lower limbs through an intelligent analysis engine, and combines a vector geometry algorithm (using the vector dot product formula) to calculate the knee joint angle and stride. This transforms the traditional qualitative assessment, which relies on visual observation or protractor measurement by medical staff and is dominated by subjective experience, into automated numerical calculation based on computer vision. This effectively eliminates human measurement errors and ensures the standardization and objective accuracy of rehabilitation data over time.

[0012] 2. This invention compares the measured data with the preset rehabilitation plan threshold in real time through the feedback and incentive module, and controls the incremental update of the virtual rehabilitation path progress variable based on the comparison results. This enables the provision of confirmation of achievement or deviation prompts at the moment the patient completes the action, solving the problems of long feedback cycle and delayed guidance in the traditional follow-up model. Through immediate positive incentives or error correction feedback, the patient's willingness to perform rehabilitation is strengthened.

[0013] 3. This invention, through the automatic marking function of abnormal tags in the rehabilitation diary module and the early warning triggering mechanism of the doctor-patient interaction module, can automatically filter out high-risk records where the deviation between the measured data and the target exceeds the safety threshold. This enables medical staff to change their work mode from full-scale passive inspection to proactive and precise intervention for abnormal cases. While ensuring that high-risk patients receive timely guidance, it also reduces the occupation of medical resources by routine follow-ups. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is a flowchart of the intelligent analysis engine logic of the present invention; Figure 3 This is the control logic diagram of the feedback and excitation module of the present invention; Figure 4 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] See attached document Figure 1 This invention provides an intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction, comprising: a cloud server, medical staff terminals, and patient terminals. The cloud server, medical staff terminals, and patient terminals are bidirectionally connected via a communication network.

[0017] The communication network is configured to use the TCP / IP protocol suite for data transmission, with the physical layer based on 4G / 5G mobile communication networks or Wi-Fi wireless local area networks. The cloud server is configured as the central node for deploying computing and database services, and its physical carrier is a high-performance computing cluster or virtualized cloud host, used to perform data storage, deep learning inference, and logical operation tasks.

[0018] The healthcare terminal is an interactive device configured for use by healthcare professionals. It includes a display unit, an input unit, and a network communication unit, and its physical form is a desktop computer, laptop computer, or tablet computer. The healthcare terminal internally runs a healthcare module, which is configured to provide a graphical human-computer interaction interface, respond to input commands from healthcare professionals, and generate phased rehabilitation plan data packages.

[0019] Phased Rehabilitation Plan Data Package Defined as a structured data object containing a set of time series and quantitative indicators, its mathematical expression is as follows: ; in, This represents a data package representing a phased rehabilitation plan. This indicates the first step in the rehabilitation plan. Each time point (in days after surgery); Indicates the first The target joint range of motion thresholds that need to be achieved at each time point; Indicates the first The target stride threshold to be achieved at each time point; This indicates the total number of time points included in the rehabilitation plan.

[0020] The patient-side terminal is configured as an interactive device for patient use, and its physical form is a smartphone or tablet. The patient-side terminal includes a camera acquisition unit, a display screen, and a network communication unit. The patient-side terminal internally runs a patient-side module. This module is configured to download and parse the phased rehabilitation plan data package from the cloud server and display the corresponding task instructions on the display screen.

[0021] The camera acquisition unit is configured to acquire raw video stream data containing information about the patient's lower limb movements. The patient-side module is also configured to encode and compress the raw video stream data and upload it to the cloud server.

[0022] Raw video stream data is defined as a sequence of image frames arranged in chronological order, and its mathematical expression is as follows: ; in, This represents the raw video stream data collected by the patient's terminal. Indicates the first in the video stream Frame image; This indicates the total number of frames in the video stream data.

[0023] The cloud server is internally deployed with an intelligent analysis engine, a feedback and incentive module, a rehabilitation diary module, and a doctor-patient interaction module.

[0024] The intelligent analysis engine communicates with the patient-side module, configured to receive raw video stream data, perform computer vision processing on the image frame sequences, and output measured rehabilitation quantitative data. This measured rehabilitation quantitative data includes calculated measured joint range of motion and measured physical stride length.

[0025] The feedback and incentive module is connected to the intelligent analysis engine and is configured to receive measured rehabilitation quantitative data and perform comparative logic operations based on the corresponding thresholds in the phased rehabilitation plan data package to generate feedback control signals.

[0026] The rehabilitation diary module is configured to store measured rehabilitation quantitative data and corresponding analysis results based on timestamp indexes, and to build a rehabilitation trend database.

[0027] The doctor-patient interaction module is configured to establish a signaling channel between the medical staff terminal and the patient terminal for transmitting text messages, control commands, and real-time audio and video streams.

[0028] See attached document Figure 2 The intelligent analysis engine is functionally divided into a video preprocessing unit, a posture estimation unit, a joint range of motion calculation unit, and a stride parameter calculation unit.

[0029] The video preprocessing unit is configured to receive raw video stream data and perform frame decomposition to convert the video stream into a discrete sequence of image frames. For each frame in the sequence, the video preprocessing unit performs resolution normalization to adjust the image size to a standard size suitable for subsequent neural network input, performs color space conversion, and outputs a normalized input tensor.

[0030] The pose estimation unit is configured to load a pre-trained convolutional neural network model, which is built based on an hourglass network architecture or a high-resolution network architecture. The pose estimation unit takes a normalized input tensor as input and outputs a set of human keypoints through multi-layer convolution operations and heatmap regression.

[0031] The human body keypoint set contains two-dimensional coordinate information and confidence scores of specific anatomical locations of the lower limbs. In this embodiment, the human body keypoint set includes at least three keypoints of the affected lower limb: the hip joint keypoint, the knee joint keypoint, and the ankle joint keypoint.

[0032] The mathematical expressions for the three key points are as follows: ; in, Represents the x and y coordinates of the hip joint in the image coordinate system; This represents the x and y coordinates of the knee joint in the image coordinate system. This represents the x and y coordinates of the ankle joint in the image coordinate system. These represent the confidence scores for the corresponding key points, with values ​​ranging from [0,1].

[0033] The pose estimation unit is also configured to perform a threshold filtering operation with a preset validity threshold (e.g., 0.6). When the confidence score of any keypoint is detected to be lower than the validity threshold, the frame data is marked as an invalid frame or completed using coordinate data from the preceding and following frames through a temporal interpolation algorithm.

[0034] The joint mobility calculation unit is configured to perform vector geometry operations based on filtered keypoint coordinates. First, the joint mobility calculation unit constructs a thigh vector representing the thigh bone axis and a lower leg vector representing the lower leg bone axis based on the coordinates.

[0035] The logic for constructing vectors is as follows: ; Subsequently, the joint range of motion calculation unit uses the vector dot product formula to calculate the knee joint angle corresponding to each frame of the image. This knee joint angle represents the angle between the thigh vector and the lower leg vector, and is used to reflect the flexion and extension state of the knee joint.

[0036] knee joint angle The calculation formula is as follows: ; in, Indicates the first The knee joint angle (in degrees) corresponding to the frame image; This represents the vector dot product operation; The magnitude (length) of a vector; This represents the inverse cosine function.

[0037] After processing the entire video sequence, the joint range of motion calculation unit traverses the knee joint angle values ​​of all frames, extracts the maximum and minimum values, and determines the difference between the two as the measured joint range of motion for this movement.

[0038] The stride parameter calculation unit is configured to operate in gait analysis mode. This unit identifies the coordinates of the left and right ankle joints in the video stream and calculates the maximum Euclidean distance between them within a single walking cycle to obtain the pixel stride.

[0039] The stride parameter calculation unit pre-stores a scaling factor. This scaling factor is calculated by comparing the user's actual height data with a statistical human limb proportion model, or by identifying a reference object of known size in the video background. Its unit is centimeters per pixel.

[0040] The stride parameter calculation unit converts pixel stride into measured physical stride. The calculation logic is as follows: ; in, Indicates the measured physical stride length; Indicates the scaling factor; Indicates the first The horizontal and vertical coordinates of the left ankle joint in the frame; Indicates the first The horizontal and vertical coordinates of the right ankle joint in the frame.

[0041] Finally, the intelligent analysis engine packages and outputs the calculated measured physical stride length to the feedback and incentive module and the rehabilitation diary module.

[0042] See attached document Figure 3 The feedback and incentive module includes a numerical comparison unit, a virtual scene state update unit, and an adaptive guidance retrieval unit in its logical architecture.

[0043] The numerical comparison unit is configured to receive measured rehabilitation quantitative data from the intelligent analysis engine and obtain the threshold values ​​of the phased rehabilitation plan corresponding to the current time point from the medical care module. The numerical comparison unit performs logical judgments based on a preset evaluation function, outputting a binary state determination signal and a deviation vector of continuous values.

[0044] The logical definition of the evaluation function is as follows: ; in, This indicates a status determination signal, with a value of 1 representing a qualified state and a value of 0 representing a non-qualified state. Indicates the measured range of motion of the joint; This indicates the preset target joint mobility at the current time point; Indicates the measured physical stride length; This indicates the preset target step size at the current time point.

[0045] When the judgment result is that the standard is not met, the numerical comparison unit further calculates the deviation vector, the mathematical expression of which is as follows: ; in, Represents the deviation vector; Indicates the missing value of joint range of motion; This indicates missing stride values.

[0046] The virtual scene state update unit is configured to maintain a persistent virtual rehabilitation path progress variable. This virtual rehabilitation path progress variable is mapped to virtual map coordinates or task progress bar values ​​in the user interface. The virtual scene state update unit uses a finite state machine model to perform state transitions and variable updates based on the input state judgment signals.

[0047] When received When the progress status is set to 1 (achieving the target status), the virtual scene status update unit executes positive incentive logic: it increments the current progress variable by a preset step size, triggers the rendering engine to generate virtual reward credentials (such as electronic certificates or virtual badge objects), and writes the updated progress status to the cloud database. The update logic is as follows: ; in, This indicates the updated progress value of the virtual rehabilitation path; This indicates the progress value of the virtual rehabilitation path before the update; This indicates the fixed increment step size corresponding to a single achievement.

[0048] When received When the signal is 0 (not met), the virtual scene state update unit keeps the progress variable value unchanged and activates the adaptive guidance retrieval unit.

[0049] The adaptive guidance retrieval unit is configured to receive a deviation vector and perform a weighted search in a pre-stored rehabilitation knowledge base based on this deviation vector. If the missing value of joint range of motion is significantly greater than zero, the system retrieves guidance data for specific stretching movements for joint stiffness; if the missing value of stride length is significantly greater than zero, the system retrieves training suggestions for gait balance. The search results are encapsulated as multimedia feedback messages and pushed to the patient's display interface in real time.

[0050] See attached document Figure 4 This section details the data organization of the rehabilitation plan generated by the medical staff module, as well as the data storage and anomaly marking mechanism of the rehabilitation diary module.

[0051] The phased rehabilitation plan data package is transmitted and parsed internally using a serialized data structure (e.g., JSON or XML format). The data structure consists of several time node objects arranged in chronological order. Each time node object is encapsulated as an independent data unit containing multi-dimensional control parameters.

[0052] The specific data field definitions for the time node object include: Time-series index field: identifies the number of days after surgery or the absolute calendar date corresponding to this time point, used for system scheduling task triggering time; Quantitative target fields: contain a set of rehabilitation indicator thresholds that should be achieved at this time point, specifically including target joint range of motion values ​​and target stride length values; Teaching resource field: Stores the Uniform Resource Locator (URL) or hash index value of the standard teaching video file corresponding to the rehabilitation movement; Safety constraint field: Defines the maximum allowable range of motion during this phase to prevent patients from overtraining and causing secondary injuries.

[0053] The rehabilitation diary module is configured to build a time-series-based database table structure. Each time the intelligent analysis engine completes a calculation, the rehabilitation diary module creates a new diary entry. Each diary entry includes: measurement timestamp, measured joint range of motion, measured physical stride length, the storage path index of the raw video data, and a status tag field.

[0054] The rehabilitation diary module has a built-in trend analysis engine, configured to respond to query commands and retrieve all diary entries within a specific time span. This trend analysis engine extracts the measured joint activity values ​​and measured physical stride values ​​from each entry as the ordinate and the timestamp as the abscissa. It then uses linear interpolation or spline interpolation algorithms to plot a rehabilitation trend curve and outputs it to the display interfaces of both the medical staff terminal and the patient terminal.

[0055] The rehabilitation diary module is also configured to execute anomaly detection logic. This module has a preset safety deviation threshold. When generating diary entries, the module calculates the negative deviation rate between the measured data and the planned goals. The calculation logic for the negative deviation rate is as follows: ; in, Indicates the negative deviation rate; This indicates the preset target joint mobility at the current time point; This indicates the measured range of motion of the joint.

[0056] If the calculated negative deviation rate exceeds the safety deviation threshold, the rehabilitation diary module will change the status label field of the current diary entry to "Abnormal" and generate a unique abnormal event ID. This abnormal event ID is used to trigger subsequent doctor-patient interaction processes and is highlighted in the trend curve graph with a specific visual identifier (such as a red data point) to distinguish it from normal records.

[0057] See attached document Figure 4 This section details how the doctor-patient interaction module triggers remote intervention based on data-driven approaches, and the process of establishing real-time communication channels.

[0058] The doctor-patient interaction module includes an abnormal event listener, an early warning message distribution unit, and a real-time audio and video communication controller in its architecture.

[0059] The exception event listener is configured to reside in a background process on the cloud server, employing the observer pattern or a message queue subscription mechanism to monitor the data writing status of the rehabilitation diary module in real time. When the rehabilitation diary module updates the status tag field of a rehabilitation diary record entry to "abnormal," the exception event listener immediately captures the status change event and extracts the corresponding exception event ID and patient identification.

[0060] The early warning message distribution unit is configured to generate a structured early warning data packet in response to the trigger signal of the abnormal event listener. The early warning data packet includes basic patient information, the timestamp that triggered the abnormality, the specific negative deviation rate value that caused the abnormality, and the access link to the corresponding original video segment.

[0061] The early warning message distribution unit pushes the early warning data packet to the healthcare terminal via a WebSocket long connection channel or mobile push service (APNs / FCM). Upon receiving the early warning data packet, the healthcare module displays a high-priority alert window in a prominent position on the interactive interface and highlights the patient's recovery record.

[0062] The real-time audio and video communication controller is configured to establish a low-latency streaming media transmission channel between the healthcare provider's terminal and the patient's terminal. This low-latency streaming media transmission channel is established in accordance with the WebRTC protocol standard.

[0063] The specific establishment process is as follows: When medical staff click the intervention control in the warning window on the medical staff terminal, the real-time audio and video communication controller sends a connection request to the cloud signaling server. The signaling server exchanges session description protocol objects between the two communicating parties, including Offer signaling and Answer signaling, as well as interactive connection establishment (ICE) candidate address information.

[0064] After completing NAT traversal and P2P connection negotiation, the real-time audio and video communication controller establishes a Secure Transport Layer Protocol (SRTP) channel for bidirectional transmission of encrypted audio and video streams. If the P2P connection fails to establish, the system automatically switches to TURN relay server mode to forward media streams, ensuring the connectivity of the communication link.

[0065] During the communication process, the medical staff module is also configured to use picture-in-picture (PiP) or split-screen mode. While displaying the real-time video call screen, it simultaneously replays the patient's rehabilitation action history video and quantitative data charts that triggered the warning, assisting medical staff in remote diagnosis and guidance.

[0066] See attached document Figure 4 This section details the complete closed-loop workflow of the system, from rehabilitation plan development to execution, analysis, feedback, and intervention.

[0067] Step S1: Rehabilitation Plan Development and Distribution. The healthcare provider module responds to input commands from healthcare personnel, receiving rehabilitation parameter settings for a specific patient through a graphical interface. The system encapsulates the input time points and their corresponding target joint range of motion and target stride length, generating a structured, phased rehabilitation plan data package. The cloud server receives this phased rehabilitation plan data package and pushes it to the designated patient terminal. The patient module receives and parses the package, locking in the rehabilitation task indicators to be performed today based on the current date.

[0068] Step S2: Video Stream Data Acquisition and Upload. Based on the parsed task indicators, the patient terminal displays guidance prompts on the screen. The camera acquisition unit starts working, acquiring a continuous sequence of images of the patient performing lower limb rehabilitation movements at a preset frame rate (e.g., fps or fps), generating raw video stream data. After acquisition, the patient terminal module uploads the raw video stream data to the intelligent analysis engine on the cloud server via a secure encrypted channel.

[0069] Step S3: The kinematic parameter quantification and intelligent analysis engine performs pipelined processing on the received raw video stream data. First, the video preprocessing unit decomposes the video stream into frames and normalizes them. Next, the pose estimation unit uses a convolutional neural network model to infer each frame, outputting a set of lower limb key points containing the coordinates of the hip, knee, and ankle joints. Subsequently, the joint range of motion calculation unit and the stride parameter calculation unit, based on the key point coordinates, use the aforementioned vector construction and dot product formula algorithm to calculate the measured joint range of motion and the measured physical stride, collectively referred to as the measured rehabilitation quantification data.

[0070] Step S4: The threshold comparison and logical judgment feedback and incentive module obtains the quantitative indicator threshold at the current time point and compares the measured rehabilitation quantitative data with the threshold. The system calculates the evaluation function to determine whether the measured joint range of motion is greater than or equal to the target joint range of motion and whether the measured physical stride is greater than or equal to the target stride.

[0071] Step S5: Feedback Execution and Virtual Scene Update. Based on the judgment result of step S4, the system executes the divergence control strategy: If the judgment result is yes (achieved), the virtual scene status update unit reads the current virtual rehabilitation path progress variable, performs an addition operation on it, increases the preset incremental step size, and writes the updated progress value into the database. Simultaneously, it renders the forward animation and reward certificate on the patient's interface. If the judgment result is no, the virtual scene status update unit keeps the progress variable value locked. At the same time, the system calculates the deviation vector, retrieves the corresponding rehabilitation guidance multimedia content based on the deviation type, and pushes it to the patient.

[0072] Step S6: Anomaly Monitoring and Tiered Intervention While performing step S5, the rehabilitation diary module calculates the negative deviation rate between the measured data and the target threshold. If the negative deviation rate exceeds the preset safety deviation threshold, the system performs the following tiered intervention operations: Add abnormal status tags to entries in the recovery diary; After the doctor-patient interaction module detects the abnormal status tag, it immediately sends a warning data packet containing details of the abnormal event to the medical staff terminal; In response to confirmation instructions from medical staff, a real-time audio and video communication channel using WebRTC is established, enabling medical staff to view historical playbacks and provide remote real-time guidance.

Claims

1. A human-computer interaction-based intelligent follow-up management system for lower limb fracture surgery, characterized in that, include: The medical staff module is used to respond to the operations of medical staff and generate a phased rehabilitation plan that includes time nodes and quantitative rehabilitation indicator thresholds. The patient-side module is used to receive the phased rehabilitation plan and collect video stream data containing the patient's lower limb movement information; The intelligent analysis engine, deployed on a cloud server, receives the video stream data, performs frame decomposition and pose estimation on the video stream data, extracts the coordinates of key points of the human lower limbs, and calculates the measured rehabilitation quantitative data based on the coordinates of the key points. The feedback and incentive module is used to compare the measured rehabilitation quantitative data with the quantitative rehabilitation indicator thresholds in the phased rehabilitation plan, and generate corresponding feedback control signals based on the comparison results.

2. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction as described in claim 1, characterized in that, The intelligent analysis engine includes a pose estimation unit based on a convolutional neural network. The pose estimation unit is used to extract features from each frame of the video stream data and output a set of lower limb key points containing confidence scores. The set of key points for the lower limbs includes at least the coordinates of the hip joint, the knee joint, and the ankle joint.

3. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction as described in claim 2, characterized in that, The intelligent analysis engine also includes a joint range of motion calculation unit, which performs the following calculation logic: Construct a thigh vector based on the hip joint coordinates and the knee joint coordinates; A lower leg vector is constructed based on the knee joint coordinates and the ankle joint coordinates; The angle between the thigh vector and the calf vector is calculated using the vector dot product formula to obtain the knee joint angle corresponding to each frame of the image. The difference between the maximum and minimum knee joint angles in the video sequence is calculated to obtain the joint mobility data used as the measured rehabilitation quantification data.

4. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction as described in claim 1, characterized in that, The intelligent analysis engine also includes a stride parameter calculation unit, which performs the following calculation logic: Identify the coordinates of the left and right ankle joints in the video stream data; Calculate the maximum Euclidean distance between the coordinates of the left and right ankle joints within a single walking cycle to obtain the pixel stride. Obtain a preset scaling factor and map the pixel stride to the physical stride as the measured rehabilitation quantitative data.

5. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction according to claim 1, characterized in that, The comparison logic executed by the feedback and incentive module includes: Define an evaluation function to determine whether the measured quantitative rehabilitation data is greater than or equal to the corresponding quantitative rehabilitation indicator threshold in the phased rehabilitation plan; If the judgment result is yes, a positive compliance status signal is generated; If the judgment result is negative, a non-compliance deviation status signal is generated, and the difference vector between the measured rehabilitation quantitative data and the quantitative rehabilitation index threshold is calculated.

6. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction according to claim 5, characterized in that, The feedback and incentive module further includes a virtual scene status update unit, which is used to maintain the user's virtual rehabilitation path progress variables. The virtual scene status update unit includes: When the positive achievement status signal is received, the virtual rehabilitation path progress variable is increased by a preset increment step, and a virtual reward certificate is generated. When the non-compliance deviation status signal is received, the virtual rehabilitation path progress variable is kept unchanged, and the corresponding rehabilitation guidance information is retrieved according to the difference vector.

7. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction according to claim 1, characterized in that, The data structure of the phased rehabilitation plan includes several ordered time node objects; Each time point object contains preset target joint range of motion values ​​and target stride values.

8. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction according to claim 1, characterized in that, It also includes a rehabilitation diary module, which includes: The measured quantitative rehabilitation data are stored in time series, and a rehabilitation trend curve is generated. The rehabilitation diary module also includes adding an anomaly label to the measured rehabilitation quantitative data when the deviation between the measured rehabilitation quantitative data and the quantitative rehabilitation index threshold exceeds a preset safety threshold.

9. The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction according to claim 8, characterized in that, It also includes a doctor-patient interaction module, which includes: The system monitors abnormal tags in the rehabilitation diary module. When an abnormal tag is detected, it automatically sends an early warning message to the medical care module and opens a real-time audio and video communication channel between the medical care module and the patient module.

10. A method for intelligent follow-up management of lower limb fractures based on human-computer interaction, characterized in that, The intelligent follow-up management system for lower limb fracture surgery based on human-computer interaction, as described in any one of claims 1-9, includes the following steps: Step S1: Set up a phased rehabilitation plan, including time nodes and quantitative rehabilitation indicator thresholds, through the medical staff module and push it to the patient module; Step S2: Collect video stream data of the patient performing rehabilitation actions through the patient-side module and upload it to the intelligent analysis engine; Step S3: Use the intelligent analysis engine to perform frame decomposition on the video stream data, extract the coordinates of key points of the lower limbs based on the convolutional neural network, and use the vector geometry algorithm to calculate the measured rehabilitation quantitative data. Step S4: Use the feedback and incentive module to compare the measured rehabilitation quantitative data with the quantitative rehabilitation indicator threshold; Step S5: If the measured rehabilitation quantitative data reaches the threshold of the quantitative rehabilitation index, update the virtual rehabilitation path progress variable and increase the incremental step size; if it does not reach the threshold, calculate the deviation value and keep the progress variable unchanged.

Images