Rehabilitation follow-up visit system for head and neck tumors

Abstract

The invention discloses a head and neck tumor intelligent rehabilitation follow-up visit system based on dynamic risk-rehabilitation demand two-dimensional layering and multi-mode fusion, and belongs to the technical field of video telephone medical treatment. The system comprises a medical semantic enhanced edge video terminal used for extracting and transmitting a lip-tongue-soft palate key point rehabilitation semantic layer under a low bandwidth; the rehabilitation session management core initiates video follow-up with prescription information based on an extension protocol; the multi-modal closed-loop guidance engine fuses visual, physiological and acoustic data to realize AR real-time error correction and scheme dynamic adjustment; and the blockchain evidence storage and privacy protection gateway ensures data security and auditing of the process. The problems that traditional follow-up visit is high in visit losing rate and rehabilitation guidance lacks real-time performance and individuation are solved, hospital-home closed-loop management is achieved through deep combination of narrow-band video communication and rehabilitation medicine semantics, and rehabilitation efficiency and quality are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of video telephone medical technology, specifically a rehabilitation follow-up system for head and neck tumors. Background Technology

[0002] Postoperative head and neck tumor patients often face functional impairments such as swallowing and speech. Rehabilitation follow-up is a crucial step in ensuring postoperative functional recovery and reducing complications. Traditional rehabilitation follow-up methods mainly rely on video communication between doctors and patients and simple assistive tools. For example, a head and neck tumor rehabilitation follow-up system disclosed in Chinese Patent CN115334275A uses a head and neck model display module combined with an angle adjustment component, allowing doctors to adjust the display angle of the head and neck model. Images are transmitted to the patient's smart screen via a camera, assisting in visual comparison between doctors and patients regarding the location of the affected area and rehabilitation movements. This method, to some extent, improves the difficulty of location recognition for elderly patients or those with poor comprehension, enhancing the intuitiveness of follow-up.

[0003] However, existing technologies based on physical models and basic video communication still have significant drawbacks: First, the system relies on static model display and manual operation, making it unable to provide real-time, accurate quantitative assessment and correction of patients' rehabilitation movements, and lacking the ability to dynamically capture the biomechanical characteristics of key areas such as the lips, tongue, and soft palate; second, the system lacks network adaptability, making it prone to video interruption or delay in environments with fluctuating bandwidth or poor network conditions, affecting the continuity of follow-up; third, it lacks multimodal physiological data fusion and intelligent decision-making mechanisms, making it impossible to dynamically adjust rehabilitation plans according to the patient's real-time status, and making it difficult to achieve personalized, closed-loop rehabilitation guidance; furthermore, existing systems are relatively weak in terms of data security and privacy protection, lacking credible evidence storage and anonymization processing of rehabilitation process data. These shortcomings limit the efficiency and effectiveness of rehabilitation follow-up, especially in terms of high-precision rehabilitation guidance, remote adaptability, and data security. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a rehabilitation follow-up system for head and neck tumors, solving the problems mentioned in the background section.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: a rehabilitation follow-up system for head and neck tumors, comprising: The medical semantic enhancement edge video terminal has a built-in rehabilitation semantic extraction module for real-time analysis of the acquired 4K video stream, extracting the lip-tongue-soft palate biomechanical key point sequence that defines the semantics of rehabilitation actions, and generating an independent low-bandwidth rehabilitation semantic layer data stream; it also integrates a network adaptive coding module for seamless switching between a first mode containing the full video base layer and rehabilitation semantic layer, a second mode containing only the video region of interest and rehabilitation semantic layer, and a third mode containing only the rehabilitation semantic layer, based on the real-time network bandwidth. The core of rehabilitation session management is used to run a dynamic risk-rehabilitation needs dual-dimensional hierarchical engine to generate and dynamically adjust personalized rehabilitation prescriptions for patients; and to establish video follow-up based on an extended rehabilitation session initiation protocol, in which the prescription information for the current rehabilitation session is embedded in the call signaling. The multimodal closed-loop guidance engine is used to fuse rehabilitation semantic layer data from the video channel and physiological modality data from the auxiliary channel in real time during video sessions. Based on the fusion result, it generates augmented reality error correction guidance in real time and overlays it onto the downlink video stream, and dynamically adjusts subsequent training parameters to form a real-time perception-decision-control closed loop. The blockchain-based evidence storage and privacy protection gateway is used to generate aggregate hashes for keyframes, AI scores, and operation instructions of each rehabilitation video session and store them on the blockchain; and to perform real-time desensitization processing on the original video stream on the terminal side to ensure that only rehabilitation semantic layer data or video streams with blurred backgrounds are transmitted.

[0006] Preferably, the biomechanical key point sequence extracted by the rehabilitation semantic extraction module is synchronously encapsulated and transmitted with the compressed video base layer through SEI frames of an H.265 encoder, forming a layered video stream. At the receiving end, the rehabilitation semantic layer data is parsed before the video base layer to drive the virtual coaching model and perform motion assessment. By utilizing the extension mechanism of the international standard encoder, precise synchronization between the rehabilitation semantic data and the video frames is achieved, ensuring that the motion assessment and the video footage remain strictly consistent in an asynchronous network transmission environment. This provides a technical foundation for accurate quantitative analysis of rehabilitation, while also ensuring the compatibility and feasibility of the solution.

[0007] Preferably, when the network adaptive coding module detects that the available bandwidth is less than 300 kbps, it automatically switches to the third mode and uses forward error correction technology to ensure the complete recovery of rehabilitation semantic layer data under a 10% network packet loss rate. By prioritizing the transmission of core semantic data under the worst network conditions, it ensures that the core function of rehabilitation guidance and assessment is never interrupted, effectively solving the industry pain point of patient loss to follow-up or rehabilitation interruption due to network problems. It is particularly suitable for rural areas or areas with weak network infrastructure.

[0008] Preferably, the extended rehabilitation session initialization protocol is an extension based on the SIP protocol. The rehabilitation prescription information field added to its call signaling includes at least: the action identifier of this training, the target repetition count, the AI ​​evaluation mode identifier, and the expected key physiological parameter threshold. The terminal clearly defines the goal and rules of this rehabilitation at the beginning of the session, which greatly improves the standardization and efficiency of remote rehabilitation and avoids the tediousness of medical staff having to repeat the training content in each call.

[0009] Preferably, the multimodal closed-loop guidance engine employs a cross-modal attention mechanism for data fusion. Its state space includes real-time acquired keypoint coordinates, root mean square values ​​of electromyography signals, and joint angles, while its action space consists of AR visual cues or voice correction commands superimposed on the video stream. The dynamic adjustment uses a deep deterministic strategy gradient algorithm, with the improvement of rehabilitation effect and patient fatigue as reward functions, outputting adjustments to the difficulty of subsequent movements or the interval time. Through the attention mechanism, the AI ​​can focus on the most relevant abnormal signals like an expert, reducing misjudgments. Through reinforcement learning, the system can simulate the behavior of "experts adjusting plans in real time," providing each patient with dynamically optimized and unique rehabilitation training, thereby accelerating the rehabilitation process.

[0010] Preferably, in the blockchain evidence storage and privacy protection gateway, the real-time desensitization process includes: locating the facial area on the terminal side using face detection technology and performing real-time Gaussian blurring on the background of non-lip and tongue areas, or directly discarding the original video stream and only uploading the rehabilitation semantic layer data.

[0011] Preferably, the system further includes an offline mode management module, which automatically enables locally cached lightweight AI models and rehabilitation task packages to perform offline assessments when a network interruption is detected, and temporarily stores the assessment results in the terminal secure storage area; after the network is restored, the offline data is uploaded to the server through a differential synchronization algorithm, and a hash value representing the integrity of the offline treatment is generated and written to the blockchain.

[0012] Preferably, the system supports a tiered diagnosis and treatment model, where community hospital nodes can initiate lightweight follow-up sessions based on rehabilitation semantic layer data. When the AI ​​assessment detects abnormal or high-risk indicators, the system automatically generates upward referral suggestions and pushes an assessment report containing rehabilitation semantic layer data to the expert end of the superior hospital.

[0013] A method for follow-up rehabilitation of head and neck tumors, using the aforementioned head and neck tumor rehabilitation follow-up system, includes the following steps: S1. Collect patient rehabilitation videos through edge video terminals and generate low-bandwidth rehabilitation semantic layer data streams in real time; S2, the rehabilitation session management core initiates rehabilitation video sessions based on the patient's dynamic risk level and sends rehabilitation prescriptions to the terminal through an extended rehabilitation session initialization protocol; S3. During the video session, the multimodal closed-loop guidance engine integrates rehabilitation semantic layer and physiological data in real time, generates AR error correction guidance and sends it through the video link to form closed-loop guidance; S4. After the session ends, the blockchain evidence gateway performs hash-based evidence storage on the key data of the session and ensures that the original video data is privacy-protected on the terminal side.

[0014] Preferably, the assessment of the dynamic risk level is a continuous process. The system receives the imaging assessment results, patient-reported outcome indicators, and multimodal rehabilitation indicator sequences from each patient follow-up visit. It calculates the probability distribution of the risk level using a Transformer-based prognostic prediction model, and automatically triggers adjustments to the rehabilitation prescription when the probability exceeds a preset threshold. Through continuous learning and prediction, the system can proactively identify the risks of disease deterioration and rehabilitation stagnation, achieving a shift from "post-event follow-up" to "pre-event intervention." This truly reflects the predictive nature of smart healthcare, and is expected to detect relapse signs early, improving patient prognosis.

[0015] (III) Beneficial Effects This invention provides a rehabilitation follow-up system for head and neck tumors, which has the following beneficial effects: 1. This invention enables real-time, accurate assessment and interactive correction of rehabilitation movements. The system extracts key biomechanical points of the lips, tongue, and soft palate via an edge terminal, integrates physiological data, and uses AI algorithms to generate real-time AR (augmented reality) guidance overlaid on the video stream, forming a closed-loop guidance system. This overcomes the shortcomings of traditional methods that rely on static models and visual observation and cannot quantify error correction, significantly improving the standardization and effectiveness of rehabilitation training.

[0016] 2. This invention possesses strong network adaptive capabilities, ensuring continuity of follow-up. The system can seamlessly switch between modes such as transmitting full video, partial video, or only low-bandwidth semantic data based on real-time bandwidth, and ensure the integrity of core data even under weak network conditions. This overcomes the pain points of traditional video systems, which are overly dependent on network quality and prone to interruptions, thus ensuring the accessibility of rehabilitation services.

[0017] 3. This invention achieves personalized solutions and secure, reliable data through dynamic risk stratification and blockchain-based evidence storage. The system can predict risks and dynamically adjust prescriptions based on continuous patient data, enabling proactive intervention. Simultaneously, key data is stored on the blockchain for evidence storage, and original videos are anonymized in real-time on the terminal. This solves the problems of traditional systems lacking dynamic optimization and having weak data privacy protection, providing a reliable basis for hierarchical medical treatment. Attached Figure Description

[0018] Figure 1This is a schematic diagram of the system of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: like Figure 1 As shown, this embodiment provides a system implementation plan based on a dedicated hardware terminal, which is suitable for patients with high requirements for rehabilitation quality and those under key management by hospitals.

[0021] Hardware Configuration: The edge computing camera terminal uses the Hisilicon Hi3519DV500 main control chip, with a built-in NPU boasting 4 TOPS of computing power. The camera module uses the Sony IMX415, supporting video stream output at 3840×2160 resolution (4K) @ 30 fps. The terminal integrates an IMU (Inertial Measurement Unit) and Bluetooth 5.0 module for connecting peripherals such as surface electromyography (sEMG) sensors. On the hospital server side, the rehabilitation session management core, dual-dimensional hierarchical engine, and blockchain nodes are deployed on the hospital's private cloud, using Docker containerization.

[0022] Software and Protocol Implementation: Custom development is based on FFmpeg 4.4. Within the H.265 encoding standard, SEI (Supplemental Enhancement Information) frames are used to carry coordinate data of 26 key points on the lips, tongue, and soft palate, enabling synchronous encapsulation and transmission of the rehabilitation semantic layer and the video base layer. For the communication protocol, the standard SIP protocol is extended, defining a new message header to carry JSON-formatted rehabilitation prescription information, including fields such as action identifiers and target repetition counts. A multimodal fusion AI assessment model runs on the terminal side, performing real-time fusion analysis of video key point sequences, sEMG signals, and IMU angle data.

[0023] Workflow: After surgery, doctors enter pathology information into the HIS system, which automatically generates an initial risk level and rehabilitation prescription. At a fixed time each day, the terminal automatically initiates a rehabilitation video session. The patient faces the terminal screen and trains with a virtual coach. The terminal's AI analyzes the patient's movements in real time; if an error is detected, AR guidance is immediately generated on the screen for correction. Training data is locally anonymized, and key information is hashed and uploaded to a blockchain-based evidence storage platform built on FISCO-BCOS. If the system detects that a patient has missed two consecutive training sessions, the smart contract automatically triggers a voice call and notifies a community nurse to intervene at home, forming a closed-loop management system.

[0024] Example 2: This embodiment provides a lightweight implementation based on a smartphone APP, aiming to reduce hardware costs and expand the system's reach.

[0025] Hardware and Platform Configuration: No dedicated terminal is required for patients; a customized app needs to be installed on a smartphone supporting iOS 12.0 or Android 8.0 and above. External portable sEMG and IMU sensors connect to the phone via Bluetooth. The core system services are provided by a medical cloud service provider in a SaaS (Software as a Service) format.

[0026] Software optimization and features: The key point detection and fusion evaluation model is pruned and quantized to create a lightweight AI model smaller than 50MB, meeting the requirements for smooth operation on mobile devices. The app has network adaptive capabilities, monitoring network conditions in real time and automatically switching to a key point-only data stream transmission mode when bandwidth is poor, while employing forward error correction technology to ensure data integrity. The app supports offline functionality, allowing for pre-caching of rehabilitation guidance videos and AI models, and supports training and data storage in environments without network access.

[0027] Workflow: Patients scan a QR code provided by their doctor via the app to link their personal rehabilitation plan. During training, the phone's front-facing camera captures facial video, and the app uses the phone's NPU to calculate key points and perform AI assessments in real time. Rehabilitation data is encrypted and uploaded to a cloud platform, which automatically generates a treatment report that meets medical insurance requirements. Community doctors can manage patients in their jurisdiction through a web-based backend, view rehabilitation trends after desensitization, and initiate referrals to higher-level hospitals when abnormalities are detected, thus achieving tiered medical services.

[0028] In summary, the present invention provides feasible solutions with both high and low configurations through the above specific embodiments, effectively solving key problems in the follow-up rehabilitation of head and neck tumor patients.

[0029] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A rehabilitation follow-up system for head and neck tumors, characterized in that, include: The medical semantic enhancement edge video terminal has a built-in rehabilitation semantic extraction module for real-time analysis of the acquired 4K video stream, extracting the lip-tongue-soft palate biomechanical key point sequence that defines the semantics of rehabilitation actions, and generating an independent low-bandwidth rehabilitation semantic layer data stream; it also integrates a network adaptive coding module for seamless switching between a first mode containing the full video base layer and rehabilitation semantic layer, a second mode containing only the video region of interest and rehabilitation semantic layer, and a third mode containing only the rehabilitation semantic layer, based on the real-time network bandwidth. The core of rehabilitation session management is used to run a dynamic risk-rehabilitation needs two-dimensional hierarchical engine to generate and dynamically adjust personalized rehabilitation prescriptions for patients. And video follow-up is established based on the extended rehabilitation session initiation protocol, in which the prescription information for the current rehabilitation session is embedded in the call signaling; The multimodal closed-loop guidance engine is used to fuse rehabilitation semantic layer data from the video channel and physiological modality data from the auxiliary channel in real time during video sessions. Based on the fusion result, it generates augmented reality error correction guidance in real time and overlays it onto the downlink video stream, and dynamically adjusts subsequent training parameters to form a real-time perception-decision-control closed loop. The blockchain-based evidence storage and privacy protection gateway is used to generate aggregate hashes for keyframes, AI scores, and operation instructions of each rehabilitation video session and store them on the blockchain; and to perform real-time desensitization processing on the original video stream on the terminal side to ensure that only rehabilitation semantic layer data or video streams with blurred backgrounds are transmitted.

2. The rehabilitation follow-up system for head and neck tumors according to claim 1, characterized in that: The biomechanical key point sequence extracted by the rehabilitation semantic extraction module is synchronously encapsulated and transmitted with the compressed video base layer through SEI frames of the H.265 encoder to form a layered video stream. At the receiving end, the rehabilitation semantic layer data is parsed prior to the video base layer and used to drive the virtual coaching model and perform motion assessment.

3. The rehabilitation follow-up system for head and neck tumors according to claim 1, characterized in that: When the network adaptive coding module detects that the available bandwidth is less than 300 kbps, it automatically switches to the third mode and uses forward error correction technology to ensure the complete recovery of the rehabilitation semantic layer data under a 10% network packet loss rate.

4. The rehabilitation follow-up system for head and neck tumors according to claim 1, characterized in that: The extended rehabilitation session initialization protocol is an extension based on the SIP protocol. The rehabilitation prescription information fields added to its call signaling include at least: the action identifier for this training, the target number of repetitions, the AI ​​evaluation mode identifier, and the expected key physiological parameter thresholds.

5. The rehabilitation follow-up system for head and neck tumors according to claim 1, characterized in that: The multimodal closed-loop guidance engine uses a cross-modal attention mechanism for data fusion. Its state space includes real-time acquired key point coordinates, root mean square values ​​of electromyography signals, and joint angles. Its action space consists of AR visual cues or voice correction commands superimposed on the video stream. The dynamic adjustment uses a deep deterministic strategy gradient algorithm, with the improvement of rehabilitation effect and patient fatigue as reward functions, and outputs the adjustment amount for the difficulty of subsequent actions or the interval time.

6. The rehabilitation follow-up system for head and neck tumors according to claim 1, characterized in that: In the blockchain-based evidence storage and privacy protection gateway, the real-time desensitization process includes: locating the facial area on the terminal side using face detection technology, and performing real-time Gaussian blurring on the background of non-lip and tongue areas, or directly discarding the original video stream and uploading only the rehabilitation semantic layer data.

7. A method for follow-up rehabilitation of head and neck tumors, applied to the head and neck tumor rehabilitation follow-up system according to any one of claims 1-6, characterized in that, Includes the following steps: S1. Collect patient rehabilitation videos through edge video terminals and generate low-bandwidth rehabilitation semantic layer data streams in real time; S2, the rehabilitation session management core initiates rehabilitation video sessions based on the patient's dynamic risk level and sends rehabilitation prescriptions to the terminal through an extended rehabilitation session initialization protocol; S3. During the video session, the multimodal closed-loop guidance engine integrates rehabilitation semantic layer and physiological data in real time, generates AR error correction guidance and sends it through the video link to form closed-loop guidance; S4. After the session ends, the blockchain evidence gateway performs hash-based evidence storage on the key data of the session and ensures that the original video data is privacy-protected on the terminal side.

8. The rehabilitation follow-up system for head and neck tumors according to claim 7, characterized in that: The assessment of the dynamic risk level is a continuous process. The system receives the imaging assessment results, patient-reported outcome indicators and multimodal rehabilitation indicator sequences of each patient follow-up visit, calculates the probability distribution of the risk level through a Transformer-based prognostic prediction model, and automatically triggers the adjustment of the rehabilitation prescription when the probability exceeds a preset threshold.

Images