A position-linked, multimodal, safe feeding system and method for bedridden patients.
By designing a position-linked multimodal safe feeding system, the problems of positional changes and limited safety strategies during feeding for bedridden care recipients were solved. This system enables a safe and reliable mouth-to-mouth feeding process, improves the safety of swallowing monitoring and robotic arm feeding, and provides a data foundation for a layered human-in-the-loop safety mechanism and abnormal handling.
Patent Information
- Application Number
- CN202611234489.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-10
AI Technical Summary
Existing assisted feeding robots are unable to adapt to changes in the body position of bedridden caregivers, resulting in problems such as body slippage, changes in head and face posture, failure to stop changes in mouth target in a timely manner, mismatch between swallowing monitoring and the timing of food delivery by the robotic arm, and insufficient protection due to a single safety strategy.
Design a position-linked multimodal safe feeding system for bedridden caregivers, including a bed and position sensing subsystem, a robotic arm food delivery subsystem, a swallowing/breathing safety subsystem, a voice and caregiver interaction subsystem, an edge control and log subsystem, and a hard-wired emergency stop subsystem. The system uses a mouth-by-mouth safety state machine to achieve linkage between the subsystems, ensuring that each food is delivered only after regaining safety permission, and pausing or exiting when critical conditions fail.
It enables a safe and reliable mouth-to-mouth feeding process for bedridden patients, reduces the risks caused by body slippage and changes in head and face posture, improves the accuracy of swallowing monitoring and the safety of feeding by the robotic arm, forms a layered human-environment safety mechanism, and provides a data foundation for abnormal handling.
Smart Images

Figure CN123060657A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of intelligent health and wellness robots, nursing aids, multimodal physiological signal processing and robot safety control technology, specifically relating to a position-linked multimodal safe feeding system and method for bedridden care recipients. Background Technology
[0002] Existing assisted feeding robots are mainly designed for individuals in a sitting or semi-sitting position, typically using robotic arms, visual positioning, and tableware gripping devices to pick up food and deliver it orally. However, for individuals who are bedridden, lying on their side, or hemiplegic, the body support conditions, head and neck posture, and mouth reach areas differ significantly from those of sitting individuals. Current solutions are still not adequately adapted to the risks of body slippage, head and neck rotation, delivery to the unaffected side of the mouth, and aspiration.
[0003] During feeding, bedridden patients may experience trunk slippage and changes in head and face posture due to changes in bed tilt, insufficient muscle strength, or unstable local support. For hemiplegic bedridden patients, the ability to close the corner of the mouth, sensory feedback, and food retention may also differ between the affected and healthy sides. If a fixed feeding trajectory is followed based solely on a single mouth positioning result, it is easy for the system to fail to stop in time when the relative position between the mouth target and the robotic arm's end effector changes.
[0004] Existing electric nursing beds, turning beds, bed surface pressure sensor matrices, and visual posture recognition devices can perform head-up, tilting, turning, and position monitoring. However, these devices typically serve primarily as posture assistance, pressure monitoring, or nursing prompts, and do not yet require the patient's position to be reached and stabilized in a feeding-ready state as a prerequisite for robotic arm feeding. Furthermore, when the nursing bed, pressure sensing device, and robotic arm operate independently, it is difficult to promptly cancel the ongoing feeding action after the positional permission has expired.
[0005] Current swallowing monitoring can identify swallowing events using RGB-D images, neck movements, respiration, acoustics, or other physiological signals. However, most solutions focus on event detection, offline screening, or nursing-assisted assessment, lacking real-time linkage with robotic arm feeding timing, single-bite volume, feeding time, and next-bite clearance. Detecting only laryngeal movement is not sufficient to indicate that the next bite is ready; a comprehensive judgment is needed, considering post-swallowing respiratory recovery, feedback from the bedridden patient, and signal quality.
[0006] Current robot safety strategies largely rely on body collision detection or a uniform force threshold, failing to adequately differentiate between stages such as feeding, aerial movement, close approach, lip contact, waiting to swallow, and safe withdrawal. The objects of contact, permissible movement forces, and abnormal response methods differ at each stage. Using a single threshold may lead to oversensitivity during the feeding stage or insufficient protection during the close approach stage.
[0007] Whether the robotic arm actually carries food after picking it up from the plate is a crucial condition for entering the mouth approach stage. If the fork fails to pick up food, and there is no food detection mechanism, situations may still occur where an empty fork approaches the mouth, the bedridden patient opens their mouth accidentally, or the caregiver mistakenly believes the food has been delivered. Therefore, it is necessary to confirm the presence of deliverable food on the fork and spoon at both visual and force sensory levels, and to restrict subsequent actions when the detection fails.
[0008] Furthermore, if the bed control, robotic arm control, swallowing monitoring, voice interaction, and caregiver terminal use independent time bases, coordinate relationships, or safety statuses, it may cause a mismatch between sensor information and the current oral frequency, making it difficult to determine whether to pause, retract, softly shut down, or trigger a hard-wired emergency stop when an abnormality occurs.
[0009] Therefore, there is a need for a technical solution that integrates bed positioning, bed pressure sensing, robotic arm food delivery, fork food detection, swallowing and breathing monitoring, patient expression, caregiver confirmation, and independent emergency stop into a single bite-by-bite safety state machine. This would ensure that each bite of food is delivered only after regaining safety permission, and that the delivery is paused or terminated according to a predetermined priority if any critical condition fails. Summary of the Invention
[0010] This invention provides a position-linked multimodal safe feeding system and method for bedridden care recipients, aiming to solve the problems in the prior art such as the disconnect between position adjustment and feeding execution, the lack of mouth-to-mouth linkage between multimodal monitoring and robot actions, the possibility of continuing to feed after a failed fork-feeding attempt, and the unclear levels of abnormal handling.
[0011] To achieve the above objectives, the present invention provides the following technical solution.
[0012] A position-linked multimodal safe feeding system for bedridden care recipients includes a bed and position sensing subsystem, a robotic arm food delivery subsystem, a swallowing / breathing safety subsystem, a voice and caregiver interaction subsystem, an edge control and log subsystem, and a hard-wired emergency stop subsystem. The aforementioned subsystems do not perform monitoring or execution independently, but rather form a continuous permission chain around the single-mouth feeding process. The bed and position sensing subsystem first confirms that the bedridden caregiver 5 is in a position that is suitable for eating and relatively stable; the robotic arm feeding subsystem completes the feeding and delivery after obtaining position permission; the swallowing / breathing safety subsystem determines whether the conditions for the next mouthful are met after the food is delivered; the voice and caregiver interaction subsystem and the hard-wired emergency stop subsystem are used to supplement, reject, or interrupt the automatic permission results; The bed and body position sensing subsystem includes a smart nursing bed 6 that can tilt or turn over, a bed surface pressure monitoring area 7, and a lower controller, which is used to drive the bedridden care subject 5 into a side-lying feeding position, collect the bed surface pressure distribution to calculate the regional pressure, pressure peak, pressure center and left-right pressure ratio, and output a position permission or prohibition signal. The positional authorization is not a single signal indicating that the nursing bed has completed a predetermined angle movement, but rather is formed based on the bed angle, bed surface load distribution, pressure center stability, head and neck posture, and mouth accessibility. The positional authorization is continuously updated during feeding; when the bedridden patient 5 experiences significant slippage, persistent local high pressure, head and neck deviating from the safe range, or mouth leaving the accessible area, the system cancels the positional authorization and stops the current propulsion action.
[0013] The robotic arm feeding subsystem includes a bedside collaborative robotic arm 1, a six-dimensional force / torque sensor 3 connected in series between the robotic arm flange and the self-developed feeding end effector 4, and a wrist depth camera 2. The self-developed feeding end effector 4 is used to perform feeding, feeding, mouth-side hovering, mouth-retraction, and safe exit. The robotic arm employs different motion constraints in the tray area, the air transfer area, and the near-mouth area. The primary goal in the tray area is to achieve stable food retrieval, while the primary goals in the near-mouth area are low speed, low force, and retraction. A wrist depth camera provides near-range geometric correction, and a six-dimensional force / torque sensor provides external contact information after gravity compensation, enabling visual positioning and contact safety to jointly participate in the food delivery process.
[0014] The swallowing / breathing safety subsystem includes an external RGB-D camera 8 and an rPPG breathing processing module. The external RGB-D camera 8 is used to acquire facial, head, neck, and laryngeal movement information; the rPPG breathing processing module is used to estimate the respiratory waveform, respiratory rate, and respiratory phase from the facial RGB signals. The two types of information are fused within a per-oral time window to form a swallowing / breathing safety gate. The swallowing / breathing safety gate does not require a single swallowing event as a sufficient condition for the next swallow. The system simultaneously judges the process of larynx elevation and repositioning, the recovery of respiratory rhythm after swallowing, the quality of image and rPPG signals, and the rejection information of the bedridden care subject 5 or the caregiver, and outputs the status of release, waiting, interruption, or indeterminability.
[0015] The voice and caregiver interaction subsystem includes a microphone, a speaker, and a caregiver terminal, used to issue prompts to the bedridden care subject 5, collect keyword responses such as stop, refusal, not swallowed, or continue, and receive start, pause, single-word release, withdrawal, termination, and takeover instructions from the caregiver.
[0016] The edge control and logging subsystem is used to uniformly receive information from the bed, pressure, vision, force, swallowing, breathing, and the person in the loop. It executes a mouth-to-mouth safety state machine and records body position, food, mouth posture, robotic arm status, contact force, swallowing, breathing, confirmation information, and abnormal handling in single-mouth increments. The hardwired emergency stop subsystem operates independently of the edge controller and cuts off or prohibits motion execution in the event of manual emergency stop, critical communication failure, or failure to meet motion safety conditions.
[0017] The per-mouth safety status machine operates according to the principle of "re-permission for each mouthful". Position permission constitutes a prerequisite for taking and delivering food; food on the fork constitutes a prerequisite for mouth approach; swallowing / breathing safety gate constitutes a prerequisite for the next mouthful; the bedridden patient's call to stop, the caregiver's pause or take over, and hard-line emergency stop can interrupt the current action in any state.
[0018] The bite-by-bit safety status machine includes at least the following states: preparation and self-check, position attainment, feeding permission, feeding, fork-mounted food determination, mouth positioning, force-controlled feeding, swallowing confirmation, next bite or end, and abnormal interruption and withdrawal states. The priority of handling safety events is as follows: hard-line emergency stop and watchdog, caregiver suspension or takeover, bedridden patient's call to stop or refusal, interruption of swallowing / breathing safety gate, excessive end force or collision, fork-mounted food determination failure, mouth positioning failure, position permission failure, and path planning failure.
[0019] The position permission gate is output by turning_node based on pressure distribution, pressure center, bed angle and stable duration, and is a prerequisite for initiating feeding; Food gating on the fork is controlled by feeding_node based on the point cloud difference of the wrist depth camera 2 and the force peak output of the six-dimensional force sensor, preventing an empty fork from approaching the mouth; The swallowing / breathing fusion safety gating is output by swallow_node after aligning and fusing RGB-D laryngeal displacement events with rPPG respiratory phase within a time window, which is used to determine whether to allow the next breath; The voice_node collects the bedridden care recipient's five-keyword responses and the caregiver's terminal command outputs from the human-controlled ring. The hard-wired emergency stop subsystem operates independently of the edge industrial control computer, has the highest safety priority, and independently cuts off motion execution in the event of communication abnormalities, force over-limits, or manual emergency stop.
[0020] The state machine transitions to the next state only when both the entry and duration conditions of the current state are met. For recoverable anomalies, the system maintains or returns to the safe exit position, and rechecks the patient's position permission and other gating controls after confirmation by the caregiver; for critical sensor failures, control link malfunctions, or hardwired emergency stops, the system does not automatically resume feeding.
[0021] The present invention also provides a safe feeding method using the above-described system, comprising the following steps: Step 1: Read the information of the affected or unaffected side, food characteristics and single-serving quantity of the bedridden care subject 5, and check the operating status of each sensing, execution, interaction and emergency stop device; Step 2: Adjust the nursing bed according to the target lateral decubitus direction and head corner, and determine the positional permission based on bed surface pressure, body posture, head and neck status and mouth accessibility; Step 3: Initiate feeding when the patient's position permits, the mouth is visible, the robotic arm is reachable, and the caregiver confirms. Step 4: Perform feeding based on the three-dimensional position of the food and the posture of the utensils, and monitor the contact force during the feeding phase; Step 5: Use a wrist depth camera and a six-dimensional force / torque sensor to determine if food is placed on the fork. If the determination fails, retry or prompt the caregiver. Do not bring an empty fork close to the mouth. Step 6: Based on the coarse facial positioning and close-range mouth positioning, deliver food in stages with varying speed and force limits; Step 7: Move the device back to the observation position, integrate RGB-D laryngeal movement, rPPG respiratory information and the bedridden care subject's 5 responses, and output the status of release, waiting, interruption or indeterminability. Step 8: If any abnormality occurs in body position, force, vision, swallowing, breathing, or in the ring, stop the current propulsion action and perform a retraction from the mouth, soft stop, or hard-line emergency stop according to the movement conditions. Step 9: After the abnormality is resolved, the caregiver confirms whether to continue or stop feeding and completes a mouthful-by-mouth log.
[0022] Compared with the prior art, the present invention has the following beneficial effects.
[0023] The following effects arise from the gating and linkage relationships between the various subsystems, rather than relying on a single sensor or a single algorithm: Firstly, the adjustment of the nursing bed position and the sensing of bed surface pressure are set as prerequisites for feeding by the robotic arm, so that the feeding stage will not begin if the position is not achieved, slippage occurs, or continuous local high pressure occurs.
[0024] Secondly, through the collaboration of a robotic arm, a wrist depth camera, a six-dimensional force / torque sensor, and a self-developed feeding end, it achieves feeding, mouth positioning, phased force control, and abnormal retraction.
[0025] Third, by determining the food on the fork, a closed loop is formed, which includes failure to eat, retrying, prompting, or skipping, reducing the chance of an empty fork approaching the mouth.
[0026] Fourth, RGB-D laryngeal movement is fused with rPPG respiratory information on a breath-by-breath basis, so that swallowing completion and respiratory recovery jointly participate in the permission for the next breath.
[0027] Fifth, a layered human-centered safety mechanism is formed through voice confirmation from the bedridden care recipient, caregiver confirmation, software safety gates, and independent hard-wired emergency stops.
[0028] Sixth, by recording the results of each gate control and the handling of anomalies through individual logs, a data foundation is provided for parameter calibration, safety traceability and subsequent optimization. Attached Figure Description
[0029] Figure 1 This is the overall system architecture diagram of the present invention.
[0030] Figure 2 This invention is based on the data flow, interface, and control closed-loop diagram of ROS 2.
[0031] Figure 3 This is a schematic diagram of the feeding safety status mechanism of the present invention.
[0032] Figure 4 This is a schematic diagram of the actual body position coordination of the nursing bed tilting, pressure matrix and robotic arm support of the present invention.
[0033] Figure 4 The labels indicate the following: 1 is the bedside collaborative robotic arm, namely the Fao ER5 / FR5 feeding robotic arm; 2 is the wrist depth camera, namely the Orbbec 335L depth camera; 3 is the six-dimensional force / torque sensor, namely the Yuli C075E050A-01 six-dimensional force sensor; 4 is the self-developed feeding end effector; 5 is the bedridden care object; 6 is the intelligent nursing bed; 7 is the bed surface pressure monitoring area; and 8 is the external monitoring camera.
[0034] Figure 5 This is a schematic diagram of the feeding robotic arm body and end effector of the present invention.
[0035] Figure 5 The labels indicate the following: 1 is the bedside collaborative robotic arm body, namely the Fao ER5 / FR5 feeding robotic arm; 2 is the wrist depth camera, namely the Orbbec 335L depth camera; 3 is the six-dimensional force / torque sensor, namely the Yuli C075E050A-01 six-dimensional force sensor; 4 is the self-developed feeding end effector; and 12 is the fork-spoon type feeding tool.
[0036] Figure 6 This is a schematic diagram of the quick-change structure of the feeding robotic arm of the present invention.
[0037] Figure 6 The labels indicate: 1 is the bedside collaborative robotic arm body, namely the Fao ER5 / FR5 feeding robotic arm; 2 is the wrist depth camera, namely the Orbbec 335L depth camera; 3 is the six-dimensional force / torque sensor, namely the Yuli C075E050A-01 six-dimensional force sensor; and 4 is the self-developed feeding end effector.
[0038] Figure 7 This is an exploded view of the self-developed end effector of this invention.
[0039] Figure 7 The labels are as follows: 9 is the depth camera mounting flange, 10 is the force sensor mounting flange, 11 is the magnetic flange, and 12 is the fork-spoon type food delivery tool.
[0040] Figure 8 This is the phased force control, force gate threshold, and emergency withdrawal timing diagram of the present invention.
[0041] Figure 9 This is a schematic diagram of the dual-modal fusion safety gate based on RGB-D swallowing monitoring and rPPG respiratory estimation of the present invention.
[0042] Figure 10 This is a flowchart illustrating the port-by-port safety control process of the present invention. Detailed Implementation
[0043] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. These embodiments are used to illustrate the technical solution of the present invention and do not constitute a limitation on the scope of protection. Without changing the functional relationships between the subsystems, the gate-by-gate relationships, and the abnormal handling principles, functionally equivalent devices, algorithms, or control platforms can be used.
[0044] I. System Structure and Connection Relationships like Figure 1 As shown, this embodiment uses an edge industrial control computer as the control center, connecting the nursing bed, bed surface pressure sensor matrix, bedside collaborative robotic arm, wrist depth camera, external RGB-D camera, six-dimensional force / torque sensor, feeding end effector, voice device, caregiver terminal, and hardwired emergency stop circuit into a closed-loop system. The composition and main functions of each subsystem are shown in the table below.
[0045] The nursing bed is positioned beneath the bedridden patient (5) and provides primary postural support; a matrix of bed surface pressure sensors is installed above or integrated into the mattress; a bedside collaborative robotic arm is positioned to the side of the nursing bed, its workspace covering the tray area, mouth delivery area, and mouth-free safety area; an external RGB-D camera is positioned to observe the face and neck. The caregiver terminal is located within the caregiver's operational area, and a hard-wired emergency stop is located within the caregiver's direct reach.
[0046]
[0047] In a preferred embodiment, the nursing bed can be a Xiaocheng Group turning bed or an intelligent nursing bed with equivalent tilting, turning, and status feedback capabilities; the bedside collaborative robotic arm can be a Fao ER5 or a collaborative robotic arm with equivalent degrees of freedom, workspace, and safety interfaces; the wrist depth camera can be an Orbbec 335L or a camera with equivalent RGB-D output capabilities; the six-dimensional force / torque sensor can be a Yuli Instruments C075E050A-01 or an equivalent sensor that meets the requirements for range, accuracy, and communication capabilities. The above models are only used to illustrate feasible structures.
[0048] like Figure 2 As shown, the edge computing system can run position control nodes, feeding control nodes, swallowing monitoring nodes, voice interaction nodes, and system control nodes in a ROS 2 or equivalent middleware environment. All pressure, image, force, robotic arm status, and interaction information uses a unified time reference and is associated with the bed, robotic arm, wrist camera, external camera, tools, mouth target, and throat region through calibrated coordinate transformations. Coordinate relationships can be obtained using existing rigid body registration, hand-eye calibration, PnP, point set registration, and TCP calibration methods. When calibration errors, timestamp validity, or transformation chain integrity do not meet requirements, the system revokes near-surface feeding permission.
[0049] Each module's output is associated with the current oral sequence number and acquisition time. The system control node performs consistency checks on body positioning permission, fork-on food determination, mouth positioning, external contact force, and swallowing / breathing safety gate within the same oral session to avoid using the previous or expired monitoring results. The specific node names, interface types, and communication buses of the software platform can be adjusted according to the implementation environment.
[0050] like Figure 3 As shown, the system operates on a per-entry state machine basis. Swallowing / breathing safety gates, feedback from the bedridden patient, and caregiver confirmation from the previous entry do not automatically carry over to the next entry; relevant gate conditions are rechecked before each feeding or delivery. High-priority safety events can interrupt low-priority actions in any state.
[0051] Under normal circumstances, the state machine sequentially completes the following steps: body position confirmation, food intake, fork-mounted food confirmation, mouth approach, food delivery, observation after removal from mouth, and swallowing confirmation. If a recoverable anomaly occurs, the system stops approaching or delivering food and prioritizes maintaining the current safe position or returning to the safe position after removal from mouth. Once the cause of the anomaly is eliminated, it restarts from the corresponding permitted state, rather than continuing directly from the interrupted position.
[0052] II. Posture Adjustment and Posture Permission The caregiver selects the target lateral decubitus direction and headboard angle based on the affected or unaffected side of the bedridden patient, and then performs head-up, side-tilt, turning, or repositioning maneuvers on the nursing bed. The bed surface pressure sensor matrix, after zero-drift compensation, dead-point shielding, and filtering, obtains the regional average pressure, regional peak pressure, left-right load relationship, high-pressure duration, and pressure center. The regional average pressure and peak pressure are calculated using conventional statistical methods, and the left-right load relationship is determined by the sum of the effective pressures of the left and right effective areas of the bed surface.
[0053] The bed surface area can be divided into head and shoulder area, trunk area, pelvic area, and lower limb area, or other equivalent partitions can be used depending on the actual sensor layout. The system distinguishes between short-term noise, single-point anomalies, transient pressure changes caused by bed movement, and persistent postural anomalies; only after the pressure distribution and bed angle have reached a stable state are the postural characteristics used to establish feeding permission calculated.
[0054]
[0055] In the formula, c CoP The vector representing the position of the center of pressure on the bed surface; p i The pressure value of the i-th effective pressure unit; x i , y i The coordinates of the pressure unit in the bed coordinate system are given; N is the number of effective pressure units. The pressure unit is uniformly Pa or kPa, and the position unit is uniformly m or mm. The dimensions are kept consistent throughout the same calculation process. Equation (1) is used to determine the position of the resultant force of the bed load and serves as the basis for judging whether the lateral recumbent position has entered the target area.
[0056] To avoid judging body position based on a single pressure characteristic, the system also combines changes in the center of pressure, changes in key points of body posture, and changes in bed angle to assess the risk of slippage.
[0057]
[0058] In the formula, I slide Δc is a dimensionless slip index; Δq is the displacement of the pressure center; Δq is the average displacement of the key points of the effective body posture. Δθ b This represents the change in the angle of the bed. d 0、 l 0 and θ 0 represents the corresponding positive normalization scale; w c , w q and w θThe weights are non-negative, and the sum of the three is 1. Equation 2 is used to comprehensively characterize the degree of relative slippage after positional adjustment; only when... I slide Position permission is only granted when the body position falls below a preset threshold and remains stable for a sustained period of time.
[0059] When the head and lateral angles are within preset safe ranges, the pressure distribution matches the target lateral position, localized high pressure does not consistently exceed limits, the pressure center enters the target area, the slippage index is below the threshold, the head and neck posture is suitable for feeding, and the mouth target is within the reach of the robotic arm, the system begins stabilization timing. After the above conditions have been maintained for a preset stabilization period, the system outputs a positional approval. If any condition is not met, the system will not initiate feeding and will instead make low-speed, gentle adjustments based on bed adjustments, caregiver prompts, or under monitoring. The thresholds are determined based on mattress calibration results, the baseline data of the bedridden patient, and their risk level.
[0060] Positioning permission is continuously monitored for effectiveness during feeding and delivery. If only slight but persistent pressure changes occur, the system may pause in a non-proximal safe position and request adjustment; if the mouth target suddenly deviates, the bed continues to move, or the slippage indicator reaches the interruption condition, the feeding permission is immediately revoked. When the robotic arm is used for fine-tuning of position, it is only performed under the condition of caregiver confirmation and low-speed compliance, and is not used as the main support structure of the nursing bed.
[0061] III. Robotic arm food delivery components and external contact forces like Figure 5 As shown, a six-dimensional force / torque sensor 3 is connected in series between the robotic arm flange and the self-developed feeding end effector 4. The feeding end effector includes a depth camera mounting flange 9, a force sensor mounting flange 10, a magnetic flange 11, and a fork-spoon type feeding tool 12.
[0062] The depth camera mounting flange 9 secures the depth camera to the end of the robotic arm, the force sensor mounting flange 10 secures the force sensor to the end of the six-dimensional force / torque sensor 3, and the magnetic flange 11 is fixed to the force sensor with screws. The rear side of the magnetic flange 11 has six tiny magnets that can be magnetically connected to the fork-spoon type feeding tool 12, allowing for quick switching between different types of fork-spoon type feeding tools 12. The back of the fork-spoon type feeding tool 12 has six magnets that match the magnetic flange 11, allowing for quick connection or disconnection. The angle is determined by the pin control between the magnetic flanges 11 and 12. The type of fork-spoon type feeding tool 12 is not limited to a spoon; it can be replaced with a fork or the end of a flexible rehabilitation support, etc. The key is that it can be quickly interchanged with the magnetic flange 11.
[0063] The pose offset of the fork-spoon feeding tool 12 relative to the robotic arm flange is obtained using existing TCP calibration methods. The wrist depth camera 2 is mounted on the wrist or near the end of the arm, and its field of view covers the plate, fork-spoon area, and near-mouth area.
[0064] The quick-change clamping structure is used to replace different types of forks, spoons, or equivalent serving utensils while maintaining the repeatable positioning relationship of the tools after installation. After tool replacement, the system rereads or verifies the corresponding tool parameters.
[0065] The six-dimensional force / torque sensor 3 collects static data under multiple end-effector postures without external contact, and obtains zero bias, equivalent mass, and center of mass through least squares or equivalent parameter identification methods. During operation, the direction of gravity in the sensor coordinate system is determined based on the robot arm's posture, and the external contact force and torque are calculated according to the following formula.
[0066]
[0067] In the formula, W ext (t) represents the compensated external six-dimensional force / torque vector; F ext (t) and M ext (t) represent the external force and the external torque, respectively; F m (t) and M m (t) represents the force and torque measured by the sensor, respectively; b F and b M The corresponding zero bias is denoted by ; m is the equivalent mass of the tools, tableware, and attached food. g s (t) represents the gravitational acceleration in the sensor coordinate system; r c is the position vector of the equivalent centroid relative to the sensor origin. The units of force and torque are N and N·m, respectively. Equation (3) is used to compensate for the zero bias and gravity terms, and obtain the external forces used for the phased contact safety determination.
[0068] The compensated external forces and torques are correlated with the robotic arm's movement phases to determine the contact status during food insertion, aerial movement, near-face approach, lip contact, and safe withdrawal. When the equivalent load changes significantly due to food adhesion, the parameters can be updated from a fixed detection pose.
[0069] Force data can be low-pass filtered, zero-biased corrected, and anomaly suppressed. The filtering delay should be lower than the preset safety response limit to avoid lag in near-surface anomaly detection. The system simultaneously monitors external force amplitude, torque amplitude, rate of change, and duration to distinguish between stable feeding resistance, short-term contact, and continuous abnormal contact. The system does not enter low-force near-surface feeding mode if the gravity compensation result or sensor status is invalid.
[0070] IV. Determining the Food on the Fork The "Food-on-Fork" feature is used to prevent the robotic arm from continuing to approach the mouth after a failed food-taking attempt. Before taking the food, the wrist depth camera 2 records an empty fork reference in a fixed detection pose. After taking the food, it acquires the current depth map and point cloud in the same or pose-compensated detection pose, and obtains the point cloud difference volume, depth residual, and visible food area in the region of interest on the fork teeth or spoon surface. Simultaneously, it reads the peak contact force during the food-taking process. These features are normalized to form the "Food-on-Fork Confidence".
[0071] A fixed detection pose ensures that the viewing angle of the fork and spoon relative to the wrist depth camera 2 remains consistent, which helps suppress differences caused by background point clouds, plate reflections, and robotic arm movements. In cases of point cloud occlusion or insufficient depth quality, the system can reduce the modal weight or output "cannot determine," but it must not directly approach the mouth to feed the food when the state of the food on the fork is unknown.
[0072]
[0073] In the formula, p food σ(·) represents the confidence that food is available for delivery on the fork and spoon; σ(·) is the Sigmoid function. It is a dimensionless feature vector, consisting of normalized point cloud difference volume, depth residual, visible food area and peak contact force during feeding process; β 0 and β are model parameters determined based on labeled samples. Equation (4) is used to fuse visual and force information; only when p food The system only allows the robotic arm to enter the food delivery stage when the preset threshold is reached.
[0074] When the confidence level of the food on the fork is below a threshold, the system does not enter the mouth approach state and performs a re-feeding based on the food characteristics and a preset number of retries; after reaching the maximum number of retries, it skips the current bite or requests assistance from a caregiver. The confidence level model can employ logistic regression, a lightweight classifier, or a judgment method with equivalent input-output relationships.
[0075] During retry, the fork position, tool posture, or insertion depth can be adjusted, but each retry is recorded as a feeding attempt for the current feeding session. Caregivers can choose to resume feeding, adjust the food, switch tools, or end the current feeding session based on the food's form and remaining amount.
[0076] V. Phased Force Control and Abnormal Withdrawal like Figure 8As shown, the system divides the robotic arm's movements into four stages: feeding, aerial movement, near-face approach, lip intake, waiting to swallow, and safe withdrawal. The feeding stage can employ impedance control or force-gated insertion motion; the near-face approach and feeding stages employ low-speed admittance control or equivalent compliant control; and different stages have separate limits set for speed, contact force, rate of change of force, and duration.
[0077] During the feeding phase, the robotic arm is allowed to make necessary contact along the working direction of the tableware, but stops in case of lateral collision or abnormal force changes; during the aerial movement phase, the main focus is on avoiding the bed, the bedridden patient 5, and other obstacles; during the near-face approach phase, the speed is reduced and the oral target is continuously updated; during the lip-holding phase, the insertion depth and axial force are limited; during the waiting-to-swallow phase, the robotic arm does not continue to advance into the mouth; during the safe exit phase, the tableware is kept away from the face.
[0078]
[0079] In the formula, F lim (s,c,u) represents the upper limit of contact force corresponding to the motor stage s, food category c, and risk level u of the bedridden care subject, in N; F hw (s) represents the upper limit of the stages allowed by the hardware and controller; F cal (s,c) is the set of calibration samples for successful operations; F risk (s,u) represents the upper limit set according to the sensitivity and risk level of the oral surface; k s is a non-negative statistical coefficient. Equation (5) takes the minimum value of the hardware upper limit, the calibration statistical upper limit and the risk upper limit to form a contact force safety threshold that varies with the stage of exercise, food type and risk level.
[0080] Lower force limits are used during the near-face approach and lip contact phases; limits during the feeding phase are determined based on food piercing, scooping, or clamping of calibrated samples; and the aerial movement phase simultaneously meets conservative force limits, obstacle spacing, and braking space requirements. Related parameters are not limited to uniform, fixed values.
[0081] If an anomaly occurs but the robotic arm still has the conditions for controlled movement, the system will preferentially perform a low-speed partial retraction along the direction away from the mouth target, and then return to the preset safe position. Controlled retraction must meet the clearance conditions shown in the following formula.
[0082]
[0083] In the formula, d clear is the available clearance distance along the proposed retraction direction; v is the terminal velocity scalar at the time of abnormal triggering; td Total delay for detection, communication, and control; a min This is the absolute value of the minimum deceleration that the system can guarantee; d margin To provide additional safety margin, distance, speed, time, and acceleration are all measured in compatible units. Equation (6) is used to determine whether the robotic arm has the minimum clearance required to complete the controlled exit retraction; if this condition is not met, the system executes a local soft stop or a hard-wired emergency stop.
[0084] When the clearance conditions are met, the coordinate transformation is effective, and the robotic arm controller is normal, the departure retraction is executed; when the controller, force sensor, communication or coordinate information fails, or the clearance is insufficient, a local soft stop is executed; when the hard-wired emergency stop is triggered, the hardware-level stop is directly initiated.
[0085] The initial retraction from the mouth is used to quickly increase the distance between the cutlery and the mouth, followed by returning to the observation position or a safer position further away, depending on the planned scenario. For mouth movement caused by changes in body position, the retraction direction is redefined based on the latest effective mouth position and tool posture; if a reliable direction cannot be obtained, stopping the movement and having a caregiver take over takes priority.
[0086] VI. Swallowing / Breathing Safety Gate like Figure 9 As shown, an external RGB-D camera acquires facial key points, the region of interest (ROI) in the larynx, and depth sequences. The system suppresses overall head movement using existing head and neck rigid body motion compensation methods, extracts larynx elevation, peak, and reset trajectories, and identifies the start and end of swallowing based on these. The rPPG respiratory processing module extracts respiratory modulation information from the facial RGB sequence to obtain respiratory waveforms, respiratory rate, and respiratory phase. The two types of signals are aligned within a sliding time window according to the single-port input time.
[0087] The sliding time window uses the moment of food delivery, removal of utensils, or caregiver confirmation as a reference, covering the pre-swallowing waiting period, laryngeal movement, and post-swallowing respiratory recovery phase. The system evaluates signal quality for facial occlusion, changes in lighting, large head movements, and deep loss; when the quality is insufficient, the missing modality is not misclassified as a normal state.
[0088] Upon first use or recalibration, the system collected baseline resting respiration and swallowed samples confirmed by caregivers from bedridden patients (subject 5). Individualized thresholds for continuous characteristics were determined according to the following formula.
[0089]
[0090] In the formula, τ x + and τ x- These are the individualized upper and lower thresholds for feature x, respectively; x 0 represents the resting baseline sample set or the sample set confirmed by the caregiver; MAD represents the median absolute deviation. k x is a positive threshold coefficient that includes scale correction. Equation (7) is used to establish an individualized decision interval that is insensitive to outliers, and is used to determine the start of swallowing, completion of swallowing, resumption of breathing, and other continuous features.
[0091] The determination of swallowing completion includes at least the completion of larynx elevation-reduction and maintenance in the baseline neighborhood; the determination of respiratory recovery includes at least the satisfaction of signal quality requirements, re-continuity of respiratory rhythm, and return of respiratory rate to the individualized range. Information such as cough probability, decrease in respiratory amplitude, deviation of respiratory rate, sudden head movement, repetitive larynx oscillations, and abnormal end force are used to construct the multimodal interruption risk score.
[0092] Risk fusion is used to determine the robot's action state, rather than to make a medical diagnosis of a single physiological event. When a swallowing signal indicates that it may be completed but breathing has not yet resumed, the system remains inactive; when different modal conclusions contradict each other or the signal quality is below the required level, the system outputs "cannot determine" and requests confirmation from the caregiver; the patient's call to stop, the caregiver's pause, and the hard-line emergency stop are not limited by the risk score.
[0093]
[0094] In the formula, S int Multimodal interruption risk score; z j This represents the j-th risk feature mapped to the interval [0,1]. w j where represents the corresponding non-negative weights, and the sum of all weights is 1; J represents the number of risk features. Therefore, S int The value range is [0,1]. Equation (8) is used to fuse risk information such as cough probability, respiratory abnormalities, sudden head movements, repetitive laryngoscopy, and abnormal terminal forces; when S int Reaching the interruption threshold τ int When a hard trigger condition such as a stop command for bedridden patient 5, pause by caregiver, or hard-wired emergency stop occurs, the system outputs an interrupt status.
[0095]
[0096] Swallowing / breathing safety gates are used for robot movement authorization and risk indication, and do not replace clinical diagnosis. The observation time and thresholds for different bedridden care subjects, food characteristics, and single-serving portions were determined based on baseline samples and caregiver confirmation results.
[0097] VII. Human-centered safety mechanisms and port-by-port logs The voice interaction module issues prompts such as "Prepare to eat," "Open mouth," "Swallow," "Continue," or "Pause" based on the state machine stages. Considering that some bedridden caregivers may have speech impairments, the system uses keywords, wake words, and confidence levels for judgment, rather than relying solely on complete natural language recognition. When the bedridden caregiver expresses "stop," "refuse," or has not yet swallowed, the system pauses the next bite; the caregiver can perform actions such as confirmation, reduction, withdrawal, termination, or takeover.
[0098] Voice prompts and robotic arm movements are sequentially linked according to a state machine to prevent prompting the bedridden care recipient (5) to speak before the robotic arm has reached a safe position. In cases where voice recognition confidence is insufficient, the bedridden care recipient (5) cannot respond, or there is significant environmental noise, the system uses confirmation from the caregiver's terminal as a substitute for permission, and does not presume consent to continue based on no response.
[0099] Hardwired emergency stop and watchdog timer are independent of the host state machine. In case of manual emergency stop, critical control link abnormality, or failure to meet local safety conditions, movement can be prohibited without waiting for a decision from the host machine.
[0100]
[0101] The port-by-port log associates the above data with anonymous IDs and port sequence numbers, and is used for parameter calibration, security event tracing, and subsequent algorithm optimization.
[0102] The log records events in chronological order, including permission creation, permission revocation, action start, action end, and exception handling results. For manual corrections by caregivers, both the original system output and the manual confirmation can be retained for subsequent analysis to determine if gating parameters need adjustment. The log storage method and access permissions are determined based on the data management requirements of the implementation site.
[0103] VIII. Implementation Procedures for Safe Feeding Methods Step 1: The system reads the affected or unaffected side information, food characteristics, and planned single-serving volume of the bedridden patient 5, and performs self-checks on the nursing bed, pressure sensor matrix, robotic arm, wrist depth camera, external RGB-D camera, six-dimensional force / torque sensor, voice device, caregiver terminal, watchdog, and hard-wired emergency stop. The feeding process will not begin if any critical device is offline.
[0104] Step 2: The nursing bed is adjusted according to the target lateral decubitus direction and headboard angle. The system calculates the pressure center shown in Equation 1 and the slip index shown in Equation 2, and combines regional pressure, lateral load, high pressure duration, head and neck posture, and mouth accessibility to form a permissible position.
[0105] Step 3: After the patient's position is valid and confirmed by the caregiver, the system establishes the coordinate relationship between the tray, robotic arm, tools, and mouth target, and plans the movement from the safe position to the feeding position. If the coordinate calibration or time synchronization requirements are not met, the patient is prohibited from entering the feeding or near-face position.
[0106] Step 4: The robotic arm performs feeding based on the food position and the orientation of the utensils, and monitors the feeding process using the external contact force obtained from Equation 3. If feeding abnormalities occur, insertion is stopped and the procedure is replanned or a caregiver is requested to handle the situation.
[0107] Step 5: The robotic arm enters a fixed detection pose and uses equation (4) to obtain the confidence level of the food on the fork. When the confidence level reaches the threshold, it enters mouth positioning; if the threshold is not reached, it retryes. After reaching the preset number of retry attempts, it skips the current bite or prompts the caregiver.
[0108] Step 6: The external RGB-D camera completes coarse positioning of the face and mouth, and the wrist depth camera completes fine positioning at close range. The robotic arm approaches the healthy corner of the mouth or the deliverable mouth position in a slow and compliant manner according to the stage force limit determined by equation (5), and withdraws to the observation position after delivery.
[0109] Step 7: Within the observation position, the system assesses swallowing completion, respiratory recovery, and abnormal risk based on individualized thresholds and multimodal risk scores, while simultaneously receiving responses from the bedridden care subject 5 and commands from the caregiver. The next breath clearance check only begins when the swallowing / breathing safety gate is cleared and other gates are active.
[0110] Step 8: If the following occurs: failure of position permission, loss of mouth target, failure of food determination on fork, excessive external contact force, interruption of swallowing / breathing safety gate, or the bedridden caregiver calling for a stop or the caregiver pausing, the system shall immediately stop the current progress. If formula (6) is met, the mouth retraction shall be performed; if the controlled retraction condition is not met, a soft shutdown shall be performed; if a hard-wired emergency stop is triggered, a hardware-level shutdown shall be performed.
[0111] Step 9: After the abnormality is resolved, the caregiver confirms that the patient has returned to the feeding permission state or the feeding has ended. The system saves a complete log of the current feeding session and reconfirms the patient's position permission, swallowing / breathing safety gate, and person-in-the-loop permission before the next feeding session; after the next feeding session is completed, the food-on-the-fork judgment is re-executed.
[0112] like Figure 10As shown, body position monitoring, force sensation monitoring, swallowing / breathing monitoring, voice monitoring, and emergency stop monitoring can be performed in parallel. The order of the above steps is used to illustrate the main single-channel workflow; any high-priority safety event can interrupt the current step and transition to abnormal handling.
[0113] During continuous feeding, the nursing bed does not need to repeat the full turning action between each bite, but the position must remain valid and be reconfirmed before the next bite. The amount per bite, feeding speed, waiting time, and number of retries can be adjusted according to the food characteristics, the baseline of the bedridden patient, and the caregiver's settings. The adjusted parameters are recorded in the current feeding log.
[0114] IX. Equivalent Implementation Methods This invention does not limit the specific models of nursing beds, robotic arms, depth cameras, six-dimensional force / torque sensors, edge controllers, and communication buses, nor does it limit the specific algorithms for fork-on food determination, swallowing detection, breathing estimation, and risk fusion. Any alternative solution that can achieve the functional relationships between positional authorization, fork-on food gating, phased force safety, next swallow / breath authorization, person-in-the-loop confirmation, abnormal withdrawal, and mouth-to-mouth logging can be used to implement this invention.
Claims
1. A position-linked, multimodal, safe feeding system for bedridden patients, characterized in that, The system includes a bed and body position sensing subsystem, a robotic arm food delivery subsystem, a swallowing / breathing safety subsystem, a voice and caregiver interaction subsystem, an edge control and log subsystem, and a hard-wired emergency stop subsystem. The bed and body position sensing subsystem includes an intelligent nursing bed (6) that can tilt or turn over, a bed surface pressure monitoring area (7) and a lower controller, which are used to drive the bedridden care subject (5) into a side-lying feeding position, collect the bed surface pressure distribution to calculate the regional pressure, pressure peak, pressure center and left-right pressure ratio, and output a position permission or prohibition signal; The robotic arm feeding subsystem includes a bedside collaborative robotic arm (1), a six-dimensional force / torque sensor (3) connected in series between the robotic arm flange and the self-developed feeding end effector (4), and a wrist depth camera (2). The self-developed feeding end effector (4) is used to perform feeding, feeding, mouth-side hovering, mouth-retraction and safe exit. The swallowing / breathing safety subsystem includes an external RGB-D camera (8) and an rPPG breathing processing module, which are used to acquire the three-dimensional displacement of the larynx, key points of the face and head and neck status, and estimate the breathing rhythm from the RGB signals of the face, and output the swallowing / breathing safety gate after fusion and judgment. The voice and caregiver interaction subsystem includes a microphone / speaker and a caregiver terminal, which is used to broadcast prompts, collect keyword responses from bedridden care subjects (5) or caregiver commands, and provide start, pause, single-voice release, retraction, termination and takeover instructions; The edge control and logging subsystem includes an edge industrial control computer and a hardwired emergency stop and watchdog. The edge industrial control computer runs turning_node, feeding_node, swallow_node, voice_node and system_controller in the ROS2 environment. Through unified DOMAIN_ID, unified header.stamp timestamp, unified tf2 coordinate system and shared .msg interface, the monitoring of turning over, feeding and swallowing is serialized into an interpretable, interruptible and traceable mouth-to-mouth safety state machine. The feed-by-feed safety state machine operates on the principle of "re-permission for each feed," ensuring that the feed delivery action must satisfy a combination of the following gating conditions: The position permission gate is output by turning_node based on pressure distribution, pressure center, bed angle and stable duration, and is a prerequisite for initiating feeding; Food gating on the fork is based on the point cloud difference of the wrist depth camera (2) and the force peak output of the six-dimensional force sensor by feeding_node, which prevents an empty fork from approaching the mouth; The swallowing / breathing fusion safety gating is output by swallow_node after aligning and fusing RGB-D laryngeal displacement events with rPPG respiratory phase within a time window, which is used to determine whether to allow the next breath; The voice_node collects the keyword responses of the bedridden care recipient (5) and the terminal command output of the caregiver from the ring gate; The hard-wired emergency stop subsystem operates independently of the edge industrial control computer, has the highest safety priority, and independently cuts off motion execution in the event of communication abnormalities, force over-limits, or manual emergency stop.
2. The safe feeding system according to claim 1, characterized in that, The security priorities of the port-by-port security state machine, from highest to lowest, are as follows: Hardline emergency stop and watchdog, caregiver pause / takeover, bedridden care subject (5) calls for a stop or refuses, interruption of swallowing / breathing safety gating, end force exceeding threshold or collision, failure of fork food determination, insufficient mouth / face positioning reliability, failure of body position permission and failure of normal pathway planning. High-priority events can interrupt low-priority actions in any state, directly entering a pause, rollback, or soft shutdown.
3. The safe feeding system according to claim 1, characterized in that, The self-developed feeding end effector (4) includes a flange adapter, a six-dimensional force sensor mounting base, and a fork-spoon type feeding tool (12); the wrist depth camera (2) is installed above or near the end of the robotic arm wrist, and its optical axis maintains a preset angle with the axis of the fork-spoon type feeding tool, so as to simultaneously cover the plate area, the food area on the fork area and the near mouth area.
4. The safe feeding system according to claim 1, characterized in that, The multiple collaborative nodes running the edge controller communicate through a unified message middleware. All messages carry timestamps. The system realizes real-time transformation between the bed coordinate system, the robot arm base coordinate system, the robot arm flange coordinate system, the end effector coordinate system, the wrist camera coordinate system, the external RGB-D camera coordinate system, the mouth target coordinate system, and the throat ROI coordinate system through a unified coordinate transformation framework.
5. The safe feeding system according to claim 1, characterized in that, The output logic of the swallowing / breathing fusion safety gating is as follows: If swallowing is detected as complete and the respiratory phase is restored after swallowing, the respiratory rate is within a safe range, no abnormal coughing is detected and the force does not exceed the threshold, then the release status is output. If swallowing is not completed or the breathing phase has not yet returned, output a waiting state; If obvious difficulty breathing, coughing, the subject calls for a stop, or the caregiver pauses the process, the output will show an interrupted status. If the image is obstructed or the signal quality is insufficient, the output cannot determine the status and requests confirmation from the caregiver.
6. The safe feeding system according to claim 1, characterized in that, The six-dimensional force / torque sensor (3) is connected in series between the robotic arm flange and the feeding end effector. The system performs gravity compensation on the original six-dimensional force / torque data based on the tool posture, tableware mass and food adhesion mass estimation value to obtain the external contact force. The mouth-to-mouth safety state machine is configured with different control modes, force thresholds and speed limits for the feeding stage, air movement stage, near-face approach stage, lip holding stage, waiting to swallow stage and safe exit stage.
7. The safe feeding system according to claim 1, characterized in that, The method for determining the food gating on the fork is as follows: Before picking up food, record the empty fork point cloud F_0. After picking up food, acquire the current point cloud F_t at a fixed detection pose. Calculate the point cloud difference volume, depth residual, visible food area, and peak contact force during the picking process. Input these values into the classifier and output the food confidence score p_food on the fork. If p_food is greater than the threshold, proceed with feeding. If p_food is less than the threshold, retry the current bite. If N consecutive failures occur, skip the bite and notify the caregiver.
8. The safe feeding system according to claim 1, characterized in that, The determination method for the position permission gating is as follows: Zero drift compensation, bad spot shielding, noise filtering and bed region division are performed sequentially on the bed pressure matrix. The average pressure of the region P_mean(r), the peak pressure of the region P_peak(r), the pressure center CoP and the left-right pressure ratio are calculated. When the CoP trajectory enters the target lateral decubitus area, the left-right pressure ratio is within the set range, the head and neck posture is within the safe window, the duration of local high pressure does not exceed the threshold, and the state is stable for the preset duration, the body position is approved. If any of the following conditions are not met: body position angle, pressure distribution, CoP, slippage index, or oral accessibility, suspend feeding and adjust the bed or alert the caregiver.
9. The safe feeding system according to claim 1, characterized in that, The system also includes a port-by-port logging module, which uses a single port as the smallest recording unit and synchronously records the following field groups: The position and pressure field group includes headboard angle, lateral decubitus angle, pressure matrix summary, CoP, peak pressure, and slip index; The robotic arm and force control field group includes status, pose, speed, contact force, force threshold, and retraction trigger reason; The visual and food field group includes food location, mouth position, confidence of food on fork, number of retries, and food delivery status; The swallowing and breathing field group includes swallowing status, laryngeal displacement, respiratory rate, respiratory phase, and safety gate output; The "person in the loop" field group includes voice keywords, caregiver commands, and pause / takeover records; The exception event field group includes exception code, level, triggering modality, handling action, and recovery conditions.
10. A position-linked, multimodal, safe feeding method for bedridden care recipients, characterized in that, The safe feeding method uses a position-linked multimodal safe feeding system for bedridden care recipients as described in any one of claims 1-9, and the safe feeding method includes the following steps: Step 1: Read the information of the bedridden care subject (5) on the affected / healthy side, food characteristics and single mouth volume, and check whether the bed pressure monitoring area (7), wrist depth camera (2), external RGB-D camera (8), six-dimensional force / torque sensor (3), voice module, caregiver terminal, watchdog and hard-wired emergency stop are online; Step 2: Based on the target lateral decubitus direction and headboard angle selected by the caregiver, drive the nursing bed to perform headboard raising, whole-bed tilting, or turning actions. The turning_node performs zero drift compensation, bad spot shielding, noise filtering, and bed surface area division on the bed surface pressure matrix, and calculates the area average pressure P_mean(r), area peak pressure P_peak(r), pressure center CoP, and left-right pressure ratio. When the CoP trajectory enters the target lateral decubitus area, the left-right pressure ratio is within the set range, the head and neck posture is within the safe window, the duration of local high pressure does not exceed the threshold, and the state is stable for the preset duration, the position is approved; if necessary, the robotic arm will provide compliant support and fine adjustment. Step 3: When the conditions of body position permission, mouth visibility, food temperature / environment status, robotic arm accessibility and caregiver confirmation are all met, the system_controller issues a feeding instruction; Step 4: The feeding_node performs feeding based on the 3D positioning of the food in the plate and the posture of the tableware. During the feeding stage, impedance control or force-gated linear insertion is used to record the peak force and feeding posture. Step 5: Compare the empty fork point cloud F_0 with the current point cloud F_t at a fixed detection pose, calculate the point cloud difference volume, depth residual, visible food area, and peak contact force during the fork picking process, and input the logistic regression or lightweight classifier to output the food confidence score p_food on the fork; if p_food is greater than the threshold, proceed with feeding; if p_food is less than the threshold, retry the current bite; if N consecutive failures occur, skip the bite and notify the caregiver. Step 6: An external RGB-D camera (8) provides coarse facial positioning, and a wrist depth camera (2) performs fine mouth positioning at the mouth. The robotic arm approaches in a low-speed Cartesian straight line and uses admittance control to deliver food to the healthy corner of the mouth or a deliverable mouth position. The near-face / feeding stage uses a lower contact force threshold determined according to the movement stage, food type and the risk level of the bedridden care subject (5). The feeding / forking stage uses a contact force threshold determined according to the calibrated sample of food piercing, scooping or pinching. The remaining movement stages use a contact force threshold that simultaneously meets the hardware upper limit, the calibration statistical upper limit and the risk upper limit. Step 7: After insertion, the robotic arm retracts to the observation position or maintains the safe observation position. The swallow_node merges the RGB-D larynx "lift-reset" trajectory and rPPG respiratory waveform in the sliding window. The voice module collects the response of the bedridden care subject (5) and outputs four types of safety gate status: allow, wait, interrupt or unknown. Step 8: When any high-priority event is triggered, the system will immediately stop the current action and retreat along the safety path to the mouth safety position or soft shutdown when the pressure is abnormal, the force exceeds the threshold, the food on the fork fails to be detected, the mouth target is lost, the swallowing / breathing safety gate is interrupted, the subject calls for a stop or the caregiver pauses. Step 9: After the abnormality is resolved, the nurse confirms and returns to Step 3 or ends the feeding; the system uses a single mouth as the smallest recording unit and simultaneously records body position and pressure, robotic arm and force control, vision and food, swallowing and breathing, person in the loop and abnormal event field groups.
