Postoperative body position monitoring and adjusting method and system based on visual identification

Abstract

The invention discloses a postoperative body position monitoring and adjusting method and system based on visual identification, and belongs to the technical field of medical auxiliary equipment.The monitoring system comprises a multi-view RGB-D camera set and an image analysis module. Key points are identified through a part-by-part optimization algorithm, and parameters such as a prostration angle and a nose tip distance are calculated; an intelligent decision module is configured to predict a risk with an LSTM model and generate an instruction. The matched adjusting system is provided with a multi-partition air bag cushion and a pressure feedback module, and attitude and pressure optimization is achieved through double-closed-loop control. Therefore, the problems that in existing postoperative body position monitoring, wearable equipment is poor in comfort level, weak in visual recognition interference resistance, single in parameter and lack of risk prediction and closed-loop adjustment are solved.

Description

Technical Field

[0001] This invention relates to the field of medical assistive device technology, and in particular to a postoperative positioning monitoring system based on visual recognition, its matching adjustment system, monitoring method, and computer-readable storage medium. Background Technology

[0002] Vitrectomy combined with intraocular tamponade is the main treatment for complex retinal detachment. Postoperatively, patients need to maintain a special position (such as prone) for a long time to utilize the buoyancy or pressure of the tamponade to promote retinal reattachment.

[0003] Currently, postoperative position monitoring technologies are mainly divided into two categories: one is wearable monitoring devices, such as the AI ​​wireless wearable monitoring device developed by Jiangsu Provincial Hospital of Traditional Chinese Medicine (reported by Modern Express on October 14, 2025). These devices collect physiological parameters and position information through sensors attached to the human body. Although they can achieve real-time monitoring, the wearable design can easily cause local pressure on patients, affecting comfort and compliance, and cannot avoid the risk of wire entanglement. The other category is visual recognition monitoring technology, such as the infant rolling over recognition method disclosed in Chinese patent CN118379320A (published on July 23, 2024). This method uses depth image analysis to achieve action recognition in occluded scenarios. However, this technology only targets rolling over actions and does not involve multi-parameter quantitative monitoring of specific positions such as the postoperative prone position. Furthermore, the single-view acquisition method is difficult to fully cover key parts of the human body.

[0004] It is understood that existing visual recognition monitoring technologies generally suffer from three major shortcomings: First, they are poorly adaptable to common postoperative low-light environments (such as nighttime monitoring) and obstructed scenarios (such as bedding coverage), resulting in insufficient accuracy in key point recognition; second, their position monitoring parameters are limited, focusing primarily on posture and movement judgment, lacking precise calculation of quantitative parameters such as prone angle and spatial distances between different body parts; third, they only possess passive monitoring functions, lacking the ability to predict positional deviation risks, and do not form a closed-loop control with the position adjustment execution mechanism, failing to achieve automated connection from monitoring to adjustment. Furthermore, while existing multi-airbag turning mats (Xiaohe Health, April 2, 2025 report) can distribute pressure and assist turning, they lack linkage with precise position monitoring data, relying on manual judgment during the adjustment process, making it difficult to achieve personalized and refined position optimization. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of existing postoperative positioning monitoring technologies, such as poor comfort of wearable devices, weak anti-interference capabilities of visual recognition systems, single monitoring parameters, and lack of predictive adjustment functions. Instead, it provides a non-contact, multi-parameter, accurate monitoring system based on visual recognition with risk prediction capabilities, along with a supporting automated adjustment system, monitoring method, and computer-readable storage medium. To achieve the above objectives, the embodiments of this application disclose the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a postoperative position monitoring system based on visual recognition, including:

[0007] The image acquisition module is used to non-contactly acquire real-time image data and depth information of patients through a multi-view RGB-D camera array;

[0008] The image analysis module, connected to the image acquisition module, is used to perform low-light enhancement and filtering noise reduction on the real-time image data, and to identify key points of the patient's body through a part-based optimization algorithm model, and to calculate parameters representing the body position based on the spatial location information of the key points. The parameters include at least the prone angle θ, the vertical distance D from the tip of the nose to the support surface, the shoulder tilt angle α, and the hip horizontal angle ΔН.

[0009] The part-specific optimization algorithm model is a neural network model trained on a prone dataset containing occlusion samples and low-light samples, and its output is the two-dimensional pixel coordinates and three-dimensional spatial coordinates of key points.

[0010] Optionally, the image acquisition module includes:

[0011] The first RGB-D camera, located at the bottom of the pillow, is used to collect depth information of the facial area.

[0012] The second and third RGB-D cameras, symmetrically positioned on both sides of the hospital bed, are used to capture multi-angle images of the shoulders and hips;

[0013] The fourth RGB-D camera, positioned above the hospital bed, is used to capture a global top-down view image.

[0014] The multi-view camera group achieves synchronized image acquisition through a time synchronization module.

[0015] Optionally, the key point recognition of the image analysis module includes:

[0016] The coordinates of key points are initially obtained using the segmented optimization algorithm model.

[0017] The coordinates of key points are verified based on depth information, and abnormal points whose depth values ​​differ from the support surface by more than a preset threshold are removed.

[0018] Based on the constraints of human anatomy, the three-dimensional coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso.

[0019] Optionally, the system further includes an intelligent decision-making module, which includes:

[0020] The deviation prediction unit is used to calculate the probability P of postural deviation risk based on the time-series data of the prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН using an LSTM prediction model.

[0021] The instruction generation unit is used to generate a predictive adjustment instruction when the risk probability P exceeds a first threshold, and to generate a corrective adjustment instruction when the body position parameters are detected to be outside the safe range.

[0022] Optionally, the instruction generation unit generates instructions based on the following judgment logic:

[0023] when At that time, a maintenance command is generated;

[0024] when At that time, fine-tuning instructions are generated;

[0025] when At that time, a correction instruction is generated;

[0026] Among them, among them, .

[0027] Thirdly, this application provides an adjustment system based on the above-mentioned postoperative positioning monitoring system, which may include a body movement execution module, a pressure feedback module, and a system control module: the body movement execution module includes a multi-zone controllable airbag cushion, the airbag cushion including at least a chest and abdomen support zone, a pelvic support zone, and a headrest zone; the pressure feedback module includes a flexible thin-film pressure sensor array disposed in each zone; the system control module is used to control the body movement execution module to perform body position adjustment according to the positioning parameters output by the image analysis module and the pressure distribution matrix output by the pressure feedback module.

[0028] Optionally, the system control module adopts a dual closed-loop control strategy:

[0029] The first closed loop adjusts the macroscopic posture based on the body position parameters θ, D, α, and ΔН;

[0030] After the body position parameters stabilize, the second closed loop performs micro-pressure optimization based on the pressure distribution matrix.

[0031] Thirdly, embodiments of this application provide a postoperative position monitoring method based on visual recognition, the method comprising the following steps:

[0032] Simultaneous acquisition of patient image data and depth information via a multi-view RGB-D camera array;

[0033] Perform low-light enhancement and filtering / denoising on image data;

[0034] The algorithm model is optimized by different body parts to identify key points and calculate the prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН.

[0035] The key point coordinates are verified and optimized based on the depth information of the key points, including the removal of outliers and the estimation of the coordinates of occluded points through the key points of the torso.

[0036] Optionally, the identification of key body points through a segmented optimization algorithm model includes:

[0037] The coordinates of key points are initially obtained using the segmented optimization algorithm model.

[0038] The coordinates of key points are verified based on depth information, and abnormal points whose depth values ​​differ from the support surface by more than a preset threshold are removed.

[0039] Based on the constraints of human anatomy, the three-dimensional coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso.

[0040] Specifically, the calculation of the three-dimensional coordinates of the obscured head key points includes: taking the midpoint of the line connecting the left and right shoulder key points as the reference point, and calculating the coordinates of the obscured nose tip key point in three-dimensional space according to the preset relative position relationship model between the head and the torso.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the monitoring method described in any of the preceding claims.

[0042] The beneficial effects of this application are as follows:

[0043] (1) Non-contact monitoring improves comfort: The multi-view RGB-D camera group is used to collect data in a non-contact manner, avoiding the local pressure and wire entanglement risks brought by wearable devices. It is in line with the concept of enhanced recovery after surgery and helps to improve patient compliance. This advantage is significantly different from existing wearable monitoring devices.

[0044] (2) Strong anti-interference ability and high recognition accuracy: The part optimization algorithm model is trained on a dataset containing occluded samples and low-light samples. Combined with the multi-view acquisition strategy, it effectively solves the key point recognition problem in postoperative monitoring scenarios such as occlusion by bedding and low light at night. Compared with the single-view visual monitoring system, the recognition stability is significantly improved. At the same time, through depth information verification and human anatomical constraint calculation, abnormal points are further eliminated and occluded points are supplemented to ensure the accuracy of key point coordinates.

[0045] (3) More comprehensive quantitative monitoring of multiple parameters: By calculating core parameters such as prone angle θ and vertical distance D from the tip of the nose, the body position status is quantitatively represented. Compared with existing monitoring technologies that can only judge the movement status, it provides a more accurate basis for body position assessment and provides data support for subsequent adjustments.

[0046] (4) Predictive adjustment reduces risk: By analyzing the time series data of body position parameters through the LSTM prediction model, the probability of deviation risk is calculated in advance, realizing the proactive intervention of "prediction-early warning-adjustment", avoiding the risk of complications that may be caused by traditional passive monitoring; the three-level judgment logic of the instruction generation unit ensures the accurate adaptation of adjustment instructions.

[0047] (5) Closed-loop adjustment system achieves personalized optimization: The matching adjustment system adopts a dual closed-loop control that combines multi-zone airbag cushions and pressure feedback. It ensures that the body position parameters are within the safe range through macro-posture adjustment, and achieves uniform pressure distribution through micro-pressure optimization. It is understood that this structure can effectively reduce the risk of pressure sores. Compared with traditional manual adjustment or single airbag adjustment, it has better adaptability and safety. Attached Figure Description

[0048] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0049] Figure 1 A schematic diagram of the overall structure of a postoperative positioning monitoring and adjustment system based on visual recognition, provided for some embodiments of this application;

[0050] Figure 2 This is a schematic diagram of the camera arrangement of an image acquisition module provided in some embodiments of this application;

[0051] Figure 3 The network structure diagram of the part-based optimization algorithm model is provided for some embodiments of this application;

[0052] Figure 4A schematic diagram of the dual closed-loop control principle of the adjustment system provided in some embodiments of this application;

[0053] Figure 5 A flowchart illustrating the steps of a monitoring method provided in some embodiments of this application. Detailed Implementation

[0054] Specific embodiments of the invention will now be described in detail. Although the invention is described in conjunction with these specific embodiments, it should be understood that it is not intended to limit the invention to these specific embodiments. Rather, these embodiments are intended to cover alternative, modified, or equivalent embodiments that may be included within the spirit and scope of the invention as defined by the claims. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. The invention may be practiced without some or all of these specific details.

[0055] When used in conjunction with the terms "comprising," "method comprising," or similar language in this specification and appended claims, the singular forms "a," "some," and "the" include plural references unless the context clearly indicates otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0056] The following is in conjunction with the appendix Figure 1-5 The preferred embodiments of the present invention will be described in detail so that those skilled in the art can implement the present invention accordingly.

[0057] The postoperative positioning monitoring and adjustment system disclosed in this embodiment has the following overall architecture: Figure 1 As shown, the system includes a monitoring system and an adjustment system. The selection of each module and the parameter settings are as follows:

[0058] The postoperative positioning monitoring and adjustment system includes: an image acquisition module, an image analysis module, an intelligent decision-making module, a body movement execution module, a pressure feedback module, and a system control module.

[0059] The image acquisition module is used to non-contactly acquire real-time image data and depth information of patients through a multi-view RGB-D camera array.

[0060] In one possible embodiment, the image acquisition module employs four RGB-D cameras. The first RGB-D camera is fixed at the bottom of the headrest, 30cm from the patient's face. The second and third RGB-D cameras are symmetrically positioned on either side of the bed, 20cm from the edge and 80cm above the bed. The fourth RGB-D camera is fixed directly above the bed, 150cm above it. Acquisition synchronization is achieved through a time synchronization module, with a frame rate of 60 frames per second and an image resolution of 1280×720. See details... Figure 2The present application provides a schematic diagram of the camera arrangement of an image acquisition module in some embodiments.

[0061] The first RGB-D camera, located at the bottom of the pillow, is used to collect depth information of the facial area. The second and third RGB-D cameras, symmetrically located on both sides of the bed, are used to collect multi-angle images of the shoulders and hips. The fourth RGB-D camera, located above the bed, is used to collect a global top-down view image. The multi-view camera group achieves synchronized image acquisition through a time synchronization module.

[0062] Understandably, a time synchronization module ensures consistent data acquisition from all four cameras, guaranteeing spatial alignment accuracy of multi-view data. Non-contact acquisition avoids disturbing postoperative patients, while the RGB-D camera captures both two-dimensional images and three-dimensional depth information, providing a data foundation for calculating body position parameters.

[0063] The image analysis module, connected to the image acquisition module, is used to perform low-light enhancement and filtering noise reduction on real-time image data. It also identifies key points of the patient's body through a part-specific optimization algorithm model and calculates parameters representing the body position based on the spatial location information of the key points. The parameters include at least the prone angle θ, the vertical distance D from the tip of the nose to the support surface, the shoulder tilt angle α, and the hip horizontal angle ΔН.

[0064] Among them, the part-specific optimization algorithm model is a neural network model trained on a prone dataset containing occluded samples and low-light samples, which outputs the two-dimensional pixel coordinates and three-dimensional spatial coordinates of key points.

[0065] It should be noted that the training data is based on a prone position dataset containing occlusion samples (such as a blanket covering the head, or clothing covering the shoulders and hips) and low-light samples to ensure model robustness. The training dataset includes occlusion samples, low-light samples, and positional variation samples. Occlusion samples cover 10 common clinical occlusion types, such as a blanket partially covering the head (60% coverage) and clothing covering the shoulders and hips (40% coverage). Low-light samples simulate ICU nighttime nursing scenarios (light intensity 5-30 lux), and the keypoint offsets under different lighting conditions are labeled. Positional variation samples contain prone position data of patients of different body types (BMI 18-35), and the relative position parameters of the head, shoulders, and hips are labeled.

[0066] In some possible embodiments, the image analysis module uses an edge computing platform for data processing, low-light enhancement uses the Retinex-Net algorithm, and filtering and denoising use a bilateral filtering algorithm. The part-specific optimization algorithm model is based on the HRNet network architecture, and the training dataset contains 3,000 sets of postoperative prone patient images (including scenes with occlusion by blankets and dim lighting). The key points include 12 core points such as the tip of the nose, left and right shoulders, and left and right hips, and the output coordinate accuracy error is ≤2mm.

[0067] Furthermore, the key point identification in the image analysis module includes: initially obtaining the coordinates of key points through a part-based optimization algorithm model; verifying the coordinates of key points based on depth information and eliminating abnormal points whose depth values ​​differ from the support surface by more than a preset threshold; and calculating the three-dimensional coordinates of the occluded head key points using the coordinates of the torso key points based on the constraints of human anatomy.

[0068] Specifically, the site-specific optimization algorithm model adopts a site-specific detection architecture of a visual Transformer backbone + multi-branch decoder, optimized into three major detection branches for the postoperative prone position scenario. These three branches are the head branch, the shoulder-hip branch, and the global calibration branch. The head branch focuses on key facial bone points (nasal tip, brow ridge, mandibular angle, etc.), leveraging the anatomical proportions of the skull (2 / 3 of head volume) and facial bones (1 / 3 of head volume) to enhance feature extraction of hard tissue key points such as the frontal eminence and nasal bone, while reducing soft tissue deformation interference. The shoulder-hip branch targets four core key points on the left and right shoulders and hips, employing dual-view feature fusion (data from cameras on both sides of the bed) and highlighting skeletal contour features through an attention mechanism, adapting to scenarios where clothing obscures the view. The global calibration branch fuses global trunk features from a top-down camera, performing coordinate correction based on the trunk skeleton (shoulder-hip line) among 33 key points.

[0069] During system initialization, point cloud data of the empty bed status is collected using cameras located at the bottom of the pillow and on both sides of the bed. The RANSAC algorithm is then used to fit the three-dimensional plane equation of the support surface (airbag cushion surface). Where a, b, and c are plane normal vectors, and d is the plane offset.

[0070] The system adopts a dynamic update mechanism, recalibrating every 30 minutes. If the pressure feedback module detects a change in the height of the airbag (such as inflation / deflation adjustment), it triggers immediate calibration to ensure the accuracy of the reference.

[0071] Furthermore, the outlier detection and removal logic employs a three-level verification strategy. The first level involves depth difference filtering, calculating the absolute difference between the depth value of each keypoint and the supporting surface reference. The second level is multi-view consistency verification, comparing the depth values ​​of the same keypoint under different cameras; if the deviation exceeds 5mm (within 2% of the inherent error of the RGB-D camera), it is marked as a suspicious point. The third level is temporal stability judgment, continuously monitoring suspicious points for 5 frames; if the standard deviation of the depth value fluctuation is >3mm, it is determined as an outlier and removed (e.g., background clutter, sensor noise points). Next, a weighted fusion algorithm is used for the remaining valid keypoints, using the depth error variance of each camera as the weight (the smaller the error, the greater the weight), to calculate the weighted average of the 3D coordinates, further reducing the impact of random noise.

[0072] Based on anatomical constraints, the 3D coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso. Specifically, these anatomical constraints include proportional, positional, and angular constraints. The proportional constraint parameter is a head length to shoulder width ratio of 1:1.5, with an individual difference correction factor dynamically adjusted based on BMI. The positional constraint parameter is that the tip of the nose is directly above the midpoint of the line connecting the left and right shoulders, with an individual difference correction factor of ≤20mm offset. The angular constraint parameter is that the angle between the head midline and the torso midline is ≤15°, with an individual difference correction factor specific to the prone position.

[0073] Key points of the left and right shoulders Using this as a reference, the specific process for calculating the key point N(x, y, z) of the nose tip is as follows: First, calculate the coordinates of the reference point. The coordinates of the reference point are... Next, the direction vector is determined. Based on the head-to-torso relative position model, the nose tip pointing vector is: vector MN = (0, 0, k) × rotation matrix R (k is the head length parameter, and R is the rotation matrix based on the prone angle θ). Finally, the coordinates are solved, and combined with the proportional constraints, the final coordinates N are: N = M + vector MN × (head length / shoulder width ratio factor).

[0074] The intelligent decision-making module includes a deviation prediction unit, which calculates the risk probability P of postural deviation using an LSTM prediction model based on time-series data of prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН; and an instruction generation unit, which generates predictive adjustment instructions when the risk probability P exceeds a first threshold and generates corrective adjustment instructions when postural parameters are detected to be outside the safe range.

[0075] The intelligent decision-making module is built on an STM32H743 microcontroller. The deviation prediction unit uses an LSTM model with two hidden layers (64 neurons per layer). It takes 5 minutes of body position parameter time series data as input (time step T=300) and outputs the risk probability P, with a prediction advance response time ≥30 seconds.

[0076] The instruction generation unit generates instructions based on the following judgment logic: when When, generate a sustain command; when When, generate fine-tuning instructions; when At that time, a correction instruction is generated; among which, .

[0077] In one possible embodiment, the parameter threshold is set as needed, and in one implementation, it is set as follows: The LSTM model input time step T = 300 (5 minutes × 60 sampling points / minute), with a first risk probability threshold of 70%. The above angle range, distance threshold, time step, and risk probability threshold can be customized according to different needs.

[0078] Furthermore, the system was adjusted, including the body motion execution module, the pressure feedback module, and the system control module.

[0079] The body movement execution module includes a multi-zone controllable airbag cushion, which includes at least a chest and abdomen support zone, a pelvic support zone, and a headrest zone; a pressure feedback module includes a flexible thin-film pressure sensor array disposed in each zone; and a system control module is used to control the body movement execution module to adjust the body position based on the body position parameters output by the image analysis module and the pressure distribution matrix output by the pressure feedback module.

[0080] Specifically, the multi-zone controllable airbag cushion uses medical-grade TPU material and is divided into three functional zones according to human anatomical structure: a chest and abdomen support zone, a pelvic support zone, and a headrest zone. The headrest zone contains three independent airbags, which adapt to the head's neutral support requirements through differentiated inflation; the chest and abdomen support zone contains four independent airbags, which achieve chest and abdomen suspension through gradient pressure control, reducing abdominal pressure traction on the spine; the pelvic support zone contains six independent airbags, which help maintain the spine's horizontal posture through symmetrical pressure adjustment. The inflation pressure adjustment range for a single airbag is 0.02-0.1 MPa, the inflation rate is 0.005 MPa / s, and the deflation rate is 0.008 MPa / s.

[0081] The pressure feedback module uses the FlexiForce A201 flexible thin-film pressure sensor, which is as thin as paper (thickness < 0.1 mm), highly flexible, and suitable for curved surface contact scenarios. It has a pressure range of 0-1 lb (approximately 0-0.0069 MPa) and an accuracy of ±2%FS. Specifically, a flexible thin-film pressure sensor array is built into each airbag zone, with an array density of one sensor per 10 cm² and a sampling frequency of 10 Hz. This allows for real-time acquisition of zone pressure distribution data, forming a 16×16 pressure distribution matrix.

[0082] The system control module uses an STM32H743 microcontroller, and the adjustment cycle of the dual closed-loop control is 500ms.

[0083] like Figure 4 As shown, the system control module adopts a dual closed-loop control strategy. The first closed loop is a macroscopic attitude adjustment strategy, with body position parameters θ, D, α, and ΔН as the control targets, and the transfer function is... The adjustment cycle is 500ms, quickly pulling the body position back to a safe range. The second closed loop is a micro-pressure optimization strategy, aiming for a pressure distribution variation coefficient CV ≤ 0.2 (meeting the standards for pressure ulcer prevention and care), and employs a model predictive control algorithm (prediction time domain H=5, penalty coefficient λ=0.1) to balance local pressure.

[0084] The specific monitoring and adjustment process for this application is as follows:

[0085] After the system starts up, the time synchronization module triggers four RGB-D cameras to work synchronously, acquiring the patient's RGB images and depth data in real time at a frame rate of 15fps and an image resolution of 1280×720 pixels. The patient's RGB images and depth data are then transmitted to the image analysis module.

[0086] Image processing begins immediately after data transmission is complete. The Retinex-Net algorithm is used, employing a 5-layer convolutional, 2-layer pooling, and 3-layer deconvolutional network to separate reflectivity and illumination maps. Image details are then restored via exponential transformation and normalized to the grayscale range of [0, 255].

[0087] Specifically, the Retinex-Net low-light enhancement algorithm decomposes an image into reflectance R (target features) and illumination map L (ambient lighting) based on Retinex theory. The core formula is:

[0088]

[0089] in, For the input image pixel values, For pixel coordinates, For reflectivity, This is a lighting diagram.

[0090] Then, bilateral filtering is performed on the depth data for noise reduction. The filtered pixel values ​​are calculated according to the formula to remove environmental noise. The specific calculation formula is as follows:

[0091]

[0092] in, These are the filtered pixel values. To normalize the weights, For the spatial domain core, It serves as the kernel for the value range.

[0093] Next, the sub-part optimization algorithm model is called to identify 12 key body points. Abnormal points with a depth value that differs from the bed support surface by more than 5 mm are removed by verifying the depth information. When the key point of the nose tip is detected to be occluded, the three-dimensional coordinates of the nose tip are calculated based on the midpoint of the line connecting the key points of the left and right shoulders and the anatomical ratio of the head and torso (the preset ratio is 1:7).

[0094] Specifically, the input layer of the part-based optimization algorithm model concatenates four RGB images (1280×720×3) and four depth images (1280×720×1) to form a 1280×720×16 tensor. After cross-resolution feature fusion via the high-resolution branch, the head sub-network extracts features using three improved dense units (deep attention mechanism) and the body sub-network (residual units + spatial pyramid pooling). The model output layer outputs 12 keypoint heatmaps, which are used to obtain two-dimensional coordinates (u, v) through sub-pixel interpolation. These coordinates are then combined with the depth Z to convert to three-dimensional coordinates using the following formula:

[0095]

[0096]

[0097] in, , Center of the image , This refers to the camera's focal length.

[0098] The initially obtained 3D coordinate points need to be optimized. Specifically, based on depth continuity verification, discrete key points with depth values ​​differing from the support surface by more than 5mm are removed. When the confidence level of the nose tip point is <0.5, the midpoint of the line connecting the left and right shoulder key points is used as the reference, and the coordinates are optimized according to the anatomical ratio of the head to the torso (preset ratio 1:7) using the formula... , , Calculate the three-dimensional coordinates to ensure the integrity of the parameters.

[0099] Furthermore, θ, D, α, and ΔН are calculated based on three-dimensional coordinates. Wherein, θ is the angle between the trunk midline and the support surface, D is the vertical distance from the tip of the nose to the support surface, α is the angle between the line connecting the two acromions and the horizontal line, and ΔН is the height difference between the two anterior superior iliac spines.

[0100] Specifically, the vertical distance from the tip of the nose to the supporting surface is the point-to-plane distance. This point-to-plane distance formula is used based on the plane equation of the supporting surface. Conclusion:

[0101]

[0102] in, The vertical distance from the tip of the nose to the supporting surface. Let a, b, and c be the three-dimensional coordinates of the tip of the nose, and a, b, and c be the plane normal vectors.

[0103] θ is the angle between the trunk's central axis and the supporting surface. The trunk's central axis is usually defined by the vector of the line connecting the midpoints of the shoulder and hip. The calculation steps are as follows: First, calculate the midpoints of the shoulder and hip using the three-dimensional coordinates of the shoulder and hip. Then, calculate the vector of the trunk's central axis by subtracting the coordinates of the midpoint of the hip from the coordinates of the midpoint of the shoulder. Next, calculate the angle between the vector of the central axis and the unit normal vector. Since the angle between the vector of the central axis and the unit normal vector is complementary to the angle θ between the trunk's central axis and the supporting surface, the angle θ between the trunk's central axis and the supporting surface can be calculated.

[0104] α is the angle between the line connecting the two acromions and the horizontal line. The horizontal line refers to the horizontal direction of the support surface (i.e., the horizontal axis within the plane of the support surface), or the absolute horizontal plane (such as the xy plane in world coordinates). The calculation steps are as follows: First, calculate the vector of the acromion connection; then, ignoring the Z-axis variation, obtain the horizontal acromion vector; then calculate the angle between the horizontal acromion vector and the x-axis (or the reference horizontal direction).

[0105] ΔН represents the height difference between the key points of the bilateral anterior superior iliac spines, which is directly the Z-coordinate difference, i.e., the difference between the left and right hips on the Z-axis.

[0106] For the calculated body position parameters, the LSTM prediction model performs sliding window analysis on the parameter time series data of the past 5 minutes, dynamically updates the cell state through forget gate, input gate and output gate, and outputs the probability P of body position deviation risk in real time.

[0107] Next, based on the parameter thresholds and risk probabilities, the three-level instruction logic is executed:

[0108] when When the airbag is in use, a maintenance command is generated to control the airbag to maintain the current pressure and automatically compensate for leakage of 0.002 MPa every 5 minutes.

[0109] when Or, when P ≥ 70%, a fine-tuning instruction is generated, and the adjustment amount is calculated according to the adjustment formula, which is:

[0110]

[0111] in, This is the airbag pressure adjustment amount. This is the proportionality coefficient. For deviation of body position parameters, The integral coefficient is... This represents the cumulative deviation over the past 5 minutes. A positive adjustment corresponds to an inflation command; a negative adjustment corresponds to a deflation command. The proportional coefficients are determined as follows: θ / α is 0.003 MPa / °; D is 0.002 MPa / cm; ΔН is 0.005 MPa / mm. The integral coefficient is a fixed value of 0.001.

[0112] A correction command is generated when θ∈[15°, 30°] or D<10cm or α>10° ​​or ΔН>5mm, and the target pressure is calculated according to the zoning formula. The target pressure consists of the reference pressure and the product of the coefficient and the parameter deviation.

[0113] Specifically,

[0114]

[0115]

[0116] Among them, 0.03MPa is the minimum support pressure for the head, and 0.05MPa is the basic suspension pressure for the chest and abdomen.

[0117] After the system receives the command, the system control module starts dual closed-loop control: the first closed loop calculates the target inflation volume of the airbags in each area based on the body position parameters, controls the inflation or deflation of the airbags, realizes macroscopic posture adjustment, and brings the parameters back to a safe range; the second closed loop, after the parameters stabilize, fine-tunes the pressure of each airbag based on the pressure distribution matrix collected by the pressure sensor, so that the pressure at the body contact points is evenly distributed and avoids local high pressure.

[0118] In one possible embodiment, the first closed loop drives the airbag to inflate or deflate, bringing the body position parameters back to a safe range. Once the parameter deviation is less than 0.5, the second closed loop is triggered. The airbag pressure is then finely adjusted by model prediction control until the pressure distribution variation coefficient CV is less than or equal to 0.2, thus completing the entire process of "posture correction - pressure equalization".

[0119] like Figure 5 As shown, the postoperative position monitoring method based on visual recognition includes the following steps:

[0120] Simultaneous acquisition of patient image data and depth information via a multi-view RGB-D camera array;

[0121] Perform low-light enhancement and filtering / denoising on image data;

[0122] The algorithm model is optimized by different body parts to identify key points and calculate the prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН.

[0123] The key point coordinates are verified and optimized based on the depth information of the key points, including the removal of outliers and the estimation of the coordinates of occluded points through the key points of the torso.

[0124] Furthermore, the identification of key body points through a segmented optimization algorithm model includes:

[0125] The coordinates of key points were initially obtained through a component-based optimization algorithm model.

[0126] The coordinates of key points are verified based on depth information, and abnormal points whose depth values ​​differ from the support surface by more than a preset threshold are removed.

[0127] Based on the constraints of human anatomy, the three-dimensional coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso.

[0128] Specifically, calculating the three-dimensional coordinates of the obscured head key points includes: taking the midpoint of the line connecting the left and right shoulder key points as the reference point, and calculating the coordinates of the obscured nose tip key point in three-dimensional space according to the preset relative position relationship model between the head and torso.

[0129] It should be noted that the foregoing explanation of the embodiment of the postoperative positioning monitoring and adjustment system based on visual recognition also applies to the postoperative positioning monitoring method based on visual recognition in this embodiment, and will not be repeated here.

[0130] The computer-readable storage medium uses an SD card or a solid-state drive. The stored computer program includes six functional modules: a data acquisition module, an image processing module, a parameter calculation module, a risk prediction module, an instruction generation module, and a closed-loop control module. When the processor executes the program, it sequentially calls each module to implement the above monitoring and adjustment process.

[0131] The data acquisition module encapsulates the camera control interface and synchronization protocol to enable collaborative acquisition and data transmission among multiple devices; the image processing module integrates the Retinex-Net enhancement algorithm and bilateral filtering algorithm to provide a standardized image processing interface; the parameter calculation module incorporates the HRNet model inference engine and geometric parameter solution logic to output body position parameters; the risk prediction module deploys an LSTM pre-trained model to achieve time-series data analysis and risk probability output; the instruction generation module solidifies the three-level instruction judgment logic to generate control instructions based on parameters and risk values; and the closed-loop control module implements a dual closed-loop control algorithm to output airbag pressure adjustment signals.

[0132] When the processor executes the program, it calls the above modules in the order of initialization → data acquisition → image processing → parameter calculation → risk prediction → instruction generation → closed-loop control, and strictly follows the above process to complete monitoring and adjustment.

[0133] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention. The actual method is not limited to this. In conclusion, if those skilled in the art are inspired by this description and design similar methods and embodiments without departing from the spirit of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A postoperative position monitoring system based on visual recognition, characterized in that, include: The image acquisition module is used to non-contactly acquire real-time image data and depth information of patients through a multi-view RGB-D camera array; The image analysis module, connected to the image acquisition module, is used to perform low-light enhancement and filtering noise reduction on the real-time image data, and to identify key points of the patient's body through a part-based optimization algorithm model, and to calculate parameters representing the body position based on the spatial location information of the key points. The parameters include at least the prone angle θ, the vertical distance D from the tip of the nose to the support surface, the shoulder tilt angle α, and the hip horizontal angle ΔН. The part-specific optimization algorithm model is a neural network model trained on a prone dataset containing occlusion samples and low-light samples, and its output is the two-dimensional pixel coordinates and three-dimensional spatial coordinates of key points.

2. The system according to claim 1, characterized in that, The image acquisition module includes: The first RGB-D camera, located at the bottom of the pillow, is used to collect depth information of the facial area. The second and third RGB-D cameras, symmetrically positioned on both sides of the hospital bed, are used to capture multi-angle images of the shoulders and hips; The fourth RGB-D camera, positioned above the hospital bed, is used to capture a global top-down view image. The multi-view camera group achieves synchronized image acquisition through a time synchronization module.

3. The system according to claim 1, characterized in that, The key point identification of the image analysis module includes: The coordinates of key points are initially obtained using the segmented optimization algorithm model. The coordinates of key points are verified based on depth information, and abnormal points whose depth values ​​differ from the support surface by more than a preset threshold are removed. Based on the constraints of human anatomy, the three-dimensional coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso.

4. The system according to claim 1, characterized in that, The system also includes an intelligent decision-making module, which includes: The deviation prediction unit is used to calculate the probability P of postural deviation risk based on the time-series data of the prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН using an LSTM prediction model. The instruction generation unit is used to generate a predictive adjustment instruction when the risk probability P exceeds a first threshold, and to generate a corrective adjustment instruction when the body position parameters are detected to be outside the safe range.

5. The system according to claim 4, characterized in that, The instruction generation unit generates instructions based on the following judgment logic: when At that time, a maintenance command is generated; when At that time, fine-tuning instructions are generated; when At that time, a correction instruction is generated; Among them, among them, .

6. An adjustment system based on the postoperative position monitoring system based on visual recognition as described in any one of claims 1-5, characterized in that, include: The body movement execution module includes a multi-zone controllable airbag cushion, which includes at least a chest and abdomen support area, a pelvic support area, and a headrest area. The pressure feedback module includes an array of flexible thin-film pressure sensors located in each zone; The system control module is used to control the body movement execution module to adjust the body position based on the body position parameters output by the image analysis module and the pressure distribution matrix output by the pressure feedback module.

7. The system according to claim 6, characterized in that, The system control module adopts a dual closed-loop control strategy: The first closed loop adjusts the macroscopic posture based on the body position parameters θ, D, α, and ΔН; After the body position parameters stabilize, the second closed loop performs micro-pressure optimization based on the pressure distribution matrix.

8. A postoperative position monitoring method based on visual recognition, characterized in that, The method includes the following steps: Simultaneous acquisition of patient image data and depth information via a multi-view RGB-D camera array; Perform low-light enhancement and filtering / denoising on image data; The algorithm model is optimized by different body parts to identify key points and calculate the prone angle θ, vertical distance D, shoulder tilt angle α, and hip horizontal angle ΔН. The key point coordinates are verified and optimized based on the depth information of the key points, including the removal of outliers and the estimation of the coordinates of occluded points through the key points of the torso.

9. The method according to claim 8, characterized in that, The identification of key body points through a segmented optimization algorithm model includes: The coordinates of key points are initially obtained using the segmented optimization algorithm model. The coordinates of key points are verified based on depth information, and abnormal points whose depth values ​​differ from the support surface by more than a preset threshold are removed. Based on the constraints of human anatomy, the three-dimensional coordinates of key points on the obscured head are calculated using the coordinates of key points on the torso. Specifically, the calculation of the three-dimensional coordinates of the obscured head key points includes: taking the midpoint of the line connecting the left and right shoulder key points as the reference point, and calculating the coordinates of the obscured nose tip key point in three-dimensional space according to the preset relative position relationship model between the head and the torso.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 8-9.

Images