Urinary surgery nursing monitoring method combining Internet of Things and image recognition

Abstract

The invention belongs to the technical field of medical care monitoring, and particularly relates to a urinary surgery nursing monitoring method combining the Internet of Things and image recognition. Acquiring multi-source monitoring data related to urinary surgery patient nursing; carrying out preprocessing and region segmentation on the acquired image data; calculating the real-time urine volume by adopting piecewise linear mapping; constructing a urine volume time sequence, predicting a future urine volume trend by adopting an LSTM model, and performing anomaly identification through prediction errors; a state area of the catheter is identified by adopting a convolutional neural network identification model, leakage detection is carried out, the color of urine is analyzed, and a leakage risk index is calculated; and multi-modal input is constructed, and a comprehensive risk index is output. According to the method, the Internet of Things technology and the image recognition technology are combined to be used for urinary surgery nursing monitoring, and intelligent monitoring and automatic early warning of the urine bag liquid level, catheter bending, leakage diffusion, urine color change, the urine volume trend and the patient body movement risk are achieved.

Description

Technical Field

[0001] This invention relates to the field of medical and nursing monitoring technology, specifically to a method for monitoring urological nursing care that combines the Internet of Things and image recognition. Background Technology

[0002] During treatments such as catheterization, bladder irrigation, cystostomy, and nephrostomy, urology patients require real-time monitoring of urine volume, urine color, catheter morphology, urine bag level, leakage, and patient position. Traditional nursing methods mainly rely on manual rounds, assessing the situation by visually observing urine bag markings, catheter folding, and whether the urine bag is full. However, manual monitoring has inherent drawbacks such as low frequency of rounds, poor real-time performance, and discontinuous data, making it difficult to detect nursing risks such as urine bag reflux, catheter obstruction, urine leakage, and dislodgement in a timely manner.

[0003] Some medical institutions have begun to use devices such as electronic urine meters, simple pressure sensors, and infrared liquid level detectors for basic monitoring. However, the data they collect is limited in scope, cannot automatically identify visual scenes, and cannot make comprehensive judgments about the condition of the tubing and the patient.

[0004] Existing IoT devices generally lack cross-terminal data fusion capabilities, making it impossible to form a unified risk assessment logic. Image recognition technology is not yet mature in areas such as urine bag scale recognition, urine volume change analysis, and catheter abnormality recognition, making it difficult to adapt to the complex lighting and environmental conditions in hospitals. The lack of a fusion and analysis mechanism for multi-source data makes abnormality identification and early warning inaccurate and fails to meet the construction needs of a smart nursing system for urology. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a urological nursing monitoring method that combines the Internet of Things and image recognition, so as to solve the technical problems existing in the current urological nursing monitoring, such as insufficient real-time performance, limited monitoring content, weak image recognition capability, and failure to integrate and analyze multi-source data.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method for monitoring urological nursing care by combining the Internet of Things and image recognition, comprising the following steps: Step S1: Acquire multi-source monitoring data for urological patient care. The multi-source monitoring data includes IoT sensor data, image data, and behavioral action data. Upload the monitoring data to the nursing monitoring terminal via a wireless communication module. Step S2: Preprocess the acquired image data using an adaptive histogram equalization algorithm to obtain preprocessed image data. The image data is segmented using a detection model, and the following parameters are output: ; in: For detection model functions; This is the preprocessed image data; This refers to the entire area of ​​the urine bag; This is the area marked with scale lines; This is the liquid level line area; This is the pipeline status area; Areas suspected of leakage; Step S3: Apply piecewise linear mapping to the scale line region and the liquid level line region to calculate the real-time urine volume; Step S4: Construct urine volume time series The LSTM model is used to predict future urine output trends, and anomalies are identified based on the prediction error. Step S5: Use a convolutional neural network recognition model to identify the pipeline status region and output the pipeline status region parameters. : in: This refers to the bend angle of the pipeline. The degree of pressure; The interface loosening index; Step S6: Conduct leakage detection on suspected leakage areas; perform urine color analysis on the liquid level line area and calculate the leakage risk index; Step S7: Construct multimodal inputs and use them as inputs to the fusion model; the fusion model outputs a comprehensive risk index to determine the patient's comprehensive nursing risk.

[0007] Preferably, the IoT sensor data It is expressed as follows: in: This refers to real-time urine flow rate; This refers to the weight of the urine bag; For pipeline pressure; Local temperature; Ambient light intensity; The image data is captured by a high-definition camera, which is positioned on the outside of the urine bag, in the middle of the catheter, and at bedside locations where leakage is common. It is expressed as follows: in: Outline of the urine bag's graduated area; This is the area representing the morphology of a urinary catheter; Areas where leakage is possible; The behavioral action data is identified using a lightweight LSTM behavior detection model. The input data sources for the lightweight LSTM behavior detection model are millimeter-wave radar and IMU. The millimeter-wave radar collects human micro-motion signals, and the IMU collects acceleration and angular velocity data. The patient's movements include turning over, getting out of bed, and large-amplitude traction movements.

[0008] Preferably, the piecewise linear mapping is performed based on the liquid level height and the scale segment; The liquid level height is extracted from the liquid level line region using a liquid level segmentation model. ; The scale segment is extracted from the scale line region using Hough line transform and OCR model; The formula for calculating real-time urine volume is as follows: in: This refers to real-time urine output; and is the conversion factor for the i-th scale segment; This represents the liquid level height boundary corresponding to the i-th scale segment.

[0009] Preferably, the urine volume time series is constructed by collecting urine volumes from the past n time points, and the anomaly identification satisfies the following formula: in: For the prediction of the LSTM model Urine volume at any given time; For prediction error, when At that time, it was automatically determined to be an abnormal urine output. The threshold for determining abnormal urine output.

[0010] Preferably, the pipeline condition risk determination rule in step S5 is as follows: At that time, it was determined to be a risk of bending; At that time, it was determined to be under pressure risk; At that time, it was determined to be a risk of interface detachment; in: This refers to the threshold for pipe bending. The pressure threshold; This is the threshold for the interface detachment index.

[0011] Preferably, the leakage detection process in step S6 is as follows: A regional growth model was used to analyze the suspected leakage areas. Extract the leakage area ; The leakage diffusion rate is calculated based on the leakage area, using the following formula: Determine whether the leakage is spreading based on the leakage diffusion rate; in: Let be the leakage area at time t; For time intervals; For the leakage diffusion rate, when It was determined to be penetration diffusion at that time. This is the threshold for the penetration diffusion rate; The HSV color analysis model obtains color analysis results by analyzing the liquid level line area; based on the color analysis results, the degree of color deviation is calculated using the following formula: in: For HSV color analysis model functions; As a real-time urine color reference; This serves as a baseline for normal urine color. The degree of deviation in urine color; The formula for calculating the leakage risk index is as follows: in: This is the weighting coefficient for the leakage area; This is a weighting coefficient for the degree of deviation in urine color.

[0012] Preferably, the multimodal input It is expressed as follows: in: This refers to real-time urine output; This represents the prediction error; These are parameters for the pipeline status region; This is the leakage risk index; For pipeline pressure; This refers to the weight of the urine bag; The calculation process of the fusion model is as follows: S701: For multimodal input Perform feature normalization to obtain normalized input. ; S702: Introducing the basic weight vector The normalized input is weighted and fused to obtain a fusion vector. ; S703: Input the fusion vector into the sigmoid function to obtain the comprehensive risk index. , ;when Automatic alarm triggered at any time; This is the warning threshold.

[0013] Preferably, the process for assessing the comprehensive nursing risk is as follows: When the comprehensive risk index exceeds the warning threshold, the nursing system automatically generates a warning package and a patient care report. The warning package is as follows: in: Risk category; This is a trend chart for a recent period of time; The warning package is pushed to the nurse station monitoring terminal, mobile nursing APP and bedside interactive screen.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: The method of this invention uses a unified data acquisition framework to simultaneously acquire IoT sensor data, image data, and behavioral data, expanding nursing monitoring from a single indicator to multi-parameter comprehensive monitoring. It achieves the fusion of multi-dimensional data such as urine volume, pressure, tubing status, fluid level image, scale image, leakage image, urine color, and behavioral data, significantly improving the completeness of monitoring.

[0015] The method of this invention provides high-precision image recognition and scale mapping to achieve non-contact, high-precision urine volume calculation without the need for manual reading by nursing staff. Based on the LSTM model, it realizes urine volume trend analysis to identify risks such as anuria, oliguria, and sudden increases in urine volume in advance, thus achieving early warning capabilities. By using regional growth model detection and HSV color analysis for leakage detection and color recognition, it automatically monitors hidden leaks and color changes, avoiding the risk of infection caused by minor leaks or color changes. By automatically identifying tubing bends, pressure, and loosening, it realizes intelligent tubing monitoring, which greatly reduces the incidence of nursing events such as catheter dislodgement, reflux, and blockage.

[0016] The method of this invention uses a multimodal fusion algorithm to fuse multi-source data, avoiding misjudgment of single parameters and significantly improving the accuracy of comprehensive risk assessment. When the comprehensive risk index triggers the warning threshold, the nursing system automatically generates a warning package and pushes it, and automatically writes all monitoring data, calculated indicators and warning content into the electronic nursing record, eliminating human recording errors and realizing automatic recording and system linkage. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the overall monitoring method of the present invention; Figure 2 This is a flowchart of the data acquisition process of the present invention; Figure 3 This is a flowchart of the real-time urine volume calculation process using the urine bag in the method of this invention; Figure 4 This is a flowchart of the urine volume trend analysis method of the present invention; Figure 5 This is a flowchart of the leakage area detection method of the present invention; Figure 6 This is a flowchart of the multimodal data fusion early warning push process of the method of the present invention. Detailed Implementation

[0019] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0020] Reference Figures 1-6 As shown, the present invention provides a urological nursing monitoring method combining the Internet of Things and image recognition, comprising the following steps: Step S1: Acquire multi-source monitoring data related to the care of urological patients. The multi-source monitoring data includes IoT sensor data, image data, and behavioral action data. Upload the monitoring data to the nursing monitoring terminal through the wireless communication module. The IoT sensor data is collected through IoT sensors, including urine flow rate sensors, weight sensors, catheter pressure sensors, multi-point temperature sensors, and ambient light sensors, used to collect real-time data related to the nursing scenario and generate IoT sensor data. It is expressed as follows: in: Real-time urine flow rate (mL / min) reflects the rate of urine excretion in real time and is used to determine the trend of urine volume changes. The urine bag weight (g) is used to correct the urine volume calculation results obtained from image recognition and improve the reliability of the final number. The pipeline pressure (kPa) is used to identify risks of pipeline pressure, blockage, or backflow. This is the local temperature (°C), used to investigate special situations such as thermal leakage; The ambient light intensity (Lux) is used to compensate the image recognition module and improve the recognition accuracy under complex lighting conditions. The collected IoT sensor data is preprocessed using Kalman filtering for noise reduction and normalization fusion to output a structured sequence. in: For fusion preprocessing functions; This is a Kalman filter noise reduction function; For normalization processing; The image data was acquired using a high-definition camera, which was positioned on the outside of the urine bag, in the middle of the catheter, and at bedside locations where leakage was common, to obtain image data from these three areas. It is expressed as follows: in: The outline of the urine bag's graduated area is used for identifying liquid level, graduations, and urine color; This area shows the shape of the urinary catheter, used to identify bends, compression, and dislodgement of the catheter. This area is designated as a potential leakage zone, used to identify the extent and spread of leakage. The high-definition camera has the following configuration parameters: Frame rate: Adjustable from 5 to 10 fps Minimum illuminance: 0.05 lux Lens focal length: 3.6mm Fixed position: 20–30 cm from the marked area on the urine bag Visual coverage area: urine bag marking area, and 10–15 cm length range of the catheter. The high-definition camera is equipped with an infrared fill light to ensure nighttime recognition capabilities; Automatic exposure compensation, illumination normalization, and noise suppression are sequentially applied to the image sequence captured by the high-definition camera to obtain image data. The behavioral data is identified using a lightweight LSTM behavior detection model that combines millimeter-wave radar and IMU. The millimeter-wave radar collects micro-motion signals of the human body (distance, speed, angle), and the IMU collects acceleration and angular velocity data. The behavioral actions include turning over, getting out of bed, and large-amplitude traction. In some embodiments, the structure of the lightweight LSTM behavior detection model is as follows: The model first synchronizes and standardizes heterogeneous data. For the three-dimensional micro-motion signals of distance, velocity, and angle acquired by millimeter-wave radar, a moving average filter is used for lightweight denoising to reduce the noise impact of environmental interference. At the same time, gravity separation and drift correction are performed on the three-dimensional acceleration and three-dimensional angular velocity data output by IMU. Both types of data are normalized by Z-Score to eliminate dimensional differences. Then, time-series segments are segmented with a window size of 100ms and a step size of 50ms to preserve the temporal continuity of the behavior and control the dimensionality of the input data to reduce computational overhead. The preprocessed radar data and IMU data are respectively entered into parallel feature extraction branches. Each branch uses 1D depthwise separable convolutions instead of traditional convolutions. The number of parameters is greatly reduced by splitting grouped convolutions and pointwise convolutions. Each branch has only two convolutional layers. The first layer uses 8 convolutional kernels of size 3 to extract local features. The second layer expands to 16 convolutional kernels and is equipped with a BatchNorm layer to accelerate convergence, avoiding feature redundancy and improving computational efficiency.

[0021] A concatenation strategy is employed to integrate the output features of the two branches into a feature vector of a unified dimension, preserving the unique features of each sensor while achieving complementarity of heterogeneous information. The fused features are fed into a lightweight temporal modeling core. Considering that GRU reduces the number of parameters by 25% compared to traditional LSTM while maintaining comparable temporal modeling capabilities, GRU units are prioritized for constructing the temporal layers. The first layer uses 64 hidden units to capture long-term dependencies, while the second layer halves the number of hidden units to 32 to further reduce computational load. A LayerNorm layer is inserted between the two layers to stabilize the training process, and a Dropout layer of 0.25 is added to suppress overfitting without increasing the computational burden of the inference stage. After temporal modeling, a global average pooling layer is used to compress the temporal dimension, transforming the variable-length temporal features into fixed-length vectors, avoiding parameter inflation caused by fully connected layers while preserving key temporal information.

[0022] The classification head adopts a minimalist design, with only two fully connected layers. The first layer uses 32 neurons and the ReLU6 activation function to adapt to the computational efficiency requirements of mobile deployment. The second layer directly maps to three neurons corresponding to three behavior categories, and outputs the probability of each category through the Softmax function. To further enhance its lightweight characteristics, the model avoids complex calculations at the junctions of each module. Both convolutional and fully connected layers use low-precision parameter initialization strategies, while also being compatible with INT8 quantization deployment. GRU units replace the forget gate and input gate structure of traditional LSTM, reducing computational steps by simplifying the gating mechanism. The entire model structure adapts to heterogeneous data through parallel design of feature branches, controls the model size through parameter simplification and pooling strategies in temporal layers, and reduces computational complexity by replacing depthwise separable convolutions with GRUs. Ultimately, while ensuring the recognition accuracy of the three behaviors of rolling over, getting out of bed, and significant traction, the model achieves lightweight deployment and adapts to the operating requirements of low-power devices.

[0023] The wireless communication module uses a timestamp synchronization mechanism to ensure that the sensed data is consistent across the scene at the same time, which facilitates subsequent evaluation. Step S2: Enhance dark details in the image using the Adaptive Histogram Equalization (CLAHE) algorithm to improve the visibility of the transparent material of the urine bag under strong light or backlight conditions, obtaining preprocessed image data: The detection model segments the image into the overall urine bag area, the scale line area, the liquid level line area, the pipeline status area, and the suspected leakage area, and automatically locates key areas: in: Detection model function; This is the preprocessed image data; This refers to the entire area of ​​the urine bag; This is the area marked with scale lines; This is the liquid level line area; This is the pipeline status area; Areas suspected of leakage; In some embodiments, the detection model is based on the YOLOv8-nano architecture, balancing detection accuracy and real-time requirements. For feature extraction, the original SPPF (Spatial Pyramid Pooling Fusion) module is replaced with an improved SPPF-Lite module. By reducing the number of convolutional kernels while retaining multi-scale feature fusion capabilities, the efficiency of feature extraction for small targets (such as millimeter-level scale lines) is enhanced while reducing computational cost. To address the issue of blurred boundaries between suspected leakage areas and the background, a CBAM (Convolutional Block Attention Module) channel-spatial attention mechanism is embedded in the Neck layer. This mechanism adaptively adjusts feature map weights to enhance the feature response in low-contrast areas, thereby improving the sensitivity of leakage trace recognition. At the output layer, a multi-scale Anchor-Free detection head structure is employed. Prediction is performed based on three different scale feature maps (80×80, 40×40, and 20×20) output from the Neck layer, corresponding to the detection needs of small targets (scale lines, liquid level lines), medium targets (pipeline status areas, suspected leakage areas), and large targets (the entire urine bag area), respectively. The output layer consists of a convolutional mapping layer, a prediction branch layer, and a post-processing layer, ultimately outputting the location information and category confidence scores of five types of regions. The details of the output layer are as follows: (1) Convolutional mapping layer The three scale feature maps (80×80, 40×40, and 20×20) output by the Neck layer are respectively subjected to 1×1 convolution dimensionality reduction and 3×3 convolution feature refinement, adjusting the number of channels to adapt to the detection task, as follows: Small-scale feature map (80×80): Focuses on small targets / linear regions such as scale lines and liquid level lines. After convolution, the number of channels is reduced from 64 to 32, preserving high-resolution details. Mesoscale feature map (40×40): For pipeline status areas and suspected leakage areas, the number of channels is reduced from 128 to 64 to balance details and semantic information; Large-scale feature map (20×20): responsible for overall urine bag region detection, the number of channels is reduced from 256 to 128, and the deep semantic features are enhanced.

[0024] (2) Predicting branch layers Each scale feature map is fed into an independent prediction convolutional layer, with an output tensor dimension of H×W×(4+1+5)×3. 4, 1, and 5 correspond to the three detection anchor points set in Anchor-Free mode, where: 4: Bounding box coordinate prediction, using YOLOv8's Anchor-Free offset prediction, outputs the offset of the target center point relative to the grid and its aspect ratio; 1: Target confidence prediction, representing the probability that a target exists at that location; 5: Category probability prediction, corresponding to the category probability of five areas: overall urine bag, scale line, liquid level line, pipeline status, and suspected leakage.

[0025] (3) Post-processing layer Bounding box decoding layer: Converts the predicted offset and aspect ratio into the true bounding box coordinates (x1, y1, x2, y2) in the image pixel coordinate system. In addition, coordinate constraints are added to the horizontal characteristics of the liquid level line (force the aspect ratio of the liquid level line bounding box to be >5:1). Confidence calibration layer: The confidence level is mapped to the [0,1] interval using the Sigmoid activation function, and the confidence level of suspected leakage areas is dynamically adjusted by threshold. For example, based on light intensity sensor data, the threshold is reduced by 15% under low light conditions. Non-maximum suppression (NMS): The category-aware NMS algorithm is used to filter duplicate detection boxes according to category to avoid the problem of overlapping boxes between the whole area of ​​the urine bag and sub-regions (scale lines, liquid level lines). The NMS threshold is set to 0.45 (for the scale line area of ​​small targets, the threshold is reduced to 0.35).

[0026] Step S3: Extract the scale line area of ​​the urine bag using Hough line transform and OCR model to obtain the scale numbers: in: The Hough line transform function is used to detect the scale lines in the scale line region, locate the spatial position and orientation of the scale lines, and filter out the valid scale lines. This is an OCR model function that recognizes characters on the tick marks. Liquid level line height is extracted using a liquid level segmentation model. : in: This is a function for the liquid level segmentation model; In some embodiments, the structure of the liquid level segmentation model is as follows: The model input consists of two types of image data from the urine bag monitoring scenario: RGB visible light images and infrared enhanced images. To reduce imaging instability caused by changes in illumination, differences in liquid color, and bag wrinkles, the model first performs uniform preprocessing on the input images, including adaptive histogram equalization to enhance the contrast of liquid level edges, and a lightweight denoising strategy based on median filtering to suppress reflective false edges. Subsequently, scale normalization and pixel value standardization are used to adjust all input images to a uniform size of 256×256, and Z-Score is used to pull the pixel distribution to the same dimension, providing a consistent input basis for subsequent convolutional feature extraction.

[0027] The preprocessed RGB and infrared images are respectively entered into parallel feature encoding branches. To meet the requirements of real-time deployment of low-power devices, each branch adopts the lightweight depthwise separable convolution of the MobileNet architecture as the backbone network, replacing the traditional large convolutional structure to minimize the amount of computation. Each branch contains three stages: The first stage uses 16-channel 3×3 depth-separable convolution to extract low-level texture and liquid level edge information; The second phase expands the number of channels to 32 and introduces the BatchNorm layer and ReLU6 activation to enhance stability and adapt to mobile computing power. The third stage further increases the number of channels to 64, and enhances the ability to perceive the overall contour of the urine bag and the structure of the liquid area through 5×5 depth convolution. All three stages include pointwise convolution for cross-channel fusion, and use Dropout of 0.2 to suppress feature layer overfitting.

[0028] The feature maps extracted from the two encoding branches are spatially aligned and then fused using a multi-scale stitching strategy, which includes two parts: layer-by-layer stitching and pyramid fusion. Layer-by-layer stitching preserves RGB edge details and infrared brightness gradient information; Pyramid fusion uses 1×1 convolution to achieve weighted integration of features at different scales, thereby enhancing the model's ability to perceive liquid level boundaries at multiple scales. The resulting unified high-dimensional feature map is then used as the input for the decoding part.

[0029] The decoding part is built on a lightweight U-Net structure, employing progressive upsampling and skip connections to preserve boundary information. Each upsampling stage uses bilinear interpolation instead of deconvolution to reduce artifacts, while also using 3×3 depthwise separable convolutions for boundary optimization. Skip connections concatenate shallow features from the encoding stage with high-level features from the decoding stage, enabling the model to utilize both fine-grained texture cues and global structural cues simultaneously, thus improving the accuracy of liquid level edge localization.

[0030] To further enhance the model's ability to identify the saliency of the liquid level line, a lightweight attention module is added at the end of the decoding process. This module adopts a joint structure of channel attention and spatial attention, but replaces the standard multi-head attention with 1×1 convolution and average pooling, thereby highlighting the high response characteristics of the liquid level region without significantly increasing the computational load.

[0031] The final output layer uses a single-channel Sigmoid segmentation head to map the decoded feature map into a binary segmentation probability map of the liquid level. To improve edge continuity and suppress noise points, the probability map is weighted pixel by pixel by the boundary confidence factor generated by global average pooling before output, so that the pixels of the real liquid level edge get a higher classification probability and weak edges and noise points are effectively suppressed.

[0032] The entire liquid level segmentation model uses lightweight convolution to replace traditional convolution, dual-branch heterogeneous image feature fusion, U-Net decoding structure, attention enhancement and boundary optimization strategies to form a complete liquid level extraction pipeline. While ensuring the accuracy of liquid level boundary detection, it significantly reduces the amount of inference computation and is suitable for the real-time operation requirements of embedded devices and low-power nursing monitoring terminals.

[0033] The current urine volume is calculated by mapping the liquid level height to the scale numbers, using a piecewise linear mapping: in: It is a piecewise linear mapping function; Real-time urine volume (mL); Liquid level height (mm); These are scale numbers; , This is the conversion factor for the scale segment; Let i be the boundary of the liquid level height of the i-th segment of the liquid level line region; Step S4: Collect urine volume data from the past n time points and construct a urine volume time series sequence: Predicting future urine output trends using an LSTM model: in: For LSTM model functions; Predicting for LSTM models Urine volume at any given time; Calculate the anomaly index based on the prediction error: in: For prediction error, when At that time, it was automatically determined to be an abnormal urine output. The threshold for determining abnormal urine output is [defined as] [the threshold that is met]. =0.25; Step S5: Use a convolutional neural network to identify the state region of the urinary catheter in the model and output parameters: in: Convolutional neural network recognition model function; This refers to the bend angle of the pipeline. The degree of pressure; The interface loosening index; The risk assessment rules are as follows: in: The catheter bending threshold must be met. ; The pressure threshold satisfies =0.6; The interface detachment exponent threshold is satisfied. =0.6; Step S6: Leakage Detection and Color Anomaly Analysis The leakage area is obtained using a region growth model, the leakage diffusion rate is calculated, and it is determined whether the leakage is spreading. The formula is as follows: in: For the regional growth model function; To extract the current leakage area; For the diffusion rate, when ⇒Permeation and diffusion, The permeation diffusion rate threshold satisfies =1.5cm² / min; In some embodiments, the structure of the regional growth model is as follows: The model first performs uniform preprocessing on the input liquid level monitoring images. To reduce reflective false boundaries and texture noise caused by the transparent material of the urine bag under complex lighting conditions, adaptive Gaussian smoothing is used for lightweight noise reduction. At the same time, guided filtering is combined to enhance the gradient response of candidate liquid level edges. On this basis, the image is normalized by standardizing the pixel distribution through Z-Score to ensure that the threshold sensitivity of the region growth process remains consistent under different imaging conditions. Subsequently, gradient magnitude maps are calculated using Sobel or Scharr operators to obtain candidate boundary regions where the liquid level line changes significantly compared to the background.

[0034] The preprocessed gradient map enters the seed point automatic selection module. Considering that the urine bag liquid level line usually presents a near-horizontal high gradient structure, the model adopts a strategy combining projection histogram and local maximum detection to find locations in the horizontal direction where the gradient accumulation exceeds a set threshold as seed points. The number of seeds is usually limited to 1-3 to reduce the risk of initial growth noise. To adapt to different urine bag shooting angles and liquid color differences, a dynamic threshold estimation mechanism is introduced inside the seed point selection module, which calculates the mean of the gradient map. With variance The threshold will be automatically set to This reduces threshold instability caused by environmental differences.

[0035] After the seed points are determined, the core process of region growing begins. The model employs a lightweight dual-condition region growing strategy, using pixel similarity and local connectivity as dual constraint factors; the pixel similarity condition is based on gray-level difference. With local gradient difference The joint metric can only satisfy the following conditions simultaneously: and Only neighboring pixels are added to the growth region; among them and The brightness distribution of the input image is adaptively calculated, and the local connectivity condition is obtained by 8-neighborhood search. However, redundant judgments are reduced by optimizing the displacement matrix to reduce computational complexity.

[0036] To avoid the "overgrowth" problem that easily occurs in traditional region growth, the model introduces a growth boundary suppression mechanism in the growth iteration, that is, calculating the average gradient value of the current region boundary after each round of growth. ,when When the drop exceeds a set percentage (e.g., 15%) for two consecutive rounds, growth automatically stops to ensure that the region expands only along the edge of the actual liquid level. At the same time, to adapt to the deployment of low-power devices, the region growth process uses a queue structure for leading-edge pixel management and uses integer operations instead of floating-point calculations to reduce computing power overhead.

[0037] After the region growing is completed, the liquid level boundary extraction module is entered. This module first removes isolated noise points through morphological opening operation, and then uses a thinning algorithm to skeletonize the growing result so that the liquid level boundary is represented by a single pixel line. Subsequently, the least squares linear fitting is used to perform global structure correction on the skeleton pixels to obtain a smooth and continuous liquid level line model. To enhance stability, a cross-frame smoothing strategy is also introduced, which uses exponential moving average to fuse the liquid level line positions of adjacent frames, suppressing jumps and jitter without increasing the real-time inference burden.

[0038] The entire region growth model achieves robust segmentation of the urine bag liquid level region through a combination of strategies including gradient enhancement, dynamic seed point selection, dual-condition pixel growth, boundary suppression, and structured liquid level line optimization. While ensuring the accuracy and continuity of the liquid level boundary, the model adopts a lightweight computation method in each stage and avoids complex morphological sequences to adapt to the low-power real-time operation requirements of embedded nursing monitoring devices.

[0039] The HSV color analysis model was used to calculate urine color and the degree of color deviation. The calculation formula is as follows: in: This is the HSV color analysis model function. The HSV color analysis model consists of three core dimensions: hue (H), saturation (S), and value (V), forming a three-dimensional cone space. Hue (H): The essential attribute that distinguishes different colors such as red, green, blue, and yellow. It is a circular dimension (0°~360° cycle). 0° and 360° are red, 30° is orange, 60° is yellow, 120° is green, 180° is cyan, 240° is blue, and 300° is purple. Saturation (S): Describes the vividness of a color, ranging from 0% to 100%. S=100% is a pure color, and S=0% is a colorless grayscale. Value (V): Describes the brightness of a color, ranging from 0% to 100%. V=0% is pure black, and V=100% is the brightest state of the corresponding hue. As a real-time urine color reference; This serves as a baseline for normal urine color. The degree of deviation in urine color; The formula for calculating the leakage risk index is as follows: in: The penetration area; The degree of deviation in urine color; This is the weighting coefficient for the leakage area; This represents the weighting coefficient for the degree of impact of color anomalies on risk. Step S7: The multimodal input adopts symbolic representation. The details are as follows: in: This refers to real-time urine output; This represents the prediction error; These are parameters for the pipeline status region; This is the leakage risk index; For pipeline pressure; This refers to the weight of the urine bag; The comprehensive risk index is output by the fusion model, and the input of the fusion model is the multimodal input; The calculation process of the fusion model is as follows: 1) For multimodal input Perform feature normalization to obtain normalized input. The processing formula is as follows: 2) Introduce the basic weight vector The normalized inputs are fused to obtain a fused vector. The formula is expressed as follows: 3) Input the fusion vector into the sigmoid function to obtain the comprehensive risk index. , ; in: The mean vector of the multimodal input; The standard deviation vector of the multimodal input; This is a comprehensive risk index; As the warning threshold, when Automatic alarm triggered at any time; The input data of the fusion model comes from multi-dimensional monitoring data from IoT sensors, image recognition, and time series analysis. The input data is comprehensively calculated through the multi-modal fusion framework built into the model, which includes weighted normalization, feature interaction layer, and dual-channel scoring layer. When the comprehensive nursing risk index exceeds the warning threshold, the nursing system automatically generates a warning package: in: This is a comprehensive risk index; Risk category; This is a trend chart for a recent period of time; To determine the numerical value; The warning package is pushed to the nurse station monitoring terminal, mobile nursing APP, and bedside interactive screen; The early warning package is a structured alarm data package that is automatically generated by the system and pushed to nursing staff when the system detects that the comprehensive risk index exceeds the early warning threshold. It helps nurses quickly locate the source of risk, provides sufficient data and trend support, and avoids missed judgments and misjudgments. Patient care reports are structured nursing event documents automatically generated by the system after an alert is issued. They record the entire monitoring process and nursing risk status, including: Basic information (patient ID, bed number, nursing time range, monitoring start / end time); Monitoring data (urine volume, pipeline status, leakage status, urine color changes); Triggered warning events; Ancillary analyses include: whether there is a prediction of overflow in the next hour, whether the urine output decline trend is significant, whether catheter compression is periodic, whether leakage has spread, and whether there are behaviors that increase the risk. Nursing recommendations (adjust the position of the urinary catheter, change the urine bag, assess whether there is urethral bleeding, strengthen physical restraint or reminders, assess the risk of dehydration).

[0040] Example 1: Urine output monitoring and trend prediction in patients with conventional indwelling urinary catheters 1. Test Scenario Patient A (45 years old, with indwelling catheterization after conventional bladder surgery) aims to monitor urine output steadily and determine the risk of oliguria.

[0041] 2. Monitoring data (real-time collection) A smoothed urine flow rate sequence is obtained by Kalman filtering, for example, a 10-minute sequence: [1.1, 1.2, 1.0, 1.3, 1.1, 1.1, 1.2, 1.2, 1.3, 1.1]mL / min 3. Image recognition results Scale markings indicate liquid level height: =38.2mm Scale area mapping ratio: =2.7、 =1.9 Real-time urine output calculation: =38.2×2.7+1.9=105mL 4. Urine volume trend prediction (LSTM) Urine volume sequences over approximately 4 hours (every 30 minutes) were used as input: [42, 57, 71, 89, 105, 118, 131, 150]mL LSTM predicts an increase of 23 mL in urine volume over the next hour. Abnormality Index: =0.08< =0.25 → No abnormality 5. Conclusion The system determined that the patient's urine output was normal and no warning was triggered.

[0042] Example 2: Abnormal identification of decreased urine output due to catheter compression 1. Scene Background Patient B (68 years old, post-prostate surgery) was found by nurses to have an abnormally low urine output.

[0043] 2. Internet of Things (IoT) data 3. Image recognition: catheter status The model obtained using a convolutional neural network is: Bending angle =62° (threshold > 55° is considered a bend) Pressure Index =0.78 (Threshold > 0.6 indicates pressure) Interface loosening index =0.12 (normal) Triggering Piping Risk Rules: =62° =0.78 → Catheter compression risk level → High 4. Determination of abnormal urine output Urine volume trend sequence over the past 1 hour (every 10 minutes): [13, 11, 9, 7, 5, 4] mL LSTM predicts next cycle urine volume: 2 mL Abnormality Index: =0.5> =0.25 → Indicates an abnormally low urine output 5. Triggering an alert Overall Risk Index =0.73>0.62 (catheter bending threshold), indicating "catheter compression leading to..." "Dying urine output" warning package.

[0044] Example 3: Automatic Compensation for Image Recognition Errors Caused by Insufficient Nighttime Light 1. Scene Background Patient C's room was dimly lit at night, making it difficult to determine the level of urine in the urine bag.

[0045] 2. IoT light data Light intensity: 2.1–3.9 lux (very low) 3. Automatic exposure compensation and infrared illumination Camera in IR mode, automatic exposure factor Adjusted to: =1.6 (originally 1.0) CLAHE image enhancement parameters: clipLimit=2.0 tileGridSize=8×8 The enhanced liquid level contrast was improved by 41%.

[0046] 4. Scale and liquid level identification Liquid level identification after compensation: =38.0mm Scale area mapping ratio: =2.7、 =1.9 Urine volume calculation: =38.0×2.7+1.9=104.5mL Error reduced by 86.6% 5. Effects Nighttime recognition accuracy improved from 71% to 97.4%, with no false alarms triggered.

[0047] Example 4: Automatic urine color analysis to identify mild hematuria 1. Scene Background Patient D (post-hydronephrosis surgery) may experience hematuria.

[0048] 2. Image HSV color recognition Collect the mean HSV value of the liquid: H=10.2, S=0.76, V=0.59 Normal benchmark: H0=9.0, S0=0.45, V0=0.65 Color deviation: ΔC=0.31 → Corresponding standard grading level: 2 (light red) 3. Image enhancement and contrast correction After ambient light compensation, ΔC stabilized at: 0.29–0.33 4. Leakage detection Leakage area measured using the zone growth method: 0 cm² (normal) 5. Judgment Result The system determines: "Mild hematuria", risk index Y=0.63>threshold 0.60→triggers hematuria warning.

[0049] Example 5: Combined risk identification of catheter traction and urine bag leakage caused by patient turning over 1. Scene Background Patient E (78 years old) experienced urinary tract leakage after turning over.

[0050] 2. Behavior recognition (millimeter-wave radar + IMU + LSTM) The behavioral model outputs a sequence of actions: [Rest, turn over, stretch extensively] Predicted probability: Rolling over motion P=0.88 The traction action P=0.74 (high risk) 3. Leakage detection Image region growing model output: The leakage area A = 18.5 cm² Diffusion rate v = 3.2 cm² / min (threshold 1.5) 4. Catheter status identification CNN output: Bending angle =44° (normal) Pressure Index =0.22 (normal) Interface loosening index =0.67 (>0.6, risk of loosening) 5. Multimodal risk fusion Integrated inputs: leakage area, loosening index, probability of turning over and pulling, and slight decreasing trend in urine output. The model outputs a comprehensive risk index: Y=0.78>Leakage threshold 0.55→Leakage warning triggered 6. Results The system simultaneously sends out warning packets for both "high risk of urinary tract leakage" and "high risk of catheter interface loosening".

[0051] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring urological surgery care by combining Internet of Things and image recognition, characterized in that, The steps are as follows: Step S1: acquiring multi-source monitoring data of urological surgery patient care, the multi-source monitoring data including Internet of Things sensor data, image data and behavior action data, uploading the monitoring data to a care monitoring terminal through a wireless communication module; Step S2: the adaptive histogram equalization algorithm is used for pre-processing the collected image data, and pre-processed image data is obtained The detection model is used for region segmentation on the image data, and the following parameters are output ; wherein: is the model function to be tested; is the pre-processed image data; is the overall area of the urine bag; is the scale line area; is the liquid level line area; is the tubing status area; is the leakage suspect area; Step S3: using segmented linear mapping on the scale line region and the liquid level line region to calculate real-time urine volume; Step S4: Constructing a urine volume time series , using the LSTM model to predict the future urine volume trend, and identifying abnormalities through prediction errors; Step S5: a convolutional neural network recognition model is used to recognize the pipeline state region, and pipeline state region parameters are output : wherein: is a pipe bend angle; is a pressure level; is an interface loosening index; Step S6: performing leakage detection on the suspected leakage region; performing urine color analysis on the liquid level line region to calculate a leakage risk index; Step S7: constructing multi-modal input and taking it as input of a fusion model; the fusion model outputs a comprehensive risk index to judge the comprehensive care risk of the patient.

2. The urinary surgery nursing monitoring method combining Internet of Things with image recognition according to claim 1, characterized in that, The Internet of Things sensor data is represented as follows: wherein: is real-time urine volume flow rate; is urine bag weight; is line pressure; is local temperature; is ambient light intensity; The image data is collected by a high-definition camera, which is arranged outside the urine bag, at the middle section of the catheter, and at a point with high risk of bed-side leakage. The image data is represented as follows: wherein: is a urine bag scale area profile; is a catheter form area; is a possible leakage area; The behavior action data is identified by a lightweight LSTM behavior detection model, the input data of the lightweight LSTM behavior detection model being sourced from a millimeter wave radar and an IMU, the millimeter wave radar collecting human micro-motion signals and the IMU collecting acceleration and angular velocity data; The behavior actions of the patient include turning over, getting out of bed and large-scale pulling.

3. The urological surgery care monitoring method combining the Internet of Things with image recognition according to claim 2, characterized in that, The segmented linear mapping is performed based on the liquid level height and the scale segments; extracting the liquid level height from the liquid level line region using a liquid level segmentation model ; The scale segments are extracted from the scale line region by using a Hough line transformation and an OCR model; The calculation formula of the real-time urine volume is as follows: wherein: is the real-time urine volume; and is the conversion coefficient of the i-th scale section; is the liquid level height boundary corresponding to the i-th scale section.

4. The urinary surgery nursing monitoring method combining Internet of Things with image recognition according to claim 3, characterized in that, The urine volume time sequence is constructed by collecting urine volumes at n past time points, and the abnormality identification satisfies the following formula: wherein: is the predicted urine volume at time t by the LSTM model; is the predicted urine volume at time t by the LSTM model; is the prediction error, when is automatically determined to be an abnormal urine volume, is the urine volume abnormality determination threshold.

5. The urinary surgery nursing monitoring method combining Internet of Things with image recognition according to claim 4, characterized in that, The pipeline state risk judgment rule in step S5 is as follows: when the value of the parameter is greater than the threshold value, the bending risk is determined; when the value of the parameter is greater than the threshold value, the compression risk is determined; when the value of the parameter is greater than the threshold value, the interface loosening risk is determined; wherein: is a pipe kink threshold value; is a pressure threshold value; is an interface loosening index threshold value.

6. The urinary surgery nursing monitoring method combining Internet of Things with image recognition according to claim 5, characterized in that, The leakage detection process in step S6 is as follows: extracting a leakage area from the leakage suspect area using a region growing model ;​ The leakage diffusion velocity is calculated based on the leakage area, and the formula is as follows: Whether the leakage is diffusing is judged according to the leakage diffusion velocity; wherein: is the leakage area at time t; is the time interval; is the leakage diffusion speed, when is determined as a permeation diffusion, is the permeation diffusion speed threshold value; The HSV color analysis model obtains a color analysis result by analyzing the liquid level line region; the color deviation degree is calculated according to the color analysis result, and the calculation formula is as follows: wherein: is a HSV color analysis model function; is a real-time urine color reference; is a normal urine color reference; is a urine color deviation degree; The leakage risk index calculation formula is as follows: wherein: is a weight coefficient for the leakage area; is a weight coefficient for the urine color deviation.

7. The urological surgery care monitoring method combining the Internet of Things with image recognition according to claim 6, characterized in that, The multi-modal input is represented as follows: wherein: is real-time urine volume; is prediction error; is a pipeline state area parameter; is a leak risk index; is a pipeline pressure; is a urine bag weight; The calculation process of the fusion model is as follows: S701: performing feature normalization processing on the multi-modal input to obtain a normalized input ; S702: Introduce base weight vector Weighted fusion of the normalized inputs to obtain a fused vector ; S703: input the fusion vector into the sigmoid function to obtain a comprehensive risk index , ; when an automatic alarm is triggered; is a pre-warning threshold.

8. The urological surgery care monitoring method combining the Internet of Things with image recognition according to claim 7, characterized in that, The judgment process of the comprehensive care risk is as follows: When the comprehensive risk index exceeds the early warning threshold, the nursing system automatically generates an early warning package and a patient care report, and the early warning package is as follows: wherein: is a risk category; is a trend chart for a recent period of time; The early warning package is pushed to a nurse station monitoring terminal, a mobile care APP and a bedside interaction screen.

Images