A deep learning-based infusion anomaly detection system

Abstract

The application relates to the technical field of infusion monitoring, in particular to an infusion anomaly detection system based on deep learning, which comprises the following steps: collecting an infusion video stream and intercepting a target frame sequence, and executing histogram equalization; extracting spatial texture and time dependence by using a convolution network and a long short-term memory network, and fusing and constructing a space-time feature vector; calculating an anomaly probability by using a classifier to generate confidence; and comparing a threshold to drive a blocking clamp to close. In the application, a double feature extraction network is constructed, liquid drop dynamics and pipeline correlation information are deeply mined, an attention mechanism is used to automatically focus on a key image area of bubbles, particles or blood reflux to suppress noise interference of ambient light and mechanical shaking, and an adaptive classifier is combined to dynamically adjust an anomaly judgment boundary under different liquid working conditions in real time, so that high-precision identification and response of various complex anomaly types in an infusion process under non-contact conditions are realized.

Description

Technical Field

[0001] This invention relates to the field of infusion monitoring technology, and in particular to an infusion anomaly detection system based on deep learning. Background Technology

[0002] The field of infusion monitoring technology involves the real-time monitoring and management of fluid input status during medical care. Traditional infusion anomaly detection systems refer to devices that use infrared photoelectric sensors or gravity sensors to physically measure the drip rate. This method typically involves installing a through-beam photodiode on the infusion line, calculating the flow rate by detecting the pulse signal generated by the droplet blocking the light beam, or using a weighing module to monitor the weight change trend of the medication bag to determine the remaining amount. When the numerical deviation exceeds a preset threshold, a buzzer alarm is triggered.

[0003] Existing technologies rely solely on the on / off state of photoelectric signals or a single physical quantity such as weight changes for judgment. They cannot effectively distinguish the morphological differences between bubbles and droplets, and are prone to false alarms when affected by changes in ambient light or tubing vibrations. They also struggle to accurately identify subtle abnormalities such as blood backflow or needle detachment. Furthermore, the mechanical judgment logic based on fixed thresholds lacks the ability to adapt to dynamic patterns of different drug viscosities and flow rates, resulting in severely insufficient sensitivity and reliability for detecting various infusion abnormalities in complex clinical environments. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a deep learning-based infusion anomaly detection system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a deep learning-based infusion anomaly detection system includes: The image acquisition module acquires the infusion video stream of the infusion dripping area through an optical sensor, extracts the target frame sequence from the infusion video stream according to a preset sampling frequency, performs adaptive histogram equalization processing on the target frame sequence to calculate the pixel gray value distribution, and adjusts the image brightness distribution. The feature extraction module inputs the target frame sequence into a residual convolutional neural network to extract spatial texture information, and inputs the target frame sequence into a bidirectional long short-term memory network to capture the temporal dependency of droplet falling frequency. It calculates weights based on the channel attention mechanism, and performs weighted fusion and splicing of spatial texture information and temporal dependency to construct a spatiotemporal feature vector. The anomaly analysis module inputs the spatiotemporal feature vector into the fully connected layer for linear transformation, maps it to the Softmax classifier for matrix operations, calculates the probability distribution value of the current infusion state belonging to the categories of air bubble injection, blood backflow, or needle blockage, and generates anomaly confidence based on the probability distribution value. The early warning control module calculates the difference between the abnormal confidence level and the preset safety benchmark value, determines whether the difference exceeds the preset threshold, and generates a control signal when the determination result is yes. Based on the control signal, the electromagnetic blocking clamp is driven to close to block the pipeline and trigger an alarm command.

[0006] As a further aspect of the present invention, the image acquisition module specifically includes: The video stream acquisition submodule is used to drive a high-resolution optical sensor to continuously monitor the dynamic changes in the infusion drip chamber area, acquire raw data in real time including the droplet falling trajectory and the state of liquid surface fluctuation, and transmit the data to the processor through a high-speed transmission interface to generate an infusion video stream; The frame sequence extraction submodule is used to read the infusion video stream, and based on the preset time window length and sampling interval parameters, it filters and extracts continuous image frames containing the complete droplet drop cycle from the continuous video stream data to generate the target frame sequence. The brightness equalization adjustment submodule is used to traverse the pixels of each frame in the target frame sequence, count the local grayscale histogram of the local region, and use the limited contrast adaptive histogram equalization algorithm to nonlinearly stretch and remap the dynamic range of image brightness, suppress noise amplification and enhance the contrast of image details, and generate a preprocessed frame sequence.

[0007] As a further aspect of the present invention, the feature extraction module specifically includes: The spatial texture extraction submodule is used to input the pre-processed target frame sequence into the pre-trained residual convolutional neural network model, and use multi-layer convolutional kernels to perform dimensionality reduction convolution and pooling operations on the image to extract high-dimensional spatial feature maps that reflect the shape of droplets, the edges of bubbles and the color of blood, and generate spatial texture information. The time-dependent capture submodule is used to expand the target frame sequence according to the time step, input it into the bidirectional long short-term memory network, and capture the dynamic temporal change pattern of the droplet falling process and the logical relationship between the previous and next frames through the forward and backward gating units respectively, thereby generating the time dependency relationship. The spatiotemporal fusion construction submodule is used to obtain spatial texture information and temporal dependencies. It uses a channel attention mechanism network to automatically learn the importance coefficients of different feature channels, generates weights accordingly, and uses the weights to perform weighted dot product and concatenation operations on the features to construct spatiotemporal feature vectors.

[0008] As a further aspect of the present invention, the anomaly analysis module specifically includes: The linear transformation mapping submodule is used to obtain spatiotemporal feature vectors, input them into a multilayer fully connected neural network, and map the high-dimensional feature space to a low-dimensional class space through matrix multiplication and bias addition operations. Discriminative features for distinguishing different infusion states are extracted, and class logistic regression values ​​are generated. The probability distribution calculation submodule receives the categorical logistic regression values, performs normalization exponential operations on the logistic regression values ​​using the Softmax activation function, converts the output values ​​into a probability form with a sum of 1, calculates the likelihood of the current state belonging to normal infusion, bubble injection, blood reflux, and needle blockage, and generates probability distribution values. The anomaly confidence generation submodule is used to analyze the probability distribution values, filter out the maximum probability value of abnormal infusion categories, and perform weighted correction by combining preset category weight factors to quantify the degree of certainty that the current infusion process has occurred and generate anomaly confidence.

[0009] As a further aspect of the present invention, the early warning control module specifically includes: The difference calculation and judgment submodule is used to obtain the abnormal confidence level, retrieve the preset safety benchmark value representing the upper limit of the confidence level of the normal infusion state from the memory, calculate the numerical deviation between the two, and compare the deviation with the maximum allowable fluctuation range threshold to generate the difference exceeding the limit judgment result. The closed signal generation submodule is used to immediately trigger the logic level conversion circuit of the microcontroller unit to generate a high-level pulse control signal in response to the result of the difference exceeding the limit judgment. The pipeline blockage alarm submodule is used to send control signals to the actuator of the electromagnetic blocking clamp to physically clamp the infusion pipeline, and at the same time send a trigger command to the audible and visual alarm to drive the buzzer to sound and the warning light to flash, generating an alarm command.

[0010] As a further aspect of the present invention, the process of performing adaptive histogram equalization processing on the target frame sequence to calculate the pixel grayscale value distribution specifically includes: Each frame in the target frame sequence is divided into multiple non-overlapping rectangular sub-blocks. The local gray-level histogram of each rectangular sub-block is calculated, and the frequency of each gray level is counted to construct a local cumulative distribution function and generate local gray-level statistical characteristic data. Obtain a preset contrast limit threshold, use this threshold to crop the number of pixels in the local grayscale histogram that exceed the limit, and evenly distribute the cropped pixels to other grayscale levels in the histogram. Recalculate the cumulative distribution function after cropping to generate a mapping transformation curve. The pixel values ​​at the center of each rectangular sub-block are transformed using a mapping transformation curve. For the pixels in the boundary region of the sub-block, a bilinear interpolation algorithm combined with the transformation function of adjacent sub-blocks is used to perform smooth transition processing, eliminate block artifacts, and generate pixel grayscale values.

[0011] As a further aspect of the present invention, the process of calculating weights based on the channel attention mechanism specifically includes: The intermediate feature map obtained by concatenating spatial texture information and temporal dependency is then processed by global average pooling and global max pooling operations to compress the spatial dimension and aggregate global statistical features of the channel dimension, thereby generating average pooling descriptors and max pooling descriptors. The average pooling descriptor and the max pooling descriptor are input into a multilayer perceptron network with shared weights for nonlinear mapping. The two outputs are then fused element-wise, and the fused result is mapped to the range of 0 to 1 using the Sigmoid activation function, according to the formula: ; The response coefficients of each channel are calculated, and weights are generated. in, Represents the generated weights, Represents the Sigmoid activation function. Represents a multilayer perceptron network. This represents the global average pooling operation. This represents the global max-pooling operation. The intermediate feature map represents the input.

[0012] As a further aspect of the present invention, the process of weighted fusion and splicing of spatial texture information and temporal dependency specifically includes: The weights are multiplied one channel at a time with the corresponding feature channels. Attention weights are used to enhance the effective information in the feature map and suppress invalid background noise, resulting in an attention-calibrated weighted feature map, which generates a spatial texture tensor and a time-dependent tensor. The spatial texture tensor is flattened into a one-dimensional vector format and concatenated with the time-dependent tensor along the feature dimension to form a high-dimensional joint feature vector, which generates the initial fusion vector. The initial fusion vector is input into the fully connected layer for dimensionality reduction. By learning the nonlinear interaction relationship between different feature modalities, the feature space is compressed to a preset length, redundant information is removed and key discriminative features are retained to construct a spatiotemporal feature vector.

[0013] As a further aspect of the present invention, the process of generating anomaly confidence levels based on probability distribution values ​​specifically includes: Obtain the probability values ​​corresponding to the three abnormal states of bubble injection, blood reflux and needle blockage from the probability distribution values, calculate the weighted sum of the probability values ​​of these three abnormal states, use the weighted sum as the preliminary abnormal score, and generate the basic abnormal index. The information entropy of the probability distribution value is calculated. The information entropy is used to measure the uncertainty of the model for the current classification result. If the information entropy is higher than the preset uncertainty threshold, the basic anomaly index is attenuated and corrected using a penalty factor to generate an anomaly index. The abnormal index is weighted and averaged with the abnormal index of historical time steps through a smoothing filter algorithm to eliminate the random fluctuation interference of single-frame prediction, and a stable value reflecting the abnormal trend of infusion status in the current period is obtained, thus generating an abnormality confidence level.

[0014] As a further aspect of the present invention, the process of determining whether the difference exceeds a preset threshold specifically includes: The system obtains the current operating environment parameters, dynamically adjusts the judgment sensitivity based on the ambient light intensity and the viscosity properties of the infusion fluid, matches the corresponding current judgment threshold from the preset threshold lookup table, and generates an adaptive threshold. Calculate the absolute value of the difference between the anomaly confidence level and the preset safety benchmark value, and compare the absolute value of the difference with the adaptive threshold. If the absolute value of the difference is strictly greater than the adaptive threshold, it is determined that there is a substantial risk of abnormality in the current infusion state, and a positive judgment result is generated. If a positive judgment result is generated, the number of consecutive frames that generate positive judgment results is further counted. Only when the number of consecutive frames exceeds the preset time window length is the abnormal state confirmed to continue to exist, in order to prevent misjudgment caused by instantaneous interference and generate a difference limit exceeding signal.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, a dual feature extraction network containing spatial morphology and temporal sequence dimensions is constructed to deeply mine the correlation information between droplet dynamic changes and tubing status in infusion video streams. An attention mechanism is used to automatically focus on key image areas of air bubbles, microparticles, or blood backflow to effectively suppress noise interference from ambient light and mechanical shaking. Combined with an adaptive classifier, the anomaly judgment boundary under different drug solution conditions is dynamically adjusted in real time, thereby achieving high-precision identification and response to various complex anomalies during infusion under non-contact conditions. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the principle of the infusion anomaly detection system of the present invention; Figure 2 This is a flowchart of the image acquisition module and its processing of the present invention; Figure 3 This is a flowchart of the feature extraction module and its processing of the present invention; Figure 4 This is a flowchart of the anomaly analysis module and its processing of the present invention; Figure 5 This is a flowchart of the early warning control module and its processing of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the software-based technical solution is described in detail below with reference to system architecture diagrams and embodiments. It should be understood that the specific embodiments described herein are only for explaining the technical solutions of this invention and do not constitute a limitation on the scope of protection.

[0018] In the description of this invention, the system architecture relationships or data processing flows indicated by terms such as "layer," "module," "interface," "data flow," "client," and "server" are all defined based on the architecture diagram or flowchart corresponding to the embodiments. This way of describing is only used to clearly illustrate the logical relationships between the elements in the technical solution, and not to limit the physical deployment form. The term "multiple" includes two or more technical units, including but not limited to multiple data nodes, processing threads, service instances, or functional components and other scalable elements. The specific number is determined according to the actual business scenario and needs to be specifically stated.

[0019] Please see Figure 1 and Figure 2 This invention provides a technical solution: a deep learning-based infusion anomaly detection system comprising: The image acquisition module acquires the infusion video stream of the infusion dripping area through an optical sensor, extracts the target frame sequence from the infusion video stream according to a preset sampling frequency, performs adaptive histogram equalization processing on the target frame sequence to calculate the pixel gray value distribution, and adjusts the image brightness distribution.

[0020] The image acquisition module specifically includes: The video stream acquisition submodule is used to drive a high-resolution optical sensor to continuously monitor the dynamic changes in the infusion drip chamber area, acquire raw data in real time including the droplet falling trajectory and the state of liquid surface fluctuation, and transmit the data to the processor through a high-speed transmission interface to generate an infusion video stream; The frame sequence extraction submodule is used to read the infusion video stream, and based on the preset time window length and sampling interval parameters, it filters and extracts continuous image frames containing the complete droplet drop cycle from the continuous video stream data to generate the target frame sequence. The brightness equalization adjustment submodule is used to traverse the pixels of each frame in the target frame sequence, count the local grayscale histogram of the local region, and use the limited contrast adaptive histogram equalization algorithm to nonlinearly stretch and remap the dynamic range of image brightness, suppress noise amplification and enhance the contrast of image details, and generate a preprocessed frame sequence.

[0021] The process of performing adaptive histogram equalization on the target frame sequence to calculate the pixel grayscale distribution specifically includes: Each frame in the target frame sequence is divided into multiple non-overlapping rectangular sub-blocks. The local gray-level histogram of each rectangular sub-block is calculated, and the frequency of each gray level is counted to construct a local cumulative distribution function and generate local gray-level statistical characteristic data. Obtain a preset contrast limit threshold, use this threshold to crop the number of pixels in the local grayscale histogram that exceed the limit, and evenly distribute the cropped pixels to other grayscale levels in the histogram. Recalculate the cumulative distribution function after cropping to generate a mapping transformation curve. The pixel values ​​at the center of each rectangular sub-block are transformed using a mapping transformation curve. For the pixels in the boundary region of the sub-block, a bilinear interpolation algorithm combined with the transformation function of adjacent sub-blocks is used to perform smooth transition processing, eliminate block artifacts, and generate pixel grayscale values.

[0022] The image acquisition module is equipped with an industrial-grade high-resolution CMOS image sensor, specifically the Sony IMX485. This sensor has a 1 / 1.2-inch photosensitive area and a single pixel size of 2.9 micrometers, maintaining a high signal-to-noise ratio in low-light environments. The sensor connects to the front-end FPGA processing chip via a MIPICSI-2 high-speed interface. The video stream acquisition resolution is set to 3840 pixels by 2160 pixels, with a constant frame rate of 60 frames per second to ensure clear capture of the instantaneous shape of high-speed falling droplets and minute surface tremors. The optical lens is a 16mm fixed-focus lens with an aperture of F1.8, and the focus distance is locked at the center plane of the infusion drip chamber, with the depth of field controlled within ±5mm. During the acquisition process, the original RAW format data is processed by the ISP image signal processing unit to perform de-mosaic, white balance correction and gamma correction, and the data format is converted into the standard YUV422 format. Then, the video stream data is written to the high-speed DDR4 cache through the DMA direct memory access channel to establish a 512-megabyte ring buffer for real-time storage of the infusion video stream data within the last 10 seconds.

[0023] The aforementioned circular buffer refers to a memory data structure where the beginning and end are connected. When the end of the buffer is full, the old data is automatically overwritten starting from the beginning, thus achieving circular storage of continuous stream data.

[0024] The video stream acquisition submodule works in conjunction with the frame sequence extraction submodule. It directly reads the video stream data from the circular buffer via memory mapping, setting the time window length to 2 seconds and the sampling interval to extract one frame every three frames, i.e., 20 frames per second. To ensure the extracted frame sequence contains the complete droplet fall cycle, a background subtraction algorithm is used to calculate the pixel change rate in the central region of the image in real time. When the pixel change rate of three consecutive frames exceeds a preset dynamic threshold of 15%, it is marked as the starting point of the droplet fall event. A 0.5-second backtracking and 1.5-second forward intercept is then used to extract the target frame sequence containing 40 consecutive images. All extracted image frames are uniformly resized to 224 pixels by 224 pixels to fit the input requirements of subsequent deep learning models and are normalized, mapping pixel values ​​from 0 to 255 to a floating-point range of 0 to 1.

[0025] The brightness equalization adjustment submodule performs fine-tuning of the Y-channel brightness of each frame of the image to address potential uneven lighting or glare interference in the infusion environment. The 224-pixel by 224-pixel image is divided into 64 non-overlapping rectangular sub-blocks (8 rows, 8 columns), each 28 pixels by 28 pixels. For each rectangular sub-block, the grayscale distribution of its 784 pixels is analyzed, constructing a local histogram with 256 grayscale levels. A contrast limit threshold of 3.0 is set; this threshold was determined after testing 1000 sets of infusion videos under different lighting conditions, effectively preventing excessive amplification of background noise in flat areas. During the calculation, for grayscale levels in the histogram whose frequency exceeds the limit threshold, these pixels are truncated, and the total number of truncated pixels is counted. These truncated pixels are then evenly distributed across the other grayscale levels in the histogram, resulting in a smoother overall histogram distribution. After histogram correction, the corresponding cumulative distribution function is generated as the mapping transformation curve. When remapping image pixels using this curve, to eliminate block artifacts at the seams between sub-blocks, for any pixel in the image, the transformation functions of its four nearest surrounding sub-blocks are selected based on its coordinates. The distance weights of this pixel relative to the centers of these four sub-blocks are calculated, and a bilinear interpolation algorithm is used to calculate the final grayscale value. Experimental data shows that after this processing, the contrast of the droplet edges is improved by 45%, and the grayscale difference between the bubbles and the liquid is improved by 30%.

[0026] The bilinear interpolation algorithm described above refers to a method that performs linear interpolation once in each of the two directions and uses the gray values ​​of the four nearest neighbor pixels and their weights to estimate the gray value of the target pixel.

[0027] Please see Figure 1 and Figure 3The feature extraction module inputs the target frame sequence into a residual convolutional neural network to extract spatial texture information, and inputs the target frame sequence into a bidirectional long short-term memory network to capture the temporal dependence of the droplet falling frequency. Based on the channel attention mechanism, weights are calculated, and the spatial texture information and temporal dependence are weighted, fused and spliced ​​to construct a spatiotemporal feature vector.

[0028] The feature extraction module specifically includes: The spatial texture extraction submodule is used to input the pre-processed target frame sequence into the pre-trained residual convolutional neural network model, and use multi-layer convolutional kernels to perform dimensionality reduction convolution and pooling operations on the image to extract high-dimensional spatial feature maps that reflect the shape of droplets, the edges of bubbles and the color of blood, and generate spatial texture information. The time-dependent capture submodule is used to expand the target frame sequence according to the time step, input it into the bidirectional long short-term memory network, and capture the dynamic temporal change pattern of the droplet falling process and the logical relationship between the previous and next frames through the forward and backward gating units respectively, thereby generating the time dependency relationship. The spatiotemporal fusion construction submodule is used to obtain spatial texture information and temporal dependencies. It uses a channel attention mechanism network to automatically learn the importance coefficients of different feature channels, generates weights accordingly, and uses the weights to perform weighted dot product and concatenation operations on the features to construct spatiotemporal feature vectors.

[0029] The process of calculating weights based on the channel attention mechanism specifically includes: The intermediate feature map obtained by concatenating spatial texture information and temporal dependency is then processed by global average pooling and global max pooling operations to compress the spatial dimension and aggregate global statistical features of the channel dimension, thereby generating average pooling descriptors and max pooling descriptors. The average pooling descriptor and the max pooling descriptor are input into a multilayer perceptron network with shared weights for nonlinear mapping. The two outputs are then fused element-wise, and the fused result is mapped to the range of 0 to 1 using the Sigmoid activation function, according to the formula: ; The response coefficients of each channel are calculated, and weights are generated. in, Represents the generated weights, Represents the Sigmoid activation function. Represents a multilayer perceptron network. This represents the global average pooling operation. This represents the global max-pooling operation. The intermediate feature map represents the input.

[0030] The process of weighted fusion and stitching of spatial texture information and temporal dependencies specifically includes: The weights are multiplied one channel at a time with the corresponding feature channels. Attention weights are used to enhance the effective information in the feature map and suppress invalid background noise, resulting in an attention-calibrated weighted feature map, which generates a spatial texture tensor and a time-dependent tensor. The spatial texture tensor is flattened into a one-dimensional vector format and concatenated with the time-dependent tensor along the feature dimension to form a high-dimensional joint feature vector, which generates the initial fusion vector. The initial fusion vector is input into the fully connected layer for dimensionality reduction. By learning the nonlinear interaction relationship between different feature modalities, the feature space is compressed to a preset length, redundant information is removed and key discriminative features are retained to construct a spatiotemporal feature vector.

[0031] The spatial texture extraction submodule uses a ResNet-50 residual network model pre-trained on the ImageNet dataset as its backbone. The input preprocessed frame sequence first passes through a 7x7 convolutional layer with a stride of 2 and padding of 3, downsampling the image size to 112 pixels by 112 pixels while expanding the number of channels to 64. This is followed by further dimensionality reduction through a 3x3 max-pooling layer. The data then flows into a feature extraction stage consisting of multiple stacked residual blocks. Each residual block contains three convolutional layers: a 1x1 convolution for dimensionality reduction and channel compression, a 3x3 convolution for spatial feature extraction, and a 1x1 convolution for dimensionality upscaling and channel recovery. During feature extraction, the ReLU activation function is used to introduce non-linearity, mathematically expressed as: After four stages of residual block processing, a high-dimensional feature map containing droplet contours, liquid surface curvature, and microbubble textures is extracted. The output dimension has 2048 channels and a spatial resolution of 7 pixels by 7 pixels.

[0032] The time-dependent capture submodule receives the target frame sequence with a time step of 40. It unfolds the image feature vector of each frame chronologically, constructing a temporal feature matrix of shape 40 x 2048. This matrix is ​​input into a bidirectional Long Short-Term Memory (LSTM) network, which contains a two-layer stacked structure with 512 hidden units. Along the forward propagation path, the LSTM units are processed chronologically... arrive The input is processed sequentially, using a forget gate to determine how much of the cell state from the previous time step is retained, and an input gate to determine how much of the new information from the current time step is updated into the cell state. Simultaneously, the backpropagation path follows... arrive The input is processed in reverse order. Finally, the 512-dimensional hidden state vectors output at each time step in both the forward and backward directions are concatenated element-wise to generate a time dependency vector with a dimension of 1024.

[0033] The spatiotemporal fusion construction submodule introduces a channel attention mechanism to optimize feature fusion. For the input intermediate feature map... Its dimensions are (Batch size, number of channels, height, width). First, perform global average pooling and global max pooling operations respectively: global average pooling will pool the data of each channel... The spatial features are averaged to obtain the descriptor. Global max pooling takes the maximum value of each channel to obtain the descriptor. These two descriptors are fed into a multilayer perceptron (MLP) with shared weights. This MLP contains two fully connected layers, with the number of nodes in the intermediate layer being one-sixteenth of the number of channels. The first layer uses the ReLU activation function, and the second layer outputs directly. According to the formula: The response coefficients of each channel were calculated.

[0034] in, This represents the generated channel weight vector, whose dimension is the same as the number of channels in the input feature map, and is used to weight the feature channels. This represents the Sigmoid activation function, which maps output values ​​to the range of 0 to 1. This represents a multilayer perceptron network, used to learn the nonlinear interaction relationships between channels; This represents the global average pooling operation, used to aggregate spatial background information; This represents the global max pooling operation, used to extract significant texture features; The intermediate feature map represents the input.

[0035] First, obtain the 2048-channel spatial feature map output by the aforementioned ResNet. The vector is obtained through average pooling. The vector is obtained through max pooling. The two vectors are input into an MLP network, and after linear mapping and nonlinear transformation, two feature response vectors of the same dimension are output. These two vectors are then added element-wise, and the fusion result is mapped to a closed interval of 0 to 1 using the Sigmoid activation function. The calculated weights are then... (2048-dimensional) It is multiplied channel-by-channel with the original feature channels. The processed spatial texture tensor is flattened into a one-dimensional vector, which is concatenated with the 1024-dimensional time-dependent tensor output by the Bi-LSTM to form an initial fusion vector of 3072 dimensions. Finally, a fully connected layer is used to reduce the dimension to 512 dimensions, which is the final spatiotemporal feature vector.

[0036] The training process uses the cross-entropy loss function to measure the difference between the predicted class and the true label. The AdamW optimizer is used for parameter updates, with an initial learning rate of 0.0001, a weight decay coefficient of 0.01, and a momentum parameter... The model was trained on an infusion anomaly dataset containing 50,000 labeled videos with a batch size of 32. After 100 epochs, the validation set accuracy reached 98.5%.

[0037] Please see Figure 1 and Figure 4 The anomaly analysis module inputs the spatiotemporal feature vector into the fully connected layer for linear transformation, maps it to the Softmax classifier for matrix operations, calculates the probability distribution value of the current infusion state belonging to the categories of air bubble injection, blood reflux, or needle blockage, and generates anomaly confidence based on the probability distribution value.

[0038] The anomaly analysis module specifically includes: The linear transformation mapping submodule is used to obtain spatiotemporal feature vectors, input them into a multilayer fully connected neural network, and map the high-dimensional feature space to a low-dimensional class space through matrix multiplication and bias addition operations. Discriminative features for distinguishing different infusion states are extracted, and class logistic regression values ​​are generated. The probability distribution calculation submodule receives the categorical logistic regression values, performs normalization exponential operations on the logistic regression values ​​using the Softmax activation function, converts the output values ​​into a probability form with a sum of 1, calculates the likelihood of the current state belonging to normal infusion, bubble injection, blood reflux, and needle blockage, and generates probability distribution values. The anomaly confidence generation submodule is used to analyze the probability distribution values, filter out the maximum probability value of abnormal infusion categories, and perform weighted correction by combining preset category weight factors to quantify the degree of certainty that the current infusion process has occurred and generate anomaly confidence.

[0039] The process of generating anomaly confidence scores based on probability distribution values ​​specifically includes: Obtain the probability values ​​corresponding to the three abnormal states of bubble injection, blood reflux and needle blockage from the probability distribution values, calculate the weighted sum of the probability values ​​of these three abnormal states, use the weighted sum as the preliminary abnormal score, and generate the basic abnormal index. The information entropy of the probability distribution value is calculated. The information entropy is used to measure the uncertainty of the model for the current classification result. If the information entropy is higher than the preset uncertainty threshold, the basic anomaly index is attenuated and corrected using a penalty factor to generate an anomaly index. The abnormal index is weighted and averaged with the abnormal index of historical time steps through a smoothing filter algorithm to eliminate the random fluctuation interference of single-frame prediction, and a stable value reflecting the abnormal trend of infusion status in the current period is obtained, thus generating an abnormality confidence level.

[0040] The linear transformation mapping submodule receives a 512-dimensional spatiotemporal feature vector and inputs it into a classification network consisting of three fully connected layers. The first fully connected layer maps the 512-dimensional features to 256 dimensions and adds a Dropout layer with a dropout rate of 0.5; the second layer maps the 256-dimensional features to 128 dimensions; the third output layer has 4 nodes, corresponding to four states: "normal infusion," "air bubble injection," "blood reflux," and "needle occlusion." Each fully connected layer performs matrix multiplication. ,in It is a weight matrix. y is the bias vector, where x represents the input feature vector of the fully connected layer and y represents the output feature vector of the fully connected layer after linear transformation.

[0041] The probability distribution calculation submodule receives four logistic regression values ​​from the output layer. Assume the current output logistic regression value vector... for Normalization is performed using the Softmax function, and the calculation formula is as follows: ,in, This represents the normalized probability value for the i-th category. The logistic regression value represents the i-th category. Let represent the logistic regression value for the j-th category, with the denominator being the sum of the exponents of the logistic regression values ​​for all four categories. Substituting the values ​​into the above calculation, , , , The sum is approximately 99.39. The calculated probability distribution value is approximately... This means that the current system considers there to be a 78% probability of a normal infusion and a 17% probability of a bubble infusion.

[0042] The anomaly confidence generation submodule first extracts the probability values ​​corresponding to the three anomaly categories from the probability distribution. Continuing the previous example, the anomaly probability set is: ,in The probability value representing the type of bubble blanking. The probability value representing the type of blood return. This represents the probability value for the needle occlusion category. A baseline anomaly index is calculated based on preset severity weighting factors (air bubble = 1.0, backflow = 1.2, occlusion = 0.8). Substitute the parameters into the calculation: To assess the reliability of the prediction, information entropy is calculated. Substituting the probability values, the calculation is as follows: Taking the absolute value, the information entropy is approximately 0.956. Setting the uncertainty threshold to 1.2, since... This indicates that the model has high determinism and no penalty factor is introduced. To eliminate jitter in single-frame prediction, an exponential moving average algorithm is used for smoothing. A smoothing coefficient is set. Obtain the anomaly confidence level of the previous time step. The confidence level is 0.2. Calculate the confidence level at the current time: This value represents the system's ultimate certainty that an abnormal state has occurred.

[0043] Please see Figure 1 and Figure 5 The early warning control module calculates the difference between the abnormal confidence level and the preset safety benchmark value, determines whether the difference exceeds the preset threshold, and generates a control signal when the determination result is yes. Based on the control signal, the electromagnetic blocking clamp is driven to close to block the pipeline and trigger an alarm command.

[0044] The early warning and control module specifically includes: The difference calculation and judgment submodule is used to obtain the abnormal confidence level, retrieve the preset safety benchmark value representing the upper limit of the confidence level of the normal infusion state from the memory, calculate the numerical deviation between the two, and compare the deviation with the maximum allowable fluctuation range threshold to generate the difference exceeding the limit judgment result. The closed signal generation submodule is used to immediately trigger the logic level conversion circuit of the microcontroller unit to generate a high-level pulse control signal in response to the result of the difference exceeding the limit judgment. The pipeline blockage alarm submodule is used to send control signals to the actuator of the electromagnetic blocking clamp to physically clamp the infusion pipeline, and at the same time send a trigger command to the audible and visual alarm to drive the buzzer to sound and the warning light to flash, generating an alarm command.

[0045] The process of determining whether the difference exceeds a preset threshold specifically includes: The system obtains the current operating environment parameters, dynamically adjusts the judgment sensitivity based on the ambient light intensity and the viscosity properties of the infusion fluid, matches the corresponding current judgment threshold from the preset threshold lookup table, and generates an adaptive threshold. Calculate the absolute value of the difference between the anomaly confidence level and the preset safety benchmark value, and compare the absolute value of the difference with the adaptive threshold. If the absolute value of the difference is strictly greater than the adaptive threshold, it is determined that there is a substantial risk of abnormality in the current infusion state, and a positive judgment result is generated. If a positive judgment result is generated, the number of consecutive frames that generate positive judgment results is further counted. Only when the number of consecutive frames exceeds the preset time window length is the abnormal state confirmed to continue to exist, in order to prevent misjudgment caused by instantaneous interference and generate a difference limit exceeding signal.

[0046] The difference calculation and judgment submodule incorporates a high-precision environmental monitoring unit, including a BH1750 light sensor and a miniature temperature sensor. During system operation, it reads ambient light intensity data (unit: Lux) and liquid temperature data in real time, and dynamically adjusts the judgment threshold based on the input liquid viscosity parameter. The system has a pre-stored adaptive threshold lookup table as shown in Table 1.

[0047] Table 1: Environmental Adaptive Threshold Lookup Table ; As shown in Table 1, when the ambient light intensity is 300 Lux and physiological saline is being infused, the system's preset safety baseline value is 0.90, and the adaptive judgment threshold is 0.10. Based on the previously calculated anomaly confidence level of 0.2042, the confidence level for the current state to be considered normal can be deduced as follows: Calculate the absolute value of the difference between it and the safety benchmark value: The difference of 0.1042 is compared with the adaptive threshold of 0.10. Since... If the result is positive, it indicates a potential anomaly risk. To prevent false alarms, the module has a sliding confirmation window with a length of 5 frames. Only when the difference judgment results of 5 consecutive frames are all positive will it be finally confirmed as an abnormal state and a difference exceeding the limit signal will be generated.

[0048] The closed-signal generation submodule is connected to the GPIO output port of the STM32F4 series microcontroller. Upon receiving a difference exceeding the limit signal, the microcontroller immediately pulls the corresponding GPIO pin level from low (0V) to high (3.3V), with the high-level signal duration set to 500 milliseconds. This signal is then transmitted to the relay driver module via an optocoupler isolation circuit. This optocoupler isolation design effectively blocks electromagnetic interference from the high current in the subsequent stage to the front-end control circuit.

[0049] The pipeline blockage alarm submodule includes a normally open electromagnet actuator and an audible and visual alarm. The electromagnetic clamp is installed on the outside of the infusion tubing and has a built-in spring reset mechanism. When a control signal with amplified power is received, the electromagnet coil is energized, generating an electromagnetic force of approximately 50 Newtons. This force overcomes the spring resistance, pushing the clamp to close and physically squeezing the inner wall of the infusion tubing to completely block the fluid flow. Simultaneously, the control signal triggers a buzzer to emit an intermittent alarm sound at a frequency of 2 Hz (85 dB), and drives a red LED warning light to flash at the same frequency. Experimental test data are shown in Table 2, demonstrating the module's response performance under different abnormal conditions.

[0050] Table 2. Abnormal Blocking Response Performance Test Table: ; Table 2 lists the test results of the embodiments. The data shows that the average response time of the system to various anomalies is within 200 milliseconds, and the blocking success rate reaches 100%, verifying the efficiency and reliability of the early warning control module. By adaptively adjusting the thresholds for illumination and viscosity, the false alarm rate of the system in complex environments is reduced by more than 15%.

[0051] The above embodiments illustrate preferred embodiments of the present invention. Any equivalent adjustments to the technical solution based on software engineering methods are within the scope of protection, including but not limited to: implementing algorithm logic using different programming languages, refactoring functional modules into services, adjusting data interaction protocols, and optimizing resource scheduling strategies. Any implementation scheme derived from reasonable modifications to the data processing flow, service call chain, or system architecture layer without departing from the core technology of the present invention should be considered within the protection scope defined by the technical solution of the present invention.

Claims

1. A deep learning-based infusion anomaly detection system, characterized in that, The system includes: The image acquisition module acquires the infusion video stream of the infusion drip chamber area through an optical sensor, extracts the target frame sequence from the infusion video stream according to a preset sampling frequency, performs adaptive histogram equalization processing on the target frame sequence to calculate the pixel gray value distribution, and adjusts the image brightness distribution. The feature extraction module inputs the target frame sequence into a residual convolutional neural network to extract spatial texture information, inputs the target frame sequence into a bidirectional long short-term memory network to capture the time dependency of droplet falling frequency, calculates weights based on a channel attention mechanism, and performs weighted fusion and splicing of the spatial texture information and the time dependency to construct a spatiotemporal feature vector. The anomaly analysis module inputs the spatiotemporal feature vector into a fully connected layer for linear transformation, maps it to a Softmax classifier for matrix operations, calculates the probability distribution value of the current infusion state belonging to the categories of air bubble injection, blood reflux, or needle blockage, and generates anomaly confidence based on the probability distribution value. The early warning control module calculates the difference between the abnormal confidence level and the preset safety benchmark value, determines whether the difference exceeds the preset threshold, and generates a control signal when the determination result is yes. Based on the control signal, the electromagnetic blocking clamp is driven to close to block the pipeline and trigger an alarm command.

2. The deep learning-based infusion anomaly detection system according to claim 1, characterized in that, The image acquisition module specifically includes: The video stream acquisition submodule is used to drive a high-resolution optical sensor to continuously monitor the dynamic changes in the infusion drip chamber area, acquire raw data in real time including the droplet falling trajectory and the state of liquid surface fluctuation, and transmit the data to the processor through a high-speed transmission interface to generate an infusion video stream; The frame sequence extraction submodule is used to read the infusion video stream, and according to the preset time window length and sampling interval parameters, to filter and extract continuous image frames containing the complete droplet falling cycle from the continuous video stream data, and generate a target frame sequence. The brightness equalization adjustment submodule is used to traverse the pixels of each frame in the target frame sequence, count the local grayscale histogram of the local region, and use the limited contrast adaptive histogram equalization algorithm to nonlinearly stretch and remap the dynamic range of image brightness, suppress noise amplification and enhance the contrast of image details, and generate a preprocessed frame sequence.

3. The deep learning-based infusion anomaly detection system according to claim 1, characterized in that, The feature extraction module specifically includes: The spatial texture extraction submodule is used to input the preprocessed target frame sequence into a pre-trained residual convolutional neural network model, and use multi-layer convolutional kernels to perform dimensionality reduction convolution and pooling operations on the image to extract high-dimensional spatial feature maps that reflect the shape of droplets, the edges of bubbles and the color of blood, thereby generating spatial texture information. The time-dependent capture submodule is used to expand the target frame sequence according to the time step, input it into the bidirectional long short-term memory network, and capture the dynamic temporal change pattern of the droplet falling process and the logical relationship between the previous and next frames through the forward and backward gating units respectively, so as to generate the time dependency relationship. The spatiotemporal fusion construction submodule is used to obtain the spatial texture information and the temporal dependency relationship. It automatically learns the importance coefficients of different feature channels using a channel attention mechanism network, generates weights accordingly, and uses the weights to perform weighted dot product and concatenation operations on the features to construct a spatiotemporal feature vector.

4. The deep learning-based infusion anomaly detection system according to claim 1, characterized in that, The anomaly analysis module specifically includes: The linear transformation mapping submodule is used to obtain the spatiotemporal feature vector, input it into a multilayer fully connected neural network, and map the high-dimensional feature space to the low-dimensional category space through matrix multiplication and bias addition operations, extract discriminative features for distinguishing different infusion states, and generate categorical logistic regression values. The probability distribution calculation submodule receives the categorical logistic regression values, performs normalization exponential operations on the logistic regression values ​​using the Softmax activation function, converts the output values ​​into a probability form with a sum of 1, calculates the likelihood of the current state belonging to normal infusion, bubble injection, blood reflux, and needle blockage, and generates probability distribution values. The anomaly confidence generation submodule is used to parse the probability distribution value, filter out the maximum probability value of abnormal infusion category, combine it with preset category weighting factor for weighted correction, quantify the degree of certainty that the current infusion process has an anomaly, and generate anomaly confidence.

5. The deep learning-based infusion anomaly detection system according to claim 1, characterized in that, The early warning control module specifically includes: The difference calculation and judgment submodule is used to obtain the abnormal confidence level, retrieve the preset safety benchmark value representing the upper limit of the confidence level of normal infusion state from the memory, calculate the numerical deviation between the two, and compare the deviation with the maximum allowable fluctuation range threshold to generate a difference exceeding the limit judgment result. The closed signal generation submodule is used to immediately trigger the logic level conversion circuit of the microcontroller unit to generate a high-level pulse control signal in response to the difference exceeding the limit judgment result being yes. The pipeline blockage alarm submodule is used to send the control signal to the actuator of the electromagnetic blocking clamp to physically clamp the infusion pipeline, and at the same time send a trigger command to the audible and visual alarm to drive the buzzer to sound and the warning light to flash, thereby generating an alarm command.

6. The deep learning-based infusion anomaly detection system according to claim 2, characterized in that, The process of performing adaptive histogram equalization on the target frame sequence to calculate the pixel grayscale distribution specifically includes: Each frame image in the target frame sequence is divided into multiple non-overlapping rectangular sub-blocks. The local gray-level histogram of each rectangular sub-block is calculated, and the frequency of occurrence of each gray level is counted to construct a local cumulative distribution function and generate local gray-level statistical characteristic data. Obtain a preset contrast limit threshold, use this threshold to crop the number of pixels in the local grayscale histogram that exceed the limit, and evenly distribute the cropped pixels to other grayscale levels in the histogram. Recalculate the cumulative distribution function after cropping to generate a mapping transformation curve. The pixel values ​​at the center of each rectangular sub-block are transformed using the mapping transformation curve. For the pixels in the boundary region of the sub-block, a bilinear interpolation algorithm combined with the transformation function of adjacent sub-blocks is used to perform smooth transition processing, eliminate block artifacts, and generate pixel grayscale values.

7. The deep learning-based infusion anomaly detection system according to claim 3, characterized in that, The process of calculating weights based on the channel attention mechanism specifically includes: The intermediate feature map obtained by concatenating the spatial texture information and the temporal dependency is then processed by performing global average pooling and global max pooling operations on the intermediate feature map to compress the spatial dimension and aggregate the global statistical features of the channel dimension, thereby generating average pooling descriptors and max pooling descriptors. The average pooling descriptor and the max pooling descriptor are respectively input into a multilayer perceptron network with shared weights for nonlinear mapping. The two outputs are then fused element-wise, and the fused result is mapped to the range of 0 to 1 using the Sigmoid activation function, according to the formula: ； The response coefficients of each channel are calculated, and weights are generated. in, Represents the generated weights, Represents the Sigmoid activation function. Represents a multilayer perceptron network. This represents the global average pooling operation. This represents the global max-pooling operation. The intermediate feature map represents the input.

8. The deep learning-based infusion anomaly detection system according to claim 3, characterized in that, The process of weighted fusion and splicing of the spatial texture information and the temporal dependency specifically includes: The weights are multiplied one channel at a time with the corresponding feature channels. Attention weights are used to enhance the effective information in the feature map and suppress invalid background noise, resulting in an attention-calibrated weighted feature map, which generates a spatial texture tensor and a time-dependent tensor. The spatial texture tensor is converted into a one-dimensional vector format through a flattening operation, and then concatenated with the time-dependent tensor in the feature dimension to form a high-dimensional joint feature vector, generating an initial fusion vector. The initial fusion vector is input into a fully connected layer for dimensionality reduction. By learning the nonlinear interaction relationship between different feature modalities, the feature space is compressed to a preset length, redundant information is removed and key discriminative features are retained, and the spatiotemporal feature vector is constructed.

9. The deep learning-based infusion anomaly detection system according to claim 4, characterized in that, The process of generating the anomaly confidence level based on the probability distribution value specifically includes: Obtain the probability values ​​corresponding to the three abnormal states of bubble injection, blood reflux and needle blockage from the probability distribution values, calculate the weighted sum of the probability values ​​of these three abnormal states, use the weighted sum as the preliminary abnormal score, and generate the basic abnormal index. The information entropy of the probability distribution value is calculated. The information entropy is used to measure the uncertainty of the model for the current classification result. If the information entropy is higher than the preset uncertainty threshold, the basic anomaly index is attenuated and corrected using a penalty factor to generate an anomaly index. The abnormal index is weighted and averaged with the abnormal index of historical time steps using a smoothing filtering algorithm to eliminate the random fluctuation interference of single-frame prediction, thereby obtaining a stable value that reflects the abnormal trend of infusion status in the current period and generating the abnormal confidence level.

10. The deep learning-based infusion anomaly detection system according to claim 5, characterized in that, The process of determining whether the difference exceeds a preset threshold specifically includes: The system obtains the current operating environment parameters, dynamically adjusts the judgment sensitivity based on the ambient light intensity and the viscosity properties of the infusion fluid, matches the corresponding current judgment threshold from the preset threshold lookup table, and generates an adaptive threshold. Calculate the absolute value of the difference between the anomaly confidence level and the preset safety benchmark value, and compare the absolute value of the difference with the adaptive threshold. If the absolute value of the difference is strictly greater than the adaptive threshold, it is determined that there is a substantial risk of abnormality in the current infusion state, and a positive judgment result is generated. If a positive judgment result is generated, the number of consecutive frames that generate positive judgment results is further counted. Only when the number of consecutive frames exceeds the preset time window length is the abnormal state confirmed to continue to exist, in order to prevent misjudgment caused by instantaneous interference and generate a difference limit exceeding signal.

Images