Smart infusion stand with remote monitoring

Abstract

The present application discloses a remotely monitored intelligent infusion stand, which analyzes the infusion status video by using an image processing and analysis algorithm based on artificial intelligence, so as to learn and capture the infusion status characteristics of the infusion drip bucket of the patient during the infusion process, thereby using the idea of ​​inter-frame difference to determine whether liquid surface vibration occurs to extract the drip rate timing characteristics during the infusion process, so as to determine whether there is an abnormality in the infusion drip rate state. When an abnormal infusion situation is detected (such as too fast / too slow drip rate, insufficient infusion volume, etc.), the alarm controller will send an infusion alarm prompt signal to the medical staff for corresponding processing, providing a more convenient infusion experience for the medical staff and patients. In this way, it can not only help the medical staff to understand the infusion situation in real time, but also provide an automated remote monitoring function, which helps the medical staff to promptly discover and handle abnormal situations during the infusion process, and improve the safety and reliability of the infusion process.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring, and more specifically, to an intelligent infusion stand capable of remote monitoring. Background Art

[0002] Infusion is one of the common treatment methods in the medical process. It delivers drugs, nutrients and other solutions into the patient's body through intravenous infusion to play a therapeutic and supportive role. Traditional infusion monitoring methods mainly rely on regular inspections and observations by medical staff to ensure that the infusion speed and volume are within the normal range, and to check the patency of the infusion line and the safety of the infusion.

[0003] However, with the increase in medical needs and the number of patients, the traditional method of relying on manual monitoring has gradually exposed some limitations and problems. For example, medical staff need to regularly patrol the ward to check the patient's infusion status and situation, which not only consumes a lot of time and human resources, but also easily leads to hidden infusion problems that cannot be discovered in time due to human negligence, thus affecting the ability to respond to these problems in a timely manner. In addition, the traditional infusion monitoring method is limited to the location of the medical staff, and it is impossible to achieve remote monitoring of the infusion process, which is particularly inconvenient in some special circumstances (such as isolation wards and special care areas), limiting the work flexibility of medical staff and the treatment experience of patients.

[0004] Therefore, a remotely monitored intelligent infusion stand is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a remotely monitored intelligent infusion stand, which analyzes the infusion status video by using an image processing and analysis algorithm based on artificial intelligence, so as to learn and capture the infusion status characteristics of the infusion drop bucket of the patient during the infusion process, thereby using the idea of ​​inter-frame difference to determine whether the liquid surface vibration occurs to extract the drip rate timing characteristics during the infusion process, so as to determine whether there is an abnormality in the infusion drip rate state. When an abnormal infusion situation is detected (such as too fast / too slow drip rate, insufficient infusion volume, etc.), the alarm controller will send an infusion alarm prompt signal to the medical staff for corresponding processing, providing a more convenient infusion experience for the medical staff and patients. In this way, it can not only help the medical staff to understand the infusion situation in real time, but also provide an automated remote monitoring function, which helps the medical staff to promptly discover and deal with abnormal situations during the infusion process, and improve the safety and reliability of the infusion process.

[0006] According to one aspect of the present application, a remotely monitored intelligent infusion stand is provided, comprising:

[0007] An infusion stand body, the infusion stand body comprising a base, an infusion pole extending upward from the base, a lifting rod for controlling the height of the infusion pole, and a hook installed on the infusion pole;

[0008] A camera is arranged on the main body of the infusion stand, and the camera is used to collect an infusion status video of the infusion drop bucket;

[0009] an infusion status abnormality detector, the infusion status abnormality detector being communicatively connected to the camera to receive the infusion status video, and detecting an infusion abnormality according to the infusion status video of the infusion drop bucket to obtain a detection result;

[0010] An alarm controller is communicatively connected to the infusion status abnormality detector, and is used for sending an infusion alarm prompt signal to medical personnel in response to the detection result that the infusion status is abnormal.

[0011] Compared with the prior art, the present application provides a remotely monitored intelligent infusion stand, which analyzes the infusion status video by using an image processing and analysis algorithm based on artificial intelligence, so as to learn and capture the infusion status characteristics of the infusion drop bucket of the patient during the infusion process, thereby using the idea of ​​inter-frame difference to determine whether liquid surface vibration occurs to extract the drip rate timing characteristics during the infusion process, so as to determine whether there is an abnormality in the infusion drip rate state. When an abnormal infusion situation is detected (such as too fast / too slow drip rate, insufficient infusion volume, etc.), the alarm controller will send an infusion alarm prompt signal to the medical staff for corresponding processing, providing a more convenient infusion experience for medical staff and patients. In this way, it can not only help medical staff understand the infusion situation in real time, but also provide an automated remote monitoring function, which helps medical staff to promptly discover and handle abnormal situations during the infusion process, and improve the safety and reliability of the infusion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0013] Figure 1 is a block diagram of a remotely monitored intelligent infusion stand according to an embodiment of the present application;

[0014] Figure 2 A schematic diagram of data flow of a remotely monitored intelligent infusion stand according to an embodiment of the present application;

[0015] Figure 3 A block diagram of the training phase of a remotely monitored intelligent infusion stand according to an embodiment of the present application;

[0016] Figure 4 The present invention is a block diagram of an abnormal infusion status detector in a remotely monitored intelligent infusion stand according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0018] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0019] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0020] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Traditional infusion monitoring methods are limited to the location of medical staff, and cannot achieve remote monitoring of the infusion process. This is particularly inconvenient in some special situations (such as isolation wards and special care areas), limiting the work flexibility of medical staff and the treatment experience of patients. Therefore, a smart infusion stand that can be remotely monitored is desired.

[0023] In the technical solution of the present application, a remotely monitored intelligent infusion stand is proposed. Figure 14 is a block diagram of a remotely monitored intelligent infusion stand according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a remotely monitored intelligent infusion stand according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, a remotely monitored smart infusion stand 300 includes: an infusion stand body 310, wherein the infusion stand body includes a base, an infusion pole extending upward from the base, a lifting pole for controlling the height of the infusion pole, and a hook installed on the infusion pole; a camera 320 arranged on the infusion stand body, wherein the camera is used to collect an infusion status video of an infusion drop bucket; an infusion status abnormality detector 330, wherein the infusion status abnormality detector is communicatively connected to the camera to receive the infusion status video, and detects an infusion abnormality according to the infusion status video of the infusion drop bucket to obtain a detection result; an alarm controller 340, wherein the alarm controller is communicatively connected to the infusion status abnormality detector, and wherein the alarm controller is used to send an infusion alarm prompt signal to medical staff in response to the detection result that the infusion status is abnormal.

[0024] In particular, the infusion stand body 310 includes a base, an infusion pole extending upward from the base, a lifting rod for controlling the height of the infusion pole, and a hook installed on the infusion pole. It is made of lightweight and high-strength materials and has an infusion alarm installation interface for easy installation and removal.

[0025] In particular, the camera 320 provided on the infusion stand body is used to collect the infusion status video of the infusion drip bucket. It should be understood that the infusion status video of the infusion drip bucket can display the dripping speed of the liquid in the infusion drip bucket. By extracting the dripping speed timing characteristics during the infusion process, it is determined whether there is an abnormality in the infusion dripping speed state.

[0026] In particular, the infusion status abnormality detector 330 is communicatively connected to the camera to receive the infusion status video, and detects the infusion abnormality according to the infusion status video of the infusion bucket to obtain a detection result. In particular, in a specific example of the present application, Figure 4As shown, the infusion state abnormality detector 330 includes: an infusion state video key frame sampling module 331, which is used to sample the infusion state video key frames of the infusion dropper at a predetermined sampling frequency to obtain a sequence of infusion state key frames; an infusion state semantic feature extraction module 332, which is used to extract features based on the infusion state of the sequence of infusion state key frames to obtain a sequence of infusion state semantic feature maps; an infusion state semantic global perception enhancement module 333, which is used to respectively enhance each infusion state semantic feature map in the sequence of infusion state semantic feature maps through a saliency feature enhancer based on global semantic perception to obtain an enhanced infusion state. A sequence of semantic feature maps; an infusion drip rate semantic representation module 334, used to calculate the differential feature map between each adjacent two enhanced infusion state semantic feature maps in the sequence of enhanced infusion state semantic feature maps to obtain a sequence of infusion drip rate semantic representation feature maps; a drip rate full-time domain semantic interactive fusion module 335, used to pass the sequence of infusion drip rate semantic representation feature maps through a drip rate full-time domain semantic interactive fusion device based on a gated attention screening mechanism to obtain an infusion drip rate semantic full-time domain aggregate feature map; an infusion drip rate state abnormality detection module 336, used to determine whether there is an abnormality in the infusion drip rate state based on the infusion drip rate semantic full-time domain aggregate feature map.

[0027] Specifically, the infusion state video key frame sampling module 331 is used to sample the infusion state video key frames of the infusion drop bucket at a predetermined sampling frequency to obtain a sequence of infusion state key frames. Considering that there will be a large amount of information in the infusion state video, if the feature extraction and analysis are performed on the entire video, a large amount of redundant information will be generated, which also reduces the processing efficiency. In addition, considering that the liquid dripping rate has certain periodic characteristics during the infusion process, therefore, in the technical solution of the present application, the infusion state video key frames of the infusion drop bucket are sampled at a predetermined sampling frequency to obtain a sequence of infusion state key frames. It should be understood that if the sampling frequency is too high, the system needs to process a large amount of video data, which will increase the consumption of computing resources and reduce the response speed of the system. On the contrary, if the sampling frequency is too low, some key changes in the infusion state may be missed, resulting in inaccurate monitoring. Therefore, choosing a suitable sampling frequency is the key to balancing the data processing burden and monitoring accuracy. Based on this, the predetermined sampling frequency in the technical solution of the present application is obtained by deep neural network learning, which can use the dynamic changes in the drip rate in the drip bucket during the infusion process to select an adaptive sampling frequency so that it includes key frame images in each cycle of infusion dripping. It is worth mentioning that the key frame images here include images of droplets formed but not yet falling, images of droplets dripping in the tube but not reaching the liquid surface, images of droplets falling to the liquid surface causing the liquid surface to vibrate, etc. Each vibration of the liquid surface is taken as a cycle to capture these key frame images in each cycle, so as to facilitate the subsequent effective feature analysis and capture of the infusion drip rate state, and provide a basis for the detection task of whether there is an abnormality in the infusion drip rate state.

[0028] Specifically, the infusion state semantic feature extraction module 332 is used to perform feature extraction based on the infusion state on the sequence of the infusion state key frames to obtain a sequence of infusion state semantic feature maps. In a specific example of the present application, the sequence of the infusion state key frames is subjected to contrast normalization processing to obtain a sequence of contrast normalized infusion state key frames. Considering that under different lighting conditions, the contrast of the image frame may be significantly different. Therefore, in order to reduce the image quality difference caused by the change of illumination, thereby improving the overall quality of the image, in the technical solution of the present application, the sequence of the infusion state key frames is further subjected to contrast normalization processing to obtain a sequence of contrast normalized infusion state key frames. Contrast normalization processing of the sequence of the infusion state key frames can extract the local contrast features of the image, reduce the impact of the global contrast change in the image, and highlight the details and key feature information of the image, so that the liquid surface and other related features of the infusion drop bucket are more obvious. This helps the subsequent image processing and analysis algorithms to more accurately identify and understand the infusion state and drip rate semantics. Specifically, the contrast normalization processing method is to subtract the local mean of each pixel value of the image frame and divide it by the local standard deviation to achieve contrast normalization. Furthermore, the sequence of contrast-normalized infusion state key frames is passed through an infusion state feature extractor based on a hole convolutional neural network model to obtain a sequence of infusion state semantic feature maps. That is, the sequence of contrast-normalized infusion state key frames is subjected to feature mining in an infusion state feature extractor based on a hole convolutional neural network model to extract the implicit semantic feature information about the infusion state in each contrast-normalized infusion state key frame, thereby obtaining a sequence of infusion state semantic feature maps. It is worth mentioning that the hole convolutional neural network is a convolutional neural network structure that increases the receptive field by inserting holes (or dilation rates) between convolution kernels while maintaining the computational efficiency of the network. Hollow convolution can expand the field of view of the convolution kernel without increasing parameters and computational complexity, thereby better capturing long-distance contextual information.

[0029] Specifically, the infusion state semantic global perception enhancement module 333 is used to pass each infusion state semantic feature map in the sequence of the infusion state semantic feature maps through a salient feature emphasizer based on global semantic perception to obtain a sequence of enhanced infusion state semantic feature maps. Considering that in the actual process of monitoring the infusion state, the semantics expressed in the infusion state key frame includes multiple types of image features, and in order to further enhance the key features related to the infusion state in the image, such as the vibration state characteristics of the liquid surface, the droplet formation state characteristics, the droplet dripping state characteristics in the tube, etc., so as to improve the system's recognition and response capabilities to changes in the infusion state, in the technical solution of the present application, the sequence of the infusion state semantic feature maps is further passed through a salient feature emphasizer based on global semantic perception to obtain a sequence of enhanced infusion state semantic feature maps. The salient feature emphasizer based on global semantic perception can learn and capture the overall feature information and contextual semantic interaction in the image, so as to learn the key features related to the subsequent infusion drip rate state detection task, so as to adaptively express the saliency of these key features, thereby helping to detect the abnormal infusion state more comprehensively and significantly. Specifically, the salient feature emphasizer introduces a learnable linear layer to perform a linear transformation on the feature map to extract the significant infusion state features in the image, that is, the weight distribution generated by the linear transformation represents the feature importance of each spatial position, and the features of each spatial position are weighted according to the importance of the semantic representation of the infusion state in the image to obtain the global channel features of the infusion state semantic feature map. Then, the activation function is used to activate the global channel features, so that the features have more nonlinear expression capabilities, and the activated global channel features are superimposed on the original infusion state semantic features, so that the system can better capture the key features of the infusion state and improve the accuracy and reliability of abnormal infusion state detection.

[0030] In an embodiment of the present application, each infusion state semantic feature map in the sequence of the infusion state semantic feature map is respectively subjected to a salient feature emphasizer based on global semantic perception to obtain a sequence of enhanced infusion state semantic feature maps, including: performing a linear transformation on the infusion state semantic feature map to obtain an infusion state semantic linear transformation matrix; The function performs soft maximum normalization processing to obtain the infusion state semantic linear transformation weight matrix; each feature matrix along the channel dimension in the infusion state semantic feature map is multiplied by the infusion state semantic linear transformation weight matrix by position to obtain the infusion state semantic global channel feature map; the infusion state semantic global channel feature map is converted into After activation processing is performed in the activation function, the activation function is multiplied by the trainable scaling factor, and is added to the infusion state semantic feature map by position to obtain the enhanced infusion state semantic feature map.

[0031] Among them, the process of linearly transforming the infusion state semantic feature map to obtain the infusion state semantic linear transformation matrix includes: processing the infusion state semantic feature map through a convolutional layer to obtain an infusion state semantic feature matrix; after multiplying the infusion state semantic feature matrix with the weight matrix, the obtained weight-modulated infusion state semantic feature matrix and the bias matrix are summed by position to obtain the infusion state semantic linear transformation matrix.

[0032] In summary, in the above embodiment, each infusion state semantic feature map in the sequence of the infusion state semantic feature maps is processed by a salient feature emphasizer based on global semantic perception to obtain a sequence of enhanced infusion state semantic feature maps, including: each infusion state semantic feature map in the sequence of the infusion state semantic feature maps is processed by the salient feature emphasizer based on global semantic perception with the following global semantic perception enhancement formula to obtain the sequence of enhanced infusion state semantic feature maps; wherein the global semantic perception enhancement formula is:

[0033] in, is each infusion state semantic feature graph in the sequence of infusion state semantic feature graphs, To perform convolution operation on the feature map, is the weight matrix, is the bias matrix, express function, To multiply by position, is the semantic global channel feature map of infusion status, express Activation function, is the trainable scaling factor, To add by position, Each enhanced infusion state semantic feature graph in the sequence of enhanced infusion state semantic feature graphs.

[0034] Specifically, the infusion dripping speed semantic representation module 334 is used to calculate the differential feature map between each adjacent two enhanced infusion state semantic feature maps in the sequence of the enhanced infusion state semantic feature map to obtain a sequence of infusion dripping speed semantic representation feature maps. It should be understood that each enhanced infusion state semantic feature map in the sequence of the enhanced infusion state semantic feature map contains the infusion state semantic features in each key frame after the feature global and significant enhanced expression, and in order to explore the infusion dripping speed semantic information contained between the infusion state semantic features in these key frames, it is necessary to explore the semantic association features and implicit interaction relationships between each adjacent enhanced infusion state semantic feature map. Based on this, in the technical solution of the present application, the differential feature map between each adjacent two enhanced infusion state semantic feature maps in the sequence of the enhanced infusion state semantic feature map is calculated to obtain a sequence of infusion dripping speed semantic representation feature maps. By calculating the differential feature map, the dynamically changing parts and semantic information in the adjacent key frames can be captured, rather than the static background. In this way, the dynamic semantics and speed fluctuations of the infusion drip rate can be captured, thereby more accurately reflecting the dynamic changes in the infusion process. Such changes and characteristics are key information in the infusion process, which helps the system to monitor and analyze the infusion status in real time, detect abnormal infusion conditions, and reduce the interference of static features on infusion status monitoring.

[0035] Specifically, the drip rate full-time domain semantic interaction fusion module 335 is used to pass the sequence of the infusion drip rate semantic representation feature map through the drip rate full-time domain semantic interaction fusion device based on the gated attention screening mechanism to obtain the infusion drip rate semantic full-time domain aggregation feature map. Considering that each infusion drip rate semantic representation feature map in the sequence of the infusion drip rate semantic representation feature map reflects the infusion drip rate semantic features in different local time periods respectively, and these infusion drip rate semantic features have a time-series-based correlation relationship and mutual influence. At the same time, there are different importance between the infusion drip rate semantic features in different local time periods, that is, they have different contributions to the subsequent infusion state abnormality detection task. Based on this, in the technical solution of the present application, the sequence of the infusion drip rate semantic representation feature map is further passed through the drip rate full-time domain semantic interaction fusion device based on the gated attention screening mechanism to obtain the infusion drip rate semantic full-time domain aggregation feature map. Through the processing of the full-time domain semantic interaction fusion device of the drip rate based on the gated attention screening mechanism, the implicit correlation features and interactive relationships between the semantics of the infusion drip rate in different local time periods can be learned, and the differences between these features can be evaluated to identify which drip rate semantics are more important for subsequent detection tasks, so as to use the gated attention mechanism to dynamically select and emphasize those features that are most important for the recognition of the infusion drip rate state. In this way, the drip rate semantics in each local time period after the significant enhancement expression of the important drip rate semantic features are aggregated to capture and learn the changing trends and patterns of the infusion drip rate in the entire infusion process. Through the semantic interaction fusion of the full-time domain of the drip rate, the system can more accurately identify the abnormal state of the infusion drip rate. This full-time domain feature aggregation helps to reduce false positives and false negatives and improve the accuracy of detection.

[0036] In an embodiment of the present application, the sequence of the infusion drip rate semantic representation feature graphs is passed through a drip rate full-time domain semantic interactive fusion device based on a gated attention screening mechanism to obtain an infusion drip rate semantic full-time domain aggregated feature graph, including: calculating the semantic difference coefficients between each infusion drip rate semantic representation feature graph in the sequence of the infusion drip rate semantic representation feature graphs relative to other infusion drip rate semantic representation feature graphs to obtain a sequence of infusion drip rate semantic difference coefficients; normalizing the sequence of infusion drip rate semantic difference coefficients to obtain a sequence of infusion drip rate semantic difference weights; using the sequence of infusion drip rate semantic difference weights as weight gating, calculating the weighted sum of the sequence of the infusion drip rate semantic representation feature graphs to obtain the infusion drip rate semantic full-time domain aggregated feature graph.

[0037] Among them, the process of calculating the semantic difference coefficients between each infusion drip rate semantic representation feature graph in the sequence of infusion drip rate semantic representation feature graphs relative to other infusion drip rate semantic representation feature graphs to obtain a sequence of infusion drip rate semantic difference coefficients includes: extracting a predetermined infusion drip rate semantic representation feature graph from the sequence of infusion drip rate semantic representation feature graphs; calculating the logarithmic function value with base 2 of each position feature value of other infusion drip rate semantic representation feature graphs in the sequence of infusion drip rate semantic representation feature graphs except the predetermined infusion drip rate semantic representation feature graph to obtain a sequence of infusion drip rate semantic logarithmic representation feature graphs; calculating the A sequence of infusion drop rate semantic interaction feature graphs is obtained by multiplying each infusion drop rate semantic logarithmic representation feature graph in the sequence of infusion drop rate semantic logarithmic representation feature graphs with the predetermined infusion drop rate semantic representation feature graph by position point; a global mean between the feature values ​​at each position of each infusion drop rate semantic interaction feature graph in the sequence of infusion drop rate semantic interaction feature graphs is calculated to obtain an infusion drop rate semantic interaction global mean feature vector; a global mean between the feature values ​​at each position in the infusion drop rate semantic interaction global mean feature vector is calculated to obtain an infusion drop rate semantic difference coefficient corresponding to the predetermined infusion drop rate semantic representation feature graph.

[0038] In summary, in the above embodiment, the sequence of the infusion drip rate semantic representation feature map is passed through the drip rate full-time domain semantic interaction fuser based on the gated attention screening mechanism to obtain the infusion drip rate semantic full-time domain aggregation feature map, including: the sequence of the infusion drip rate semantic representation feature map is passed through the drip rate full-time domain semantic interaction fuser based on the gated attention screening mechanism with the following drip rate full-time domain semantic interaction fusion formula to obtain the infusion drip rate semantic full-time domain aggregation feature map; wherein the adaptive fusion formula is:

[0039] in, and are the first and The value of each position of the infusion drip rate semantic representation feature map, , and are the width, height and number of channels of the semantic representation feature map of the infusion drip rate, respectively. represents a sequence of semantic representation feature graphs of the infusion drip rate, The number of feature graphs in the sequence of feature graphs representing the infusion drip rate semantics is reduced by one, The semantic difference coefficients of each infusion drip rate in the sequence representing the semantic difference coefficients of the infusion drip rate, is the length of the sequence of semantic difference coefficients of the infusion drip rate, represents the value of an exponential function with a constant as the base, The semantic difference weights of each infusion drip rate in the sequence representing the semantic difference weights of infusion drip rate, It is the semantic full-time domain aggregated feature map of the infusion drip rate.

[0040] Specifically, the infusion drip speed state abnormality detection module 336 is used to determine whether the infusion drip speed state is abnormal based on the infusion drip speed semantic full-time domain aggregation feature map. In a specific example of the present application, the infusion drip speed semantic full-time domain aggregation feature map is passed through the infusion state abnormality identifier based on the classifier to obtain a detection result, and the detection result is used to indicate whether the infusion drip speed state is abnormal. That is, the full-time domain aggregation representation feature of the infusion drip speed semantics is used for classification processing to determine whether the infusion drip speed state is abnormal. When an abnormal infusion situation (such as too fast / too slow drip speed, insufficient infusion volume, etc.) is detected, the alarm controller will send an infusion alarm prompt signal to the medical staff for corresponding processing, providing a more convenient infusion experience for medical staff and patients. In this way, it can not only help medical staff understand the infusion situation in real time, but also provide an automated remote monitoring function, which helps medical staff to promptly discover and handle abnormal situations during the infusion process, and improve the safety and reliability of the infusion process.

[0041] In a preferred example, the sequence of enhanced infusion state semantic feature maps expresses the image semantic features of the infusion state key frames based on the global image semantic significance enhancement. Thus, the sequence of infusion drip rate semantic representation feature maps obtained by calculating the differential feature map between each two adjacent enhanced infusion state semantic feature maps is then passed through the drip rate full-time domain semantic interaction fusion device based on the gated attention screening mechanism. The obtained infusion drip rate semantic full-time domain aggregate feature map will also have a regression-like interpretability loss due to the interaction of complex image semantic multi-dimensional features based on the full-time domain image semantic feature interaction attention difference in the temporal direction, affecting the accuracy of the classification results. Based on this, the present application optimizes the infusion drip rate semantic full-time domain aggregate feature map.

[0042] That is, the infusion drip rate semantic full-time domain aggregated feature map is passed through an infusion state abnormality identifier based on a classifier to obtain a detection result, including: optimizing the infusion drip rate semantic full-time domain aggregated feature map to obtain an optimized infusion drip rate semantic full-time domain aggregated feature map; passing the optimized infusion drip rate semantic full-time domain aggregated feature map through an infusion state abnormality identifier based on a classifier to obtain a detection result.

[0043] Specifically, the infusion drip rate semantic full-time domain aggregated feature map is optimized to obtain an optimized infusion drip rate semantic full-time domain aggregated feature map, including the steps of:

[0044] Expanding the infusion drip rate semantic full-time domain aggregated feature map into an infusion drip rate semantic full-time domain aggregated feature vector;

[0045] The autocorrelation matrix of the full-time domain aggregated feature vector of the infusion drip rate semantics is calculated, namely:

[0046] in, represents the semantic full-time domain aggregated feature vector of the infusion drip rate, represents the transpose of a vector, represents matrix multiplication, represents the autocorrelation matrix;

[0047] The infusion drip rate semantic full-time domain aggregated feature vector is query-multiplied with the self-association matrix to obtain an intermediate feature vector, namely:

[0048] in, represents the semantic full-time domain aggregated feature vector of the infusion drip rate, represents the autocorrelation matrix, represents matrix multiplication, represents the intermediate eigenvector;

[0049] Calculate the first Row vector and Inner product sum of row vectors distance, to obtain the first weight matrix and the second weight matrix , expressed as:

[0050] in, and The autocorrelation matrix is ​​represented by Row vector and row vector, represents the transpose of a vector, represents matrix multiplication, represents the autocorrelation matrix, represents the first weight matrix, represents the first weight matrix The eigenvalues ​​of the positions, represents the second weight matrix, represents the second weight matrix The eigenvalues ​​of the positions, Represents the calculation vector distance;

[0051] The first weight matrix and the second weight matrix After matrix multiplication, further matrix multiplication is performed with the intermediate feature vector to obtain the optimized infusion drip rate semantic full-time domain aggregated feature vector, namely:

[0052] in, represents the first weight matrix, represents the second weight matrix, represents the intermediate eigenvector, represents matrix multiplication, Representing the optimized infusion drip rate semantic full-time domain aggregated feature vector;

[0053] The optimized infusion drip rate semantic full-time domain aggregated feature vector is restored to the optimized infusion drip rate semantic full-time domain aggregated feature map.

[0054] That is, for the bit-by-bit scattering association response space of the eigenvalues ​​of the infusion drip rate semantic full-time domain aggregated feature map, the deep embedding structure in the high-order association space is obtained through the joint embedding space of the vector inner product embedding space and the vector independent embedding space, and the decomposable association response sub-dimension of the bit-by-bit scattering association response space of the eigenvalues ​​of the infusion drip rate semantic full-time domain aggregated feature map is represented as a set of spatial sub-dimensions to further perform dependent spatial mapping of the decomposable association response sub-dimension of the deep embedding structure of the infusion drip rate semantic full-time domain aggregated feature map, so as to improve the interpretability of the solution space of the class regression solution of the infusion drip rate semantic full-time domain aggregated feature map based on the association distribution through the complex dimensional representation of the association response, thereby improving the accuracy of the classification result obtained by the infusion drip rate semantic full-time domain aggregated feature map.

[0055] In particular, the alarm controller 340 is communicatively connected to the infusion state abnormality detector, and the alarm controller is used to respond to the detection result that the infusion state is abnormal, and send an infusion alarm prompt signal to the medical staff. In a specific example of the present application, the alarm controller uses a high-sensitivity sensor to monitor the drip rate, remaining amount and other parameters during the infusion process in real time, and has a sound and light alarm function. In one example, the alarm controller is installed on an infusion stand. When an abnormal situation is detected (such as a drip rate that is too fast / too slow, insufficient infusion volume, etc.), the infusion alarm sends a sound and light alarm to remind medical staff to handle it.

[0056] It should be understood that before using the above-mentioned neural network model for inference, it is necessary to train the infusion state feature extractor based on the hole convolutional neural network model, the salient feature emphasizer based on global semantic perception, the drip rate full-time domain semantic interaction fusion device based on the gated attention screening mechanism, and the infusion state abnormality identifier based on the classifier. That is to say, the remotely monitored intelligent infusion stand 300 according to the present application also includes a training phase for training the infusion state feature extractor based on the hole convolutional neural network model, the salient feature emphasizer based on global semantic perception, the drip rate full-time domain semantic interaction fusion device based on the gated attention screening mechanism, and the infusion state abnormality identifier based on the classifier.

[0057] Figure 3 FIG. 1 is a block diagram of the training phase of the remotely monitored intelligent infusion stand according to an embodiment of the present application. Figure 3As shown, according to the embodiment of the present application, the remotely monitored intelligent infusion stand 300 includes: a training module 400, including: a training data acquisition unit 410, used to acquire training data, wherein the training data includes a training infusion state video of the infusion bucket collected by the camera; a training key frame sampling unit 420, used to sample the training infusion state video key frames of the infusion bucket at a predetermined sampling frequency to obtain a sequence of training infusion state key frames; a training infusion state semantic feature extraction unit 430, used to extract the sequence of training infusion state key frames based on the infusion state semantic feature extraction unit 430; The invention relates to a method for extracting the feature of the liquid state to obtain a sequence of training infusion state semantic feature maps; a training infusion state semantic global perception reinforcement unit 440, which is used to respectively pass each training infusion state semantic feature map in the sequence of the training infusion state semantic feature maps through a salient feature emphasizer based on global semantic perception to obtain a sequence of reinforced training infusion state semantic feature maps; a training infusion drip rate semantic representation unit 450, which is used to calculate the differential feature map between each adjacent two reinforced training infusion state semantic feature maps in the sequence of the reinforced training infusion state semantic feature maps to obtain a training infusion state semantic feature map. A sequence of drip rate semantic representation feature maps; a training drip rate full-time domain semantic interactive fusion unit 460, which is used to pass the sequence of training infusion drip rate semantic representation feature maps through a drip rate full-time domain semantic interactive fusion based on a gated attention screening mechanism to obtain a training infusion drip rate semantic full-time domain aggregate feature map; a classification loss unit 470, which is used to pass the training infusion drip rate semantic full-time domain aggregate feature map through an infusion state abnormality identifier based on a classifier to obtain a classification loss function value; a loss calculation unit 480, which is used to calculate the infusion drip rate semantic full-time domain aggregate loss term based on the infusion drip rate semantic full-time domain aggregate loss term. The method comprises the following steps: a weighting unit 490 for calculating the weighted sum of the classification loss function value and the infusion drip rate semantic full-time domain aggregation loss function value to obtain a final loss function value; a training unit 500 for training the infusion state feature extractor based on the hole convolutional neural network model, the saliency feature emphasizer based on global semantic perception, the drip rate full-time domain semantic interaction fuser based on the gated attention screening mechanism, and the infusion state abnormality identifier based on the classifier through gradient back propagation based on the final loss function value.

[0058] In a preferred example, the present application further introduces a predetermined loss function other than the classification loss function during the model training process, that is, training the model through gradient back propagation based on the loss function includes the following steps:

[0059] Expanding the training infusion drip rate semantic full-time domain aggregated feature map into a training infusion drip rate semantic full-time domain aggregated feature vector;

[0060] Calculate the first infusion drip rate semantic full-time domain aggregation weight matrix and the second infusion drip rate semantic full-time domain aggregation weight matrix based on the training infusion drip rate semantic full-time domain aggregation feature vector, wherein the first infusion drip rate semantic full-time domain aggregation weight matrix The feature value of the position is the first Eigenvalues ​​and The mean of the eigenvalues, and the second infusion drip rate semantic full time domain aggregation weight matrix The feature value of the position is the first Eigenvalues ​​and One-half the absolute value of the difference between the eigenvalues;

[0061] Performing a query matrix multiplication of the training infusion drip rate semantic full-time domain aggregation feature vector with the first infusion drip rate semantic full-time domain aggregation weight matrix and the second infusion drip rate semantic full-time domain aggregation weight matrix respectively to obtain the first infusion drip rate semantic full-time domain aggregation intermediate vector and the second infusion drip rate semantic full-time domain aggregation intermediate vector;

[0062] Calculate the vector inner product of the first infusion drip rate semantic full time domain aggregation intermediate vector and the second infusion drip rate semantic full time domain aggregation intermediate vector to obtain the first infusion drip rate semantic full time domain aggregation loss term;

[0063] The first infusion drip rate semantic full-time domain aggregation weight matrix and the second infusion drip rate semantic full-time domain aggregation weight matrix are matrix-multiplied, and the result matrix is ​​calculated. norm to obtain the full-time domain aggregation loss term of the second infusion drip rate semantics;

[0064] The first infusion drip rate semantic full-time domain aggregation loss term is subtracted from the product of a predetermined weight hyperparameter and the second infusion drip rate semantic full-time domain aggregation loss term to obtain the infusion drip rate semantic full-time domain aggregation loss function value; and based on the weighted sum of the infusion drip rate semantic full-time domain aggregation loss function value and the classification loss function value, the model parameters are optimized through gradient back propagation.

[0065] Among them, the semantic full-time domain aggregation loss function of the infusion drip rate is specifically expressed as:

[0066] in, The semantic full-time domain aggregated feature vector of the training infusion drip rate is: and They are the first infusion drip rate semantic full-time domain aggregation weight matrix and the second infusion drip rate semantic full-time domain aggregation weight matrix, and The first infusion drip rate semantic full-time domain aggregation weight matrix and the second infusion drip rate semantic full-time domain aggregation weight matrix are respectively The eigenvalues ​​of the positions, and are the first and second time-domain aggregated feature vectors of the semantics of the infusion drip rate training. Eigenvalues ​​and Eigenvalues, is matrix multiplication, To calculate the matrix norm, is the predetermined weight hyperparameter, It is the semantic full-time domain aggregation loss function value of infusion drip rate.

[0067] That is, in the above preferred example, the infusion drip rate semantic full-time domain aggregation loss function uses the structured feature representation of the short-range-long-range cross-scale detail link of the training infusion drip rate semantic full-time domain aggregation feature vector to perform the query composition of the detail inner product space within the training infusion drip rate semantic full-time domain aggregation feature vector, so as to approximate the low-rank independent observable composition of the link detail composition provided by the structured detail interaction of the training infusion drip rate semantic full-time domain aggregation feature vector. In this way, by training with the infusion drip rate semantic full-time domain aggregation loss function, the distributed detail group of the training infusion drip rate semantic full-time domain aggregation feature vector can be used to decompose the detail group on the basis of detail complexity, so as to promote the class regression decomposition recognition of the complex feature structure of the training infusion drip rate semantic full-time domain aggregation feature vector, and improve the classification training efficiency. In this way, the abnormal situation of the infusion drip rate state can be better detected, and timely alarm prompts and response processing can be provided, which not only provides an automated remote monitoring function, but also improves the safety and reliability of the infusion process.

[0068] As described above, the remotely monitorable intelligent infusion stand 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a remotely monitorable intelligent infusion algorithm. In a possible implementation, the remotely monitorable intelligent infusion stand 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the remotely monitorable intelligent infusion stand 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the remotely monitorable intelligent infusion stand 300 can also be one of the many hardware modules of the wireless terminal.

[0069] Alternatively, in another example, the remotely monitorable smart infusion stand 300 and the wireless terminal may be separate devices, and the remotely monitorable smart infusion stand 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0070] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A remotely monitored intelligent infusion stand, characterized in that: include: An infusion stand body, the infusion stand body comprising a base, an infusion pole extending upward from the base, a lifting rod for controlling the height of the infusion pole, and a hook installed on the infusion pole; A camera is arranged on the main body of the infusion stand, and the camera is used to collect an infusion status video of the infusion drop bucket; an infusion status abnormality detector, the infusion status abnormality detector being communicatively connected to the camera to receive the infusion status video, and detecting an infusion abnormality according to the infusion status video of the infusion drop bucket to obtain a detection result; An alarm controller, the alarm controller being communicatively connected to the infusion state abnormality detector, the alarm controller being used to send an infusion alarm prompt signal to medical personnel in response to the detection result indicating that the infusion state is abnormal; Wherein, the abnormal infusion state detector comprises: An infusion status video key frame sampling module, used for sampling the infusion status video key frames of the infusion dropper at a predetermined sampling frequency to obtain a sequence of infusion status key frames; An infusion state semantic feature extraction module, used for performing feature extraction based on the infusion state on the sequence of infusion state key frames to obtain a sequence of infusion state semantic feature graphs; An infusion state semantic global perception enhancement module, used for respectively passing each infusion state semantic feature graph in the sequence of infusion state semantic feature graphs through a salient feature enhancer based on global semantic perception to obtain a sequence of enhanced infusion state semantic feature graphs; An infusion drip rate semantic representation module, used for calculating a differential feature map between every two adjacent enhanced infusion state semantic feature maps in the sequence of enhanced infusion state semantic feature maps to obtain a sequence of infusion drip rate semantic representation feature maps; A drip rate full-time domain semantic interaction fusion module, used for passing the sequence of the infusion drip rate semantic representation feature graph through a drip rate full-time domain semantic interaction fusion device based on a gated attention screening mechanism to obtain an infusion drip rate semantic full-time domain aggregated feature graph; The infusion drip rate state abnormality detection module is used to determine whether the infusion drip rate state is abnormal based on the infusion drip rate semantic full-time domain aggregated feature map.

2. The remotely monitored intelligent infusion stand according to claim 1, characterized in that: The infusion status semantic feature extraction module is used to: performing contrast normalization processing on the sequence of infusion state key frames to obtain a sequence of contrast normalized infusion state key frames; The sequence of contrast-normalized infusion state key frames is passed through an infusion state feature extractor based on a dilated convolutional neural network model to obtain a sequence of infusion state semantic feature maps.

3. The remotely monitored intelligent infusion stand according to claim 2, characterized in that: The infusion status semantic global perception reinforcement module includes: A linear transformation unit, used for performing a linear transformation on the infusion state semantic feature map to obtain an infusion state semantic linear transformation matrix; A normalization unit is used to transform the infusion state semantic linear transformation matrix through The function performs soft maximum normalization processing to obtain the semantic linear transformation weight matrix of the infusion state; A semantic channel weighting unit, used for positionally multiplying each feature matrix along the channel dimension in the infusion state semantic feature map with the infusion state semantic linear transformation weight matrix to obtain an infusion state semantic global channel feature map; The infusion state semantic enhancement unit is used to enhance the infusion state semantic global channel feature map through After activation processing is performed in the activation function, the activation function is multiplied by the trainable scaling factor, and is added to the infusion state semantic feature map by position to obtain the enhanced infusion state semantic feature map.

4. The remotely monitored intelligent infusion stand according to claim 3, characterized in that: The linear transformation unit is used for: Processing the infusion state semantic feature map through a convolutional layer to obtain an infusion state semantic feature matrix; After multiplying the infusion state semantic feature matrix and the weight matrix, the obtained weight-modulated infusion state semantic feature matrix and the bias matrix are summed by position to obtain the infusion state semantic linear transformation matrix.

5. The remotely monitored intelligent infusion stand according to claim 4, characterized in that: The dripping speed full-time domain semantic interaction fusion module includes: A semantic difference measurement unit, used for calculating the semantic difference coefficient between each infusion drip rate semantic representation feature graph in the sequence of infusion drip rate semantic representation feature graphs relative to other infusion drip rate semantic representation feature graphs to obtain a sequence of infusion drip rate semantic difference coefficients; A semantic difference weight unit, used for normalizing the sequence of semantic difference coefficients of the infusion drip rate to obtain a sequence of semantic difference weights of the infusion drip rate; The infusion drip rate semantic feature full-time domain aggregation unit is used to use the sequence of the infusion drip rate semantic difference weights as weight gating, calculate the weighted sum of the sequence of the infusion drip rate semantic representation feature map to obtain the infusion drip rate semantic full-time domain aggregation feature map.

6. The remotely monitored intelligent infusion stand according to claim 5, characterized in that: The semantic difference measurement unit is used to: Extracting a predetermined infusion drip rate semantic representation feature graph from the sequence of infusion drip rate semantic representation feature graphs; Calculating the logarithmic function value with base 2 of each position feature value of other infusion drip rate semantic representation feature graphs in the sequence of infusion drip rate semantic representation feature graphs except the predetermined infusion drip rate semantic representation feature graph to obtain a sequence of infusion drip rate semantic logarithmic representation feature graphs; Calculate the point-by-point multiplication between each infusion drip rate semantic logarithmic representation feature graph in the sequence of infusion drip rate semantic logarithmic representation feature graphs and the predetermined infusion drip rate semantic representation feature graph to obtain a sequence of infusion drip rate semantic interaction feature graphs; Calculating the global mean between the feature values ​​of each position of each infusion drip rate semantic interaction feature graph in the sequence of infusion drip rate semantic interaction feature graphs to obtain an infusion drip rate semantic interaction global mean feature vector; The global mean between the feature values ​​at each position in the global mean feature vector of the infusion drip rate semantic interaction is calculated to obtain the infusion drip rate semantic difference coefficient corresponding to the predetermined infusion drip rate semantic representation feature map.

7. The remotely monitored intelligent infusion stand according to claim 6, characterized in that: The infusion drip rate state abnormality detection module is used to: pass the infusion drip rate semantic full-time domain aggregated feature map through a classifier-based infusion state abnormality identifier to obtain a detection result, and the detection result is used to indicate whether the infusion drip rate state is abnormal.

8. The remotely monitored intelligent infusion stand according to claim 7, characterized in that: It also includes a training module for training the infusion state feature extractor based on the hole convolutional neural network model, the saliency feature emphasizer based on global semantic perception, the drip rate full-time domain semantic interaction fuser based on the gated attention screening mechanism, and the infusion state abnormality identifier based on the classifier; Wherein, the training module includes: A training data acquisition unit, used to acquire training data, wherein the training data includes a training infusion status video of the infusion drop bucket collected by the camera; A training key frame sampling unit, used for sampling the training infusion state video key frames of the infusion drop bucket at a predetermined sampling frequency to obtain a sequence of training infusion state key frames; A training infusion state semantic feature extraction unit, used for performing infusion state-based feature extraction on the sequence of training infusion state key frames to obtain a sequence of training infusion state semantic feature graphs; A training infusion state semantic global perception reinforcement unit, used for respectively passing each training infusion state semantic feature graph in the sequence of training infusion state semantic feature graphs through a salient feature emphasizer based on global semantic perception to obtain a sequence of reinforced training infusion state semantic feature graphs; A training infusion drip rate semantic representation unit, used for calculating a differential feature map between every two adjacent enhanced training infusion state semantic feature maps in the sequence of enhanced training infusion state semantic feature maps to obtain a sequence of training infusion drip rate semantic representation feature maps; A training drip rate full-time domain semantic interaction fusion unit is used to pass the sequence of the training infusion drip rate semantic representation feature graph through a drip rate full-time domain semantic interaction fusion device based on a gated attention screening mechanism to obtain a training infusion drip rate semantic full-time domain aggregated feature graph; A classification loss unit, used for passing the training infusion drip rate semantic full-time domain aggregated feature map through a classifier-based infusion state abnormality identifier to obtain a classification loss function value; A loss calculation unit, used to calculate the infusion drip rate semantic full-time domain aggregation loss function value based on the infusion drip rate semantic full-time domain aggregation loss item; A weighting unit, used for calculating a weighted sum of the classification loss function value and the infusion drip rate semantic full time domain aggregation loss function value to obtain a final loss function value; A training unit is used to train the infusion state feature extractor based on the hole convolutional neural network model, the saliency feature emphasizer based on global semantic perception, the drip rate full-time domain semantic interaction fuser based on the gated attention screening mechanism, and the infusion state abnormality identifier based on the classifier through gradient back propagation based on the final loss function value.

Images