Recognition system based on AI visual infusion scene monitoring and infusion monitor equipment

Abstract

The invention discloses a recognition system based on AI visual infusion scene monitoring and infusion monitor equipment, and relates to the technical field of infusion scene monitoring, and the recognition system comprises an information acquisition unit, an image processing unit, a material analysis unit, a character detection unit, a character recognition unit, a medicine auditing unit and an infusion monitoring unit, according to the invention, Gaussian noise and impulse noise are added in initial image information through the image processing unit, and the two kinds of noise are most common in image information generated by actual shooting, so that the storage of the processed image information in the sample data set is greatly helpful for model training; the model can be more accurate when iteratively updating parameters, and the gray level corresponding to the image information in the sample data set is adjusted, so that the image information contained in the sample data set has diversity. And it is ensured that when the infusion bottle material identification model analyzes different types of image information, corresponding material information can be rapidly obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of infusion scene monitoring, and specifically to an identification system and an infusion monitoring device based on AI visual infusion scene monitoring. Background Art

[0002] The infusion monitoring system can synchronously display the progress information of the patient's infusion on the monitoring screen, monitor the status of the infusion in real time, and issue a prompt message when the infusion is completed, helping medical staff to complete the task of supervising the patient's infusion in a timely, accurate and efficient manner. The invention patent with application number CN202111551832.9 discloses "an infusion monitoring system and method based on conductivity detection for identifying basic infusion, the system includes a basic infusion type identification device, a drip rate monitoring device, an AC excitation source, an MCU and a monitoring server that wirelessly communicates with the MCU, the basic infusion type identification device It includes a first sensor component arranged on the outer wall of the upper or lower infusion pipeline, the dripping rate monitoring device includes a second sensor component arranged on the outer wall of the drip pot, the AC excitation source is respectively connected to the first sensor component and the second sensor component to apply an excitation voltage, the MCU is respectively connected to the first sensor component and the second sensor component, and the basic infusion type recognition, dripping rate monitoring and auxiliary abnormal alarm are realized by processing the collected signals. Compared with the prior art, the present invention has the advantages of being able to identify the basic infusion type, the system is reusable, simple to install, low cost and capable of realizing remote monitoring. ";

[0003] The above-mentioned prior art solves the problems of needing to manually check the drugs used for infusion and being unable to detect abnormal dripping rate in time. However, when the system is running, since the materials of different infusion bottles are not classified, there is a large error in the analysis of the actual weight of the liquid during the infusion process, making it impossible to detect the completion of the infusion in time during monitoring. At the same time, the system does not query the name of the infusion drug and the manufacturer, so that medical staff need to manually query the patient information corresponding to these drugs. Summary of the invention

[0004] The purpose of the present invention is to provide an identification system and an infusion monitoring device based on AI visual infusion scene monitoring to solve the problems raised in the above background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an identification system based on AI visual infusion scene monitoring, comprising a text recognition unit, a drug review unit and an infusion monitoring unit;

[0006] An information acquisition unit, wherein the information acquisition unit acquires the name and manufacturer of the corresponding medicine in the infusion list of each patient, transmits the same to a database, divides the materials of existing infusion bottles into three types, namely glass bottles, plastic bottles and plastic bags, extracts the materials corresponding to the medicine infusion bottles of different manufacturers, counts the capacity of each material, determines the weight of the infusion bottles corresponding to different capacities, and transmits the data in the form of drug name, infusion bottle material, capacity, manufacturer and weight to the database;

[0007] An image processing unit, wherein the image processing unit uses a camera to shoot infusion bottles of different materials, transmits the collected image information as initial image information to a sample data set, uses random variables to process multiple initial image information in the sample data set, obtains image information with Gaussian noise, stores it in the sample data set, counts the number of pixels corresponding to different gray levels in the image information, uses a distribution function to deduce a reference gray level, adjusts the gray level of the image information according to the reference gray level, and stores the processed image information in the sample data set;

[0008] A material analysis unit, wherein the material analysis unit uses the AlexNet neural network to construct an infusion bottle material recognition model, obtains multiple image information of the same type in a training set according to a preset value, and when transmitting the information as input to the model, uses the dropout technology to randomly select 20% of the neural networks in the two fully connected layers of the model for deletion, analyzes the image information through the remaining neural networks, iteratively updates the parameters of the model, restores the deleted neural networks, repeats the operation until the model training is completed, uses a camera to shoot the current infusion bottle, transmits the collected image information to the infusion bottle material recognition model for analysis, and outputs the obtained results;

[0009] A text detection unit, after extracting the current image information from the database, performs a sliding window operation on the feature map corresponding to the image information, selects ten bounding boxes in the central area of ​​each window to predict the text candidate area, transmits the feature vector corresponding to each row in the feature map to the recurrent neural network, analyzes it through the recurrent neural network to obtain the corresponding sequence features, uses softmax to calculate the ratio of the intersection area to the union area between the prior box corresponding to the feature map and each bounding box, if the obtained data is greater than the threshold, the corresponding bounding box is determined as a positive sample, and the offset analysis algorithm is used to analyze these bounding boxes to determine the position and height of the bounding boxes, and multiple text candidate areas with the smallest adjacent distances are spliced ​​to generate text lines.

[0010] Preferably, the image processing unit includes an initial sample acquisition module and a first noise adding module. The initial sample acquisition module uses a camera to shoot infusion bottles of different materials, and transmits the acquired image information as initial image information to the sample data set according to different material types. The first noise adding module uses a random number generation function to calculate two numerical values, and then converts them into Gaussian distributed random variables through the Box-Muller transformation formula. The random variables are used to process multiple initial image information in the sample data set to obtain image information with Gaussian noise. The Box-Muller transformation formula is a way to transform uniformly distributed random numbers into random numbers that obey normal distribution.

[0011] Preferably, the image processing unit further includes a second noise adding module and a grayscale analysis module, wherein the second noise adding module sets the signal-to-noise ratio corresponding to the image, and calculates the signal-to-noise ratio according to the total number of pixels contained in the image information in the sample data set, obtains the number of pixels for setting interference, and then uses the obtained number of pixels to determine the designated position of the pixel point in the image information, sets the pixel value of the corresponding position to zero, obtains the image information with impulse noise, and stores it in the sample data set; the grayscale analysis module counts the number of pixels corresponding to different grayscales in the image information, determines the pixel ratio corresponding to each grayscale, obtains the distribution function corresponding to the image information according to the pixel ratio, uses the distribution function to calculate the reference grayscale, and adjusts the grayscale of the image information to obtain the processed image information, which is stored in the sample data set, wherein the distribution function is specifically:

[0012]

[0013] Among them, n k represents the number of pixels of different gray levels in the original image, n represents the total number of pixels in the original image, k represents the number of the current gray level, L represents the number of gray levels, s k represents the distribution function and j represents the parameter.

[0014] Preferably, the material analysis unit includes a data set division module, a model construction module, a model optimization module, a model training module and a material determination module. The data set division module extracts 80% of the image information of each type in the sample data set and transmits it to the training data set, and the remaining image information is transmitted to the test data set. The model construction module uses the AlexNet neural network to construct an infusion bottle material recognition model. The infusion bottle material recognition model includes five convolutional layers and three fully connected layers. After the model optimization module sets the number of samples input each time, it obtains multiple image information of the same type in the training set according to a preset value, and when transmitting it as input to the model, it uses the dropout technology to randomly select 20% of the neural networks in the two fully connected layers for deletion, and backs up the deleted neural network parameters, and uses the remaining After the neural network analyzes the image information, the predicted value of the model is obtained. The model training module uses the loss function to analyze the difference between the model generation probability and the actual probability, sets the momentum parameter, and estimates the gradient in the network according to the corresponding target probability provided by the loss function. After iteratively updating the parameters of the model, the deleted neural network is restored, and the operation is repeated until the model training is completed. The material determination module uses a camera to shoot the current infusion bottle, stores the collected image information in a database, retrieves the current image information from the database, and transmits it to the infusion bottle material recognition model for analysis to obtain the probabilities corresponding to the three materials, and selects the material with the highest probability as the result for output. The AlexNet neural network is a deep convolutional neural network, and the dropout technology is mainly used to avoid the overfitting phenomenon of the neural network.

[0015] Preferably, the text detection unit includes a bounding box selection module and a sequence feature generation module. After the bounding box selection module extracts the current image information from the database, it uses the VGG16 convolutional neural network to analyze the current image information to obtain a feature map corresponding to the image information, and performs a sliding window operation in the feature map. The window size is first set to 3×3, and ten bounding boxes are selected in the center area of ​​each window to predict the text candidate area. The sequence feature generation module combines each window center with its surrounding area to generate a feature vector, and transmits the feature vector corresponding to each row in the feature map to the recurrent neural network, and analyzes it through the recurrent neural network to obtain the corresponding sequence feature. The VGG16 convolutional neural network is a deep convolutional neural network.

[0016] Preferably, the text detection unit further includes a limit frame position determination module and a text area splicing module. The limit frame position determination module uses a softmax function to delete the limit frames that exceed the image information boundary, and then calculates the ratio of the intersection area to the union area between the prior frame corresponding to the feature map and each limit frame. If the obtained data is greater than a threshold, the corresponding limit frame is determined as a positive sample. If the obtained data is less than the threshold but greater than the data obtained by the remaining limit frames, the corresponding limit frame is also determined as a positive sample. The offset analysis algorithm is used to analyze the limit frame to determine the position and height of the limit frame. After the text area splicing module analyzes the specific position of each text candidate area according to the position and height of the limit frame, multiple text candidate areas with the smallest adjacent distance are spliced ​​to generate a text line. The edge refinement algorithm is used to detect the surrounding area of ​​each text line, and the remaining adjacent text candidate areas are merged into the text line. The offset analysis algorithm is specifically as follows:

[0017]

[0018] Where, υ=(υ c ,υ h ) represents the relative prediction coordinate, υ′=(υ c ′,υ h ′) represents the prior coordinates, Indicates the y-axis center of the bounding box, h a Indicates the height of the bounding box, c y represents the predicted y coordinate in the input image, h represents the predicted height in the input image, and c′ y represents the y coordinate in the prior box, and h′ represents the height of the prior box.

[0019] Preferably, the text recognition unit includes a text sequence generation module, a text information processing module and a text sequence storage module. The text sequence generation module extracts all text candidate areas in the image information, sets the image information of these text candidate areas to the same height, and then uses a convolutional neural network model to analyze the image information to obtain a feature vector containing sequence information. The order of each frame in the feature map corresponds to the order of the text candidate areas. The feature vector is transmitted to a recurrent neural network for recognition to obtain a text sequence corresponding to multiple consecutive frames. The text information processing module uses a classifier to detect the text sequence, converts the space information contained in the sequence into a separator, and then traverses the text sequence again. Once repeated text information is found in the sequence, it is determined whether there is a separator between the repeated text information. If there is a separator between the repeated text information, the text information is retained. If there is no separator between the repeated text information, the text information at the front of the sequence is retained, and the remaining information is deleted. The processed text sequence is transmitted to the database. The text sequence storage module determines the drug name, manufacturer, expiration date and bottle capacity corresponding to the current image according to the recognized text sequence, and transmits it to the database.

[0020] Preferably, the drug review unit uses a database viewing tool to query the corresponding database according to the drug name and manufacturer. If the corresponding patient information is found, it will be output through a visual interface. If the corresponding patient information is not found, information that infusion is not allowed will be returned through the visual interface, and a prompt will be given in the form of a voice broadcast. The database viewing tool is specifically Navicat for MySQL query tool.

[0021] Preferably, the infusion monitoring unit includes an expiration date analysis module, a weight monitoring module, an end prompt module and a flow rate detection module. The expiration date analysis module uses a database viewing tool to query the database according to the drug name, infusion bottle material, capacity, and manufacturer to obtain the weight corresponding to the infusion bottle, and uses a processor to analyze whether the current time is within the expiration date range. If it is within the expiration date range, no operation is performed. Otherwise, the information that the infusion is not allowed is returned through the visual interface, and a prompt is given in the form of a voice broadcast. The weight monitoring module uses a weight sensor to monitor the actual weight of the infusion bottle during use, and transmits the obtained data and the corresponding time to the database. The end prompt module After the module obtains the data and time collected by the weight sensor from the database, it analyzes the remaining liquid weight of the current infusion bottle according to the corresponding weight of the infusion bottle and the complete weight when filled with liquid. If the remaining liquid weight is less than or equal to five grams, it will return the information that the infusion is about to be completed through the visual interface, and prompt it in the form of voice broadcast. Otherwise, the remaining liquid weight information will be returned through the visual interface. When the user needs to determine the infusion speed at that time, the flow rate detection module obtains the weight of the infusion bottle when currently filled with liquid and the weight of the infusion bottle when currently filled with liquid collected for the previous three times, and calculates the average speed and instantaneous speed of the current infusion according to the time interval for each weight sensor data collection, and outputs it through the visual interface.

[0022] An infusion monitoring device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an identification system based on AI visual infusion scene monitoring is implemented.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The present invention transmits the collected multiple image information as initial image information to the sample data set through the image processing unit, and adds Gaussian noise and impulse noise to the initial image information. Since these two types of noise are most common in the image information generated by actual shooting, storing the processed image information in the sample data set is of great help to model training, which makes it easier for the model to be more accurate when iteratively updating parameters. At the same time, the image information in the sample data set is adjusted to the grayscale level, so that the image information contained in the sample data set has diversity, ensuring that the infusion bottle material recognition model can quickly obtain the corresponding material information when analyzing different types of image information;

[0025] 2. The present invention can accurately analyze the specific location and content of the relevant information on the surface of the infusion bottle in the current image information through the text detection unit and the text recognition unit, so as to facilitate the subsequent query of patient-related information according to the drug name and manufacturer, and avoid the occurrence of incorrect use of drugs. At the same time, the weight information of the infusion bottle involved in the current image is accurately obtained according to the recognized text information and the infusion bottle material information obtained by the infusion bottle material recognition model analysis, and the gravity sensor is used to measure the complete weight of all the liquid in the infusion bottle. Through these data, the actual weight of the liquid can be accurately analyzed. When the infusion is about to be completed, the medical staff can be promptly prompted to replace it, and the flow rate of the liquid can also be monitored in real time during the infusion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Provides a schematic diagram of the overall system flow for an embodiment of the present invention;

[0027] Figure 2 An internal module block diagram of an image processing unit provided by an embodiment of the present invention;

[0028] Figure 3 A block diagram of the internal modules of a material analysis unit provided in an embodiment of the present invention;

[0029] Figure 4 A block diagram of the internal modules of a text detection unit provided in an embodiment of the present invention;

[0030] Figure 5 A block diagram of the internal modules of a text recognition unit provided by an embodiment of the present invention;

[0031] Figure 6 This is a block diagram of the internal modules of the infusion monitoring unit provided in an embodiment of the present invention.

[0032] In the figure: 1. Information acquisition unit; 2. Image processing unit; 201. Initial sample acquisition module; 202. First noise adding module; 203. Second noise adding module; 204. Grayscale analysis module; 3. Material analysis unit; 301. Data set division module; 302. Model building module; 303. Model optimization module; 304. Model training module; 305. Material determination module; 4. Text detection unit; 401. Limiting box selection module; 402. Sequence feature generation module; 403. Limiting box position determination module; 404. Text area splicing module; 5. Text recognition unit; 501. Text sequence generation module; 502. Text information processing module; 503. Text sequence storage module; 6. Drug review unit; 7. Infusion monitoring unit; 701. Expiration date analysis module; 702. Weight monitoring module; 703. End prompt module; 704. Flow rate detection module. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1-6 , the present invention provides a technical solution: an identification system based on AI visual infusion scene monitoring, including a text recognition unit 5, a drug review unit 6 and an infusion monitoring unit 7;

[0035] Information acquisition unit 1, information acquisition unit 1 obtains the corresponding drug name and manufacturer in each patient's infusion list, transmits it to the database, divides the materials of existing infusion bottles into three types, namely glass bottles, plastic bottles and plastic bags, extracts the materials corresponding to drug infusion bottles of different manufacturers, counts the capacity of each material, determines the weight of infusion bottles corresponding to different capacities, and transmits the data in the form of drug name, infusion bottle material, capacity, manufacturer and weight to the database;

[0036] Image processing unit 2, image processing unit 2 uses a camera to shoot infusion bottles of different materials, transmits the collected image information as initial image information to the sample data set, uses random variables to process multiple initial image information in the sample data set, obtains image information with Gaussian noise, stores it in the sample data set, counts the number of pixels corresponding to different gray levels in the image information, uses a distribution function to deduce a reference gray level, adjusts the gray level of the image information according to the reference gray level, and stores the processed image information in the sample data set;

[0037] Material analysis unit 3, material analysis unit 3 uses AlexNet neural network to build an infusion bottle material recognition model, obtains multiple image information of the same type in the training set according to the preset value, and transmits it to the model as input, and uses dropout technology to randomly select 20% of the neural networks in the two fully connected layers of the model for deletion, and after analyzing the image information through the remaining neural networks, iteratively updates the parameters of the model, restores the deleted neural networks, repeats the operation until the model training is completed, uses a camera to shoot the current infusion bottle, transmits the collected image information to the infusion bottle material recognition model for analysis, and outputs the obtained results;

[0038] The text detection unit 4 extracts the current image information from the database, performs a sliding window operation on the feature map corresponding to the image information, selects ten bounding boxes in the central area of ​​each window to predict the text candidate area, transmits the feature vector corresponding to each row in the feature map to the recurrent neural network, analyzes it through the recurrent neural network to obtain the corresponding sequence features, and uses softmax to calculate the ratio of the intersection area and the union area between the prior box corresponding to the feature map and each bounding box. If the obtained data is greater than the threshold, the corresponding bounding box is determined as a positive sample, and the offset analysis algorithm is used to analyze these bounding boxes to determine the position and height of the bounding boxes, and multiple text candidate areas with the smallest adjacent distances are spliced ​​to generate text lines.

[0039] The image processing unit 2 includes an initial sample acquisition module 201 and a first noise adding module 202. The initial sample acquisition module 201 uses a camera to shoot infusion bottles of different materials, and transmits the collected image information as initial image information to the sample data set according to different material types. The first noise adding module 202 uses a random number generation function to calculate two values, and then converts them into Gaussian distributed random variables through the Box-Muller transformation formula. The random variables are used to process multiple initial image information in the sample data set to obtain image information with Gaussian noise. The Box-Muller transformation formula is a way to transform uniformly distributed random numbers into random numbers that obey normal distribution.

[0040] The image processing unit 2 also includes a second noise adding module 203 and a grayscale analysis module 204. The second noise adding module 203 sets the signal-to-noise ratio corresponding to the image, and calculates the total number of pixels and the signal-to-noise ratio contained in the image information in the sample data set. After obtaining the number of pixels for setting interference, the obtained number of pixels is used to determine the designated position of the pixel point in the image information, and the pixel value of the corresponding position is set to zero to obtain image information with impulse noise, which is stored in the sample data set. The grayscale analysis module 204 counts the number of pixels corresponding to different grayscales in the image information, determines the pixel ratio corresponding to each grayscale, and obtains the distribution function corresponding to the image information according to the pixel ratio. The reference grayscale is calculated using the distribution function, and the grayscale of the image information is adjusted to obtain the processed image information, which is stored in the sample data set. The distribution function is specifically:

[0041]

[0042] Among them, n k represents the number of pixels of different gray levels in the original image, n represents the total number of pixels in the original image, k represents the number of the current gray level, L represents the number of gray levels, s k represents the distribution function, j represents the parameter;

[0043] The material analysis unit 3 includes a data set division module 301, a model construction module 302, a model optimization module 303, a model training module 304 and a material determination module 305. The data set division module 301 extracts 80% of the image information of each type in the sample data set and transmits it to the training data set, and the remaining image information is transmitted to the test data set. The model construction module 302 uses the AlexNet neural network to construct an infusion bottle material recognition model. The infusion bottle material recognition model includes five convolutional layers and three fully connected layers. After the model optimization module 303 sets the number of samples input each time, it obtains multiple image information of the same type in the training set according to the preset value. When it is transmitted to the model as input, the dropout technology is used to randomly select 20% of the neural networks in the two fully connected layers for deletion, and the deleted neural network parameters are backed up. After the neural network under the image information is analyzed, the predicted value of the model is obtained. After the model training module 304 uses the loss function to analyze the difference between the model generation probability and the actual probability, the momentum parameter is set, and the gradient in the network is estimated according to the corresponding target probability provided by the loss function. After iteratively updating the parameters of the model, the deleted neural network is restored, and the operation is repeated until the model training is completed. The material determination module 305 uses a camera to shoot the current infusion bottle, stores the collected image information in the database, retrieves the current image information from the database, and transmits it to the infusion bottle material recognition model for analysis to obtain the probabilities corresponding to the three materials, and selects the material with the highest probability as the result for output. The AlexNet neural network is a deep convolutional neural network. The dropout technology is mainly used to avoid the overfitting phenomenon of the neural network. The loss function is specifically:

[0044]

[0045] Among them, y i represents the true value, y p represents the predicted value, i represents the parameter, Loss c Represents the distance between the actual output probability and the expected output probability;

[0046] The text detection unit 4 includes a bounding box selection module 401 and a sequence feature generation module 402. After the bounding box selection module 401 extracts the current image information from the database, it uses the VGG16 convolutional neural network to analyze the current image information to obtain a feature map corresponding to the image information, and performs a sliding window operation in the feature map. The window size is first set to 3×3, and ten bounding boxes are selected in the center area of ​​each window to predict the text candidate area. The sequence feature generation module 402 combines each window center with its surrounding area to generate a feature vector, and transmits the feature vector corresponding to each row in the feature map to the recurrent neural network. The recurrent neural network analyzes it to obtain the corresponding sequence feature. The VGG16 convolutional neural network is a deep convolutional neural network.

[0047] The text detection unit 4 also includes a limit frame position determination module 403 and a text area splicing module 404. The limit frame position determination module 403 uses the softmax function to delete the limit frames that exceed the image information boundary, and then calculates the ratio of the intersection area and the union area between the prior frame corresponding to the feature map and each limit frame. If the obtained data is greater than the threshold, the corresponding limit frame is determined as a positive sample. If the obtained data is less than the threshold but greater than the data obtained by the remaining limit frames, the corresponding limit frame is also determined as a positive sample. The offset analysis algorithm is used to analyze it to determine the position and height of the limit frame. The text area splicing module 404 analyzes the specific position of each text candidate area according to the position and height of the limit frame, and then splices multiple text candidate areas with the smallest adjacent distance to generate a text line. The edge refinement algorithm is used to detect the surrounding area of ​​each text line, and the remaining adjacent text candidate areas are merged into the text line. The offset analysis algorithm is specifically as follows:

[0048]

[0049] Where, υ=(υ c ,υ h ) represents the relative prediction coordinate, υ′=(υ c ′,υ h ′) represents the prior coordinates, Indicates the y-axis center of the bounding box, h a Indicates the height of the bounding box, c y represents the predicted y coordinate in the input image, h represents the predicted height in the input image, and c′ y represents the y coordinate in the prior frame, and h′ represents the height of the prior frame;

[0050] The edge refinement algorithm is as follows:

[0051]

[0052] Among them, xside Indicates the x-coordinate of the closest horizontal edge to the current bounding box prediction, x s ' ide represents the actual edge coordinates of the x-axis, Indicates the x-axis center of the bounding box, w a represents the width of the bounding box, O represents the offset of the original edge candidate area, and O′ represents the offset of the optimized edge candidate area;

[0053] The text recognition unit 5 includes a text sequence generation module 501, a text information processing module 502 and a text sequence storage module 503. The text sequence generation module 501 extracts all text candidate areas in the image information, sets the image information of these text candidate areas to the same height, and then uses a convolutional neural network model to analyze these image information to obtain a feature vector containing sequence information. The sequence of each frame in the feature image corresponds to the sequence of the text candidate areas. The feature vector is transmitted to a recurrent neural network for recognition to obtain a text sequence corresponding to multiple consecutive frames. The text information processing module 502 detects the text sequence using a classifier, converts the space information contained in the sequence into a separator, and then traverses the text sequence again. Once repeated text information is found in the sequence, it is determined whether there is a separator between the repeated text information. If there is a separator between the repeated text information, the text information is retained. If there is no separator between the repeated text information, the text information at the front of the sequence is retained, and the remaining information is deleted. The processed text sequence is transmitted to the database. The text sequence storage module 503 determines the drug name, manufacturer, expiration date and bottle capacity corresponding to the current image according to the recognized text sequence, and transmits it to the database;

[0054] The drug review unit 6 uses a database viewing tool to query the corresponding database according to the drug name and manufacturer. If the corresponding patient information is found, it will be output through a visual interface. If the corresponding patient information is not found, the information that the infusion is not allowed will be returned through the visual interface, and a prompt will be given in the form of a voice broadcast. The database viewing tool is specifically Navicat for MySQL query tool;

[0055] The infusion monitoring unit 7 includes an expiration date analysis module 701, a weight monitoring module 702, an end prompt module 703 and a flow rate detection module 704. The expiration date analysis module 701 uses a database viewing tool to query the database according to the drug name, infusion bottle material, capacity, and manufacturer to obtain the weight corresponding to the infusion bottle, and uses a processor to analyze whether the current time is within the expiration date range. If it is within the expiration date range, no operation is performed. Otherwise, the information that the infusion is not allowed to be performed is returned through the visual interface, and a prompt is given in the form of a voice broadcast. The weight monitoring module 702 uses a weight sensor to monitor the actual weight of the infusion bottle during use, and transmits the obtained data and the corresponding time to the database, and ends the prompt. After the display module 703 obtains the data and time collected by the weight sensor from the database, it analyzes the remaining liquid weight of the current infusion bottle according to the corresponding weight of the infusion bottle and the complete weight when the liquid is contained. If the remaining liquid weight is less than or equal to five grams, the information that the infusion is about to be completed will be returned through the visual interface, and a prompt will be given in the form of a voice broadcast. Otherwise, the remaining liquid weight information will be returned through the visual interface. When the user needs to determine the infusion speed at the time, the flow rate detection module 704 obtains the weight of the infusion bottle when the liquid is currently contained and the weight of the infusion bottle when the liquid is currently contained collected for the previous three times, and then calculates the average speed and instantaneous speed of the current infusion according to the time interval for each weight sensor to collect data, and outputs it through the visual interface;

[0056] An infusion monitoring device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an identification system based on AI visual infusion scene monitoring is implemented.

[0057] Working principle: The present invention extracts the drug name and manufacturer corresponding to each patient's infusion list through the information acquisition unit 1, transmits the infusion bottle related data to the database, collects multiple image information as initial image information through the initial sample acquisition module 201 in the image processing unit 2 and stores it in the sample data set, adds noise to the initial image information using the first noise adding module 202 and the second noise adding module 203, and the grayscale analysis module 204 adjusts the grayscale corresponding to the image information in the sample data set, and the processed image information is stored in the sample data set, the data set division module 301 in the material analysis unit 3 divides the sample data set into a training data set and a test data set, and the model building module 302 generates an infusion bottle. Bottle material recognition model, model optimization module 303 uses dropout technology to randomly select 20% of neural networks in the two fully connected layers of the model for deletion, and obtains the prediction value of the model after analyzing the image information through the remaining neural networks. The model training module 304 iteratively updates the parameters of the model, and the material determination module 305 uses a camera to shoot the current infusion bottle to analyze the infusion bottle material in the image information. The limit frame selection module 401 in the text detection unit 4 selects ten limit frames in the central area of ​​multiple windows in the feature map corresponding to the image information to predict the text candidate area. The sequence feature generation module 402 obtains the corresponding sequence features, and uses the limit frame position determination module 403 to calculate the offset through the offset analysis. The method is used to analyze it, determine the position and height of the limit frame, detect the surrounding area of ​​each text line by using the edge refinement algorithm through the text area splicing module 404, merge the remaining adjacent text candidate areas into the text line, and the text sequence generation module 501 in the text recognition unit 5 extracts all text candidate areas in the image information, transmits the feature vector to the recurrent neural network for recognition, and obtains the text sequence corresponding to multiple consecutive frames. The text information processing module 502 converts the space information contained in the sequence into a separator and deletes the repeated text information. The text sequence storage module 503 determines the drug name, manufacturer, expiration date and bottle capacity corresponding to the current image, and transmits them to the database. The drug review unit 6 The corresponding patient information is queried according to the drug name and manufacturer. The validity period analysis module 701 in the infusion monitoring unit 7 uses the processor to analyze whether the current time is within the validity period range. If it is within the validity period range, no operation is performed. Otherwise, the information that the infusion is not allowed is returned through the visual interface, and a prompt is given in the form of a voice broadcast. The weight monitoring module 702 monitors the weight of the liquid contained in the infusion bottle in real time. The end prompt module 703 will return the information that the infusion is about to be completed through the visual interface when the remaining liquid weight is less than or equal to five grams, and a prompt is given in the form of a voice broadcast. The flow rate detection module 704 calculates the average speed and instantaneous speed of the current infusion, and outputs them through the visual interface.

[0058] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0059] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A recognition system based on AI visual infusion scene monitoring, comprising a text recognition unit (5), a drug review unit (6) and an infusion monitoring unit (7), characterized in that: An information acquisition unit (1), wherein the information acquisition unit (1) acquires the name and manufacturer of the corresponding medicine in the infusion list of each patient, transmits the name and manufacturer to a database, divides the materials of existing infusion bottles into three types, namely glass bottles, plastic bottles and plastic bags, extracts the materials corresponding to the medicine infusion bottles of different manufacturers, counts the capacity of each material, determines the weight of the infusion bottles corresponding to different capacities, and transmits the data in the form of a data storage of the name of the medicine, the material of the infusion bottle, the capacity, the manufacturer and the weight to the database; An image processing unit (2), wherein the image processing unit (2) uses a camera to photograph infusion bottles of different materials, transmits the collected image information as initial image information to a sample data set, uses random variables to process multiple initial image information in the sample data set, obtains image information with Gaussian noise, stores it in the sample data set, counts the number of pixels corresponding to different gray levels in the image information, uses a distribution function to deduce a reference gray level, adjusts the gray level of the image information according to the reference gray level, and stores the processed image information in the sample data set; A material analysis unit (3), wherein the material analysis unit (3) uses an AlexNet neural network to construct an infusion bottle material recognition model, obtains multiple images of the same type in a training set according to a preset value, and when transmitting the images as input to the model, uses a dropout technique to randomly select 20 percent of the neural networks in the two fully connected layers of the model for deletion, analyzes the image information through the remaining neural networks, iteratively updates the parameters of the model, restores the deleted neural networks, repeats the operation until the model training is completed, uses a camera to shoot the current infusion bottle, transmits the collected image information to the infusion bottle material recognition model for analysis, and outputs the obtained results; A text detection unit (4) is provided. After extracting the current image information from the database, the text detection unit (4) performs a sliding window operation on the feature map corresponding to the image information, selects ten bounding boxes in the central area of ​​each window to predict the text candidate area, transmits the feature vector corresponding to each row in the feature map to the recurrent neural network, analyzes it through the recurrent neural network to obtain the corresponding sequence feature, uses softmax to calculate the ratio of the intersection area to the union area between the priori box corresponding to the feature map and each bounding box, and if the obtained data is greater than a threshold, the corresponding bounding box is determined as a positive sample, and the offset analysis algorithm is used to analyze these bounding boxes to determine the position and height of the bounding boxes, and multiple text candidate areas with the smallest adjacent distances are spliced ​​to generate text lines.

2. According to claim 1, the recognition system based on AI visual infusion scene monitoring is characterized in that: The image processing unit (2) comprises an initial sample acquisition module (201) and a first noise adding module (202). The initial sample acquisition module (201) uses a camera to photograph infusion bottles of different materials, and transmits the acquired image information as initial image information to a sample data set according to different material types. The first noise adding module (202) uses a random number generation function to calculate two numerical values, and then converts them into random variables of Gaussian distribution through a Box-Muller transformation formula. The random variables are used to process multiple pieces of initial image information in the sample data set to obtain image information with Gaussian noise.

3. The recognition system based on AI visual infusion scene monitoring according to claim 2 is characterized in that: The image processing unit (2) further comprises a second noise adding module (203) and a grayscale analysis module (204), wherein the second noise adding module (203) sets a signal-to-noise ratio corresponding to the image, and calculates the signal-to-noise ratio according to the total number of pixels contained in the image information in the sample data set, obtains the number of pixels for setting interference, and then uses the obtained number of pixels to determine the designated position of the pixel point in the image information, sets the pixel value of the corresponding position to zero, obtains image information with impulse noise, and stores it in the sample data set; the grayscale analysis module (204) counts the number of pixels corresponding to different grayscales in the image information, determines the pixel ratio corresponding to each grayscale, obtains the distribution function corresponding to the image information according to the pixel ratio, calculates the reference grayscale using the distribution function, and adjusts the grayscale of the image information to obtain processed image information, which is stored in the sample data set.

4. The recognition system based on AI visual infusion scene monitoring according to claim 1 is characterized in that: The material analysis unit (3) comprises a data set division module (301), a model construction module (302), a model optimization module (303), a model training module (304) and a material determination module (305). The data set division module (301) extracts 80 percent of the image information of each type in the sample data set and transmits it to the training data set, and transmits the remaining image information to the test data set. The model construction module (302) uses the AlexNet neural network to construct an infusion bottle material recognition model. The infusion bottle material recognition model includes five convolutional layers and three fully connected layers. After setting the number of samples input each time, the model optimization module (303) obtains multiple images of the same type in the training set according to a preset value, and when transmitting them as input to the model, the dropout technology is used in the two fully connected layers. Twenty percent of the neural networks are randomly selected for deletion, and the deleted neural network parameters are backed up. After the image information is analyzed by the remaining neural networks, the predicted value of the model is obtained. The model training module (304) uses the loss function to analyze the difference between the model generation probability and the actual probability, sets the momentum parameter, estimates the gradient in the network according to the corresponding target probability provided by the loss function, iteratively updates the parameters of the model, restores the deleted neural network, and repeats the operation until the model training is completed. The material determination module (305) uses a camera to shoot the current infusion bottle, stores the collected image information in a database, retrieves the current image information from the database, transmits it to the infusion bottle material recognition model for analysis, obtains the probabilities corresponding to the three materials, and selects the material with the highest probability as the result for output.

5. The recognition system based on AI visual infusion scene monitoring according to claim 1 is characterized in that: The text detection unit (4) comprises a bounding box selection module (401) and a sequence feature generation module (402). After extracting current image information from a database, the bounding box selection module (401) uses a VGG16 convolutional neural network to analyze the current image information to obtain a feature map corresponding to the image information, and performs a sliding window operation in the feature map. The window size is first set to 3×3, and ten bounding boxes are selected in the center area of ​​each window to predict the text candidate area. The sequence feature generation module (402) combines each window center with its surrounding area to generate a feature vector, and transmits the feature vector corresponding to each row in the feature map to a recurrent neural network, and analyzes it through the recurrent neural network to obtain the corresponding sequence feature.

6. The recognition system based on AI visual infusion scene monitoring according to claim 5 is characterized in that: The text detection unit (4) further comprises a limit frame position determination module (403) and a text area splicing module (404). The limit frame position determination module (403) uses a softmax function to delete the limit frames that exceed the image information boundary, and then calculates the ratio of the intersection area to the union area between the prior frame corresponding to the feature map and each limit frame. If the obtained data is greater than a threshold, the corresponding limit frame is determined as a positive sample. If the obtained data is less than the threshold but greater than the data obtained by the remaining limit frames, the corresponding limit frame is also determined as a positive sample, and the limit frame is analyzed using an offset analysis algorithm to determine the position and height of the limit frame. The text area splicing module (404) analyzes the specific position of each text candidate area according to the position and height of the limit frame, and then splices multiple text candidate areas with the smallest adjacent distance to generate a text line. The edge refinement algorithm is used to detect the surrounding area of ​​each text line, and the remaining adjacent text candidate areas are merged into the text line.

7. The recognition system based on AI visual infusion scene monitoring according to claim 1 is characterized in that: The text recognition unit (5) comprises a text sequence generation module (501), a text information processing module (502) and a text sequence storage module (503). The text sequence generation module (501) extracts all text candidate areas in the image information, sets the image information of the text candidate areas to the same height, and then uses a convolutional neural network model to analyze the image information to obtain a feature vector containing sequence information. The sequence of each frame in the feature image corresponds to the sequence of the text candidate areas. The feature vector is transmitted to a recurrent neural network for recognition to obtain a text sequence corresponding to a plurality of consecutive frames. The text information processing module (502) uses a convolutional neural network model to store the text sequence information. The text sequence is detected by a classifier, and the space information contained in the sequence is converted into separators. The text sequence is traversed again. Once repeated text information is found in the sequence, it is determined whether there are separators between the repeated text information. If there are separators between the repeated text information, the text information is retained. If there are no separators between the repeated text information, the text information at the front of the sequence is retained, and the remaining information is deleted. The processed text sequence is transmitted to a database. The text sequence storage module (503) determines the drug name, manufacturer, expiration date and bottle capacity corresponding to the current image according to the identified text sequence, and transmits them to the database.

8. The recognition system based on AI visual infusion scene monitoring according to claim 1 is characterized in that: The drug review unit (6) uses a database viewing tool to query the corresponding database according to the drug name and manufacturer. If the corresponding patient information is found, it is output through a visual interface. If the corresponding patient information is not found, it returns information that does not allow the infusion to be performed through the visual interface and prompts in the form of voice broadcast.

9. The recognition system based on AI visual infusion scene monitoring according to claim 1 is characterized in that: The infusion monitoring unit (7) comprises an expiration date analysis module (701), a weight monitoring module (702), an end prompt module (703) and a flow rate detection module (704). The expiration date analysis module (701) uses a database viewing tool to query the database according to the drug name, infusion bottle material, capacity and manufacturer to obtain the weight corresponding to the infusion bottle, and uses a processor to analyze whether the current time is within the expiration date range. If it is within the expiration date range, no operation is performed. Otherwise, a message that the infusion is not allowed to be performed is returned through a visual interface, and a prompt is given in the form of a voice broadcast. The weight monitoring module (702) uses a weight sensor to monitor the actual weight of the infusion bottle during use, and transmits the obtained data and the corresponding time to a data In the database, the end prompt module (703) obtains the data and time collected by the weight sensor from the database, and analyzes the remaining liquid weight of the current infusion bottle according to the corresponding weight of the infusion bottle and the complete weight when the liquid is contained. If the remaining liquid weight is less than or equal to five grams, the information that the infusion is about to be completed will be returned through the visual interface, and a prompt will be given in the form of a voice broadcast. Otherwise, the remaining liquid weight information will be returned through the visual interface. When the user needs to determine the infusion speed at the time, the flow rate detection module (704) obtains the weight of the infusion bottle when the liquid is currently contained and the weight of the infusion bottle when the liquid is currently contained collected for the previous three times, and calculates the average speed and instantaneous speed of the current infusion according to the time interval for each weight sensor to collect data, and outputs it through the visual interface.

10. An infusion monitoring device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, an identification system based on AI visual infusion scene monitoring as described in any one of claims 1 to 9 is implemented.

Images