Automatic medicine identification system based on artificial intelligence

Abstract

The invention relates to the technical field of drug identification, and discloses an artificial intelligence-based drug automatic identification system, which comprises an application end, a collection end and a display end, the application end comprises a database, a medicine information acquisition module, a data preprocessing module, a medicine classification module, a medicine identification module and an information acquisition module; through the arrangement of the medicine classification module and the medicine identification module, each to-be-identified medicine in the to-be-identified medicine bag can be obtained firstly, then the shape of each to-be-identified medicine is analyzed to obtain the shape, the shape can be obtained through an edge detection algorithm, the contour of an object can be extracted through edge detection, and then the shape of the object is judged; according to the technical scheme, after the to-be-recognized medicines are classified into the corresponding categories, subsequent recognition can be better carried out on the to-be-recognized medicines, different medicines have different categories, recognition is carried out after the medicines are classified into the corresponding categories, the subsequent recognition range of the to-be-recognized medicines can be shortened, and the accuracy of medicine recognition can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drug identification, and more specifically to an automatic drug identification system based on artificial intelligence. Background Art

[0002] The safe use of medicines is related to people's physical health. In clinical nursing work, different patients take different medicines. Medical staff check and distribute the patients' daily medicines according to the doctor's order information, put them in medicine bags, and distribute them to patients through oral medicine carts.

[0003] However, there are many types of medicines, and some patients even have more than a dozen oral medicines in one medicine bag. In addition, the speed of drug updates is fast. When dispensing medicines, medical staff must check them by two people and check them at every level, which brings great challenges to clinical nursing work. Senior nurses have experience in drug identification, while junior nurses lack experience in drug identification. Some drugs are prone to identification errors and the identification speed is slow. Therefore, when distributing drugs to patients, they cannot be identified quickly and accurately. The accuracy of drug identification directly affects the patient's medication safety.

[0004] In view of this, the present invention proposes an artificial intelligence-based automatic drug identification system to replace manual drug identification, use artificial intelligence to improve patient medication safety, and not rely entirely on traditional experience, thereby ensuring the safety of oral medication for hospitalized patients. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides an automatic drug identification system based on artificial intelligence to solve the problems existing in the above-mentioned background technology.

[0006] The present invention provides the following technical solutions: an artificial intelligence-based drug automatic identification system, comprising an application end, a collection end, and a display end; The application end includes a database, a drug information acquisition module, a data preprocessing module, a drug classification module, a drug identification module and an information acquisition module; The acquisition end is an image acquisition device, which is used to obtain the picture of the medicine package to be identified and input the picture of the medicine package to be identified; the acquisition end is also provided with a reading device, through which the coding information of the medicine package to be identified is obtained; The display terminal is used for performing human-computer interactive display through a display screen; The database is used to store all drug information; the drug information acquisition module is used to obtain pictures of the drug packages to be identified collected by the acquisition end; the data preprocessing module is used to preprocess the data and transmit it to the drug classification module; the drug classification module is used to classify each drug to be identified in the drug package to be identified, and divide the drugs to be identified into corresponding categories; the drug identification module is used to identify the classified drugs to be identified through the constructed drug identification model; the information acquisition module is used to obtain the associated information of the identification result and transmit it to the display end; at the same time, when the identification result is inconsistent with the coding information of the drug obtained by the acquisition end, an early warning is issued.

[0007] Preferably, the drug information includes drug name, drug picture, drug shape, drug use, drug properties and drug dosage; when the database stores drug information, the drug pictures are taken from multiple angles, and the drug-related description information is all drug description content contained in the drug instructions. The data preprocessing module preprocesses the data including image grayscale processing, image enhancement processing and filtering processing.

[0008] Preferably, the image grayscale processing is any one of a component method, a maximum method, an average method and a weighted average method. The component method uses the brightness of the red, green and blue components in the color image as the grayscale values ​​of the three grayscale images respectively, and the maximum method uses the value with the largest brightness among the R, G and B components in the color image as the grayscale value of the grayscale image; the average method averages the brightness of the R, G and B components in the color image to obtain a more balanced grayscale image; the weighted average method performs weighted averaging of the R, G and B components with different weights to obtain a grayscale value; the weights of the R, G and B components are dynamically adjusted using a slider, and three sliders are created to correspond to the R, G and B components respectively. The value range of the slider is set to the standard RGB component range, and the slider is moved based on different drugs to be identified to obtain different weights.

[0009] Preferably, the image enhancement process adopts any one of an algorithm based on the spatial domain and an algorithm based on the frequency domain. The algorithm based on the spatial domain directly operates on the grayscale of the image, including a point operation algorithm and a domain enhancement algorithm. The point operation algorithm includes grayscale correction, grayscale transformation and histogram correction. The domain enhancement algorithm includes an image smoothing algorithm and a sharpening algorithm. The sharpening algorithm includes a gradient method, a second-order derivative operator method, a high-pass filter and a mask matching method. The frequency domain-based algorithm regards the image as a two-dimensional signal, performs two-dimensional Fourier transform signal enhancement on it, uses low-pass filtering to remove noise in the image, and uses high-pass filtering to enhance edge high-frequency signals.

[0010] Preferably, the drug classification module classifies each drug to be identified in the drug package to be identified, including the following steps: Step S01: Based on the information of each drug in the database, the drug shape of each drug is obtained, wherein the drug shape includes the shape of a tablet and the shape of a capsule, and the drugs are classified according to the drug shape to form a plurality of shape categories, and the drug shapes in each category are the same; Step S02: based on the data preprocessed by the data preprocessing module, obtaining the shape of each drug to be identified in the drug package to be identified; Step S03: Based on the shapes of the drugs to be identified obtained in step S02, classify them into corresponding categories.

[0011] Preferably, the shape of each drug to be identified in the drug package to be identified in step S02 is obtained by using a Sobel operator: The x and y components of the image gradient in the horizontal direction and vertical direction are calculated respectively. The convolution kernel used is a multi-scale convolution kernel for multi-scale feature extraction. Different drugs have different characteristics at different scales. The multi-scale convolution kernel can be used to identify drugs from multiple dimensions. The spatial characteristics of drugs are captured by convolution kernels of different scales, which can be expressed as: ; in, represents the output of the multi-scale convolution kernel, represents the input feature map, represents splicing along the channel dimension, Indicates the size The convolution operation, Indicates the size The convolution operation, Indicates the size Convolution operation; The convolution kernel is convolved with the image to obtain the horizontal gradient component G x and the vertical gradient component G y , the calculation formula is expressed as: ; ; Where I is the picture of the medicine package to be identified, i is the i-th pixel point in the x direction, and j is the j-th pixel point in the y direction; The gradient direction θ is expressed as: ; When calculating the gradient amplitude G, the Euclidean norm is used to accurately reflect the actual length of the gradient vector. The calculation formula is expressed as: ; After obtaining the gradient magnitude and gradient direction of each pixel, find the local maximum; After completing the local maximum search, the sub-pixel edge points are calculated using the gradient values ​​of the current pixel and its adjacent pixels in the gradient direction by applying the parabolic interpolation method; After calculating the sub-pixel coordinates of all edge points, set a threshold to distinguish valid edges from noise; After all edge points are obtained, feature recognition is performed on all closed figures formed by continuous edge points to obtain the shape of the closed image, that is, the shape of the medicine in the medicine package to be identified; The shapes of all drugs to be identified in the drug package to be identified are obtained. A closed figure is regarded as a drug to be identified, corresponding to a drug shape.

[0012] Preferably, the specific method of setting the threshold to distinguish effective edges from noise is: Two thresholds are set, namely the high threshold and the low threshold, which are used to process strong edges and weak edges respectively; pixels with grayscale values ​​higher than the high threshold are directly retained as edge points, and the low threshold is used to eliminate weak edge points that may be caused by noise and weak gradients. For pixels with grayscale values ​​between the high threshold and the low threshold, if the pixel is connected to one or more strong edge points higher than the high threshold, the point is retained as an edge point, otherwise the point is regarded as a pseudo edge and is eliminated.

[0013] Preferably, the process of finding the local maximum is: when and When the horizontal gradient is greater than the vertical gradient, the gradient direction is horizontal and the current point is marked as D x =1; when and , when the vertical gradient is greater than the horizontal gradient, the gradient direction is vertical, and the current point is marked as D y =1.

[0014] Preferably, the drug recognition module captures a separate picture of each closed figure based on the drug edge points detected by the drug classification module as input data for the drug recognition model; there is a corresponding drug recognition model for each drug shape, and the drug information of all drugs corresponding to the drug shape is used as training data; the input data is input into the corresponding drug recognition model.

[0015] Preferably, the constructed drug recognition model is specifically a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer, an activation function and an output layer; The input layer is used to input data; the convolution layer extracts feature image information and is composed of a plurality of convolution kernels. The convolution kernel slides on the input feature map according to a given step size and traverses each pixel point. Each pixel point will cause the convolution kernel and the input feature map to have an overlap area, multiply the elements corresponding to the overlap area and add them, and then add a bias term to generate an output feature pixel point; The pooling layer further extracts image features from the feature map after the convolution operation, reduces the parameters of the neural network and the amount of feature data while keeping the amount of input data unchanged; the activation function is expressed as: ;in, Indicates input, and is an adjustable hyperparameter; when the input When larger, Approaching 1, alleviating the gradient vanishing problem, when the input When smaller, Functions provide nonlinearity and enhance the expressiveness of the model; The fully connected layer performs nonlinear combination on the extracted features to obtain output; the output layer is used to output data, namely, drug identification results.

[0016] Technical effects and advantages of the present invention: The present invention is provided with a drug classification module and a drug identification module, which is conducive to first obtaining each drug to be identified in the drug package to be identified, and then performing shape analysis on each drug to be identified to obtain the shape, which can be obtained through an edge detection algorithm. Through edge detection, we can extract the outline of the object and then judge its shape; after the drugs to be identified are divided into corresponding categories, the subsequent identification of the drugs to be identified can be better performed. Different drugs have different categories. After being divided into corresponding categories and then identified, the scope of subsequent identification of the drugs to be identified can be shortened and the accuracy of drug identification can be improved; while ensuring the same shape, since no two drugs can have the same parameters of imprint, color, size and shape, there must be some different parameter characteristics, thereby ensuring the accuracy of drug identification; combined with the neural network model, the data modeling ability of the network model is utilized to better process the data, improve the efficiency of parallel processing, and increase robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a structural diagram of the artificial intelligence-based automatic drug identification system of the present invention.

[0018] Figure 2 This is a structural diagram of the application end in the artificial intelligence-based automatic drug identification system of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are only examples. The artificial intelligence-based automatic drug identification system involved in the present invention is not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0020] like Figure 1 As shown, the present invention provides an artificial intelligence-based drug automatic identification system, including an application end, a collection end, and a display end; like Figure 2 As shown, the application end includes a database, a drug information acquisition module, a data preprocessing module, a drug classification module, a drug identification module and an information acquisition module; The acquisition end is an image acquisition device, which is used to obtain a picture of the medicine package to be identified and input the picture of the medicine package to be identified; the image acquisition device is any device capable of image acquisition, such as a camera or a micro camera; the acquisition end is also equipped with a reading device, which includes but is not limited to a barcode scanner and an RFID reader, and a barcode, a two-dimensional code or an RFID tag can be set on the medicine to be identified, and the coding information of the medicine is obtained through the reading device of the acquisition end, and the coding information is the information of the medicine barcode, two-dimensional code or RFID tag; The display terminal is used for human-computer interactive display through a display screen and can be installed on an oral medicine cart; The database is used to store all drug information; the drug information includes but is not limited to drug name, drug picture, drug shape, drug use, drug properties, drug dosage and other drug-related information; The drug information acquisition module is used to acquire the image of the drug package to be identified collected by the acquisition terminal and transmit it to the data preprocessing module for preprocessing; The data preprocessing module is used to preprocess the data and then transmit it to the drug classification module; The drug classification module is used to classify the drugs to be identified in the drug package to be identified, and divide the drugs to be identified into corresponding categories; its purpose is to better identify the drugs to be identified after the drugs to be identified are divided into corresponding categories. Different drugs have different categories. After being classified into corresponding categories, identification can shorten the scope of subsequent identification of drugs to be identified and improve the accuracy of drug identification; The drug recognition module is used to identify the classified drugs to be identified through the constructed drug recognition model, and transmit the recognition results to the information acquisition module; since no two drugs can have the same four parameters of imprint, color, size and shape, the visual difference enables the image processing technology to effectively identify the drugs; The information acquisition module is used to obtain the associated information of the recognition result and transmit it to the display end; at the same time, when the recognition result is inconsistent with the coding information of the medicine obtained by the collection end, an early warning is issued, indicating that the recognition result may be wrong; the associated information includes but is not limited to the distribution object of the medicine package to be identified and the date of administration of the medicine package to be identified.

[0021] In this embodiment, it should be specifically explained that when the database stores drug information, the drug pictures are taken from multiple angles and there are multiple drug pictures. The drug-related description information is all the drug description content contained in the drug instructions.

[0022] In this embodiment, it should be specifically explained that the data preprocessing module performs preprocessing on the data including but not limited to image graying, image enhancement and filtering, effectively highlighting the drug features in the image and suppressing the drug packaging features in the image; when distributing drugs to patients, the daily drugs of the patients are individually packaged, with the front being transparent and the back being printed with the drug name and patient information, and the individual packages that need to be identified are the drug packages to be identified, so the pictures of the drug packages to be identified collected by the acquisition end through the image acquisition device are pictures including the outer packaging, but because the front is transparent, when preprocessing the pictures, the packaging features of the drugs in the image can be suppressed, and the drug features can be highlighted, which is convenient for subsequent target identification and helps to improve the accuracy of subsequent target identification; The image grayscale processing includes but is not limited to the component method, the maximum value method, the average value method and the weighted average method. The component method uses the brightness of the red, green and blue components in the color image as the grayscale values ​​of the three grayscale images respectively. The maximum value method uses the value with the largest brightness among the three components of R, G and B in the color image as the grayscale value of the grayscale image to highlight the bright area in the image. The formula is expressed as: The average value method averages the brightness of the three components R, G, and B in the color image to obtain a gray value, which can comprehensively consider the three components in the color image to obtain a more balanced gray image. The formula is expressed as: ; The weighted average method performs weighted average of the three components R, G, and B with different weights according to the sensitivity of the human eye to different colors to obtain a grayscale value. The weights of the three components R, G, and B can be adjusted dynamically, and a slider can be used for dynamic adjustment. Three sliders are created to correspond to the three components R, G, and B respectively, and the value range of the slider is set to the standard RGB component range. The slider is moved based on different drugs to be identified to obtain different weights; when the drug to be identified is in different forms such as tablets, pills, or capsules, the weights of the RGB components in different forms can be set, and the weights of the RGB components corresponding to different forms can be pre-set by those skilled in the art according to actual conditions; for example, the surface of capsule drugs is smooth and often green / transparent, so the G channel is significant, and the weight of the G component is more than that of R and B. The surface of tablet drugs has indented text, and red / blue marks are mostly, so the R channel and the B channel are prominent, and the weights of the R channel and the B channel are more than that of G. Pill drugs are spherical or small particles with uniform color distribution, so the weights of the three components R, G, and B are balanced; The image enhancement processing can be performed using an algorithm based on the spatial domain or an algorithm based on the frequency domain. The algorithm based on the spatial domain directly operates on the grayscale of the image, including a point operation algorithm and a field enhancement algorithm. The point operation algorithm includes grayscale correction, grayscale transformation, and histogram correction, etc. The purpose is to make the image imaging uniform, or to expand the dynamic range of the image and extend the contrast. The field enhancement algorithm includes an image smoothing algorithm and a sharpening algorithm. The smoothing algorithm is used to eliminate image noise, but may cause edge blur. The sharpening algorithm is used to highlight the edge contour of the object to facilitate target recognition, including the gradient method, the second-order derivative operator method, the high-pass filtering, and the mask matching method. The frequency-domain-based algorithm performs some correction on the transformation coefficient value of the image in a certain transformation domain of the image. It is an indirect enhancement algorithm. The image is usually regarded as a two-dimensional signal, and a two-dimensional Fourier transform signal is enhanced by using a low-pass Filtering can remove noise from the image, and high-pass filtering can enhance high-frequency signals such as edges to make blurred images clear; the filtering process includes linear filtering and nonlinear filtering. The linear filtering includes mean filtering and Gaussian filtering. The mean filtering achieves image smoothing by calculating the average value of each pixel in the image and its surrounding pixels, which is simple and fast; the Gaussian filtering uses Gaussian distribution as a weight coefficient for filtering, which can more naturally reduce image noise and details while retaining more edge information; the nonlinear filtering includes median filtering and bilateral filtering. The median filtering uses the median value in the pixel field to represent the pixel value. The median filtering is particularly good at removing salt and pepper noise while maintaining the sharpness of the edge. The bilateral filtering combines the spatial proximity of the image and the similarity of pixel values ​​for filtering. The bilateral filtering can retain edge information while smoothing the image, and is suitable for occasions where edge clarity needs to be maintained.

[0023] In this embodiment, it should be specifically explained that the drug classification module classifies each drug to be identified in the drug package to be identified, including the following steps: Step S01: Based on the information of each drug in the database, the drug shape of each drug is obtained, wherein the drug shape includes the shape of a tablet and the shape of a capsule. The shape of the tablet includes a circle, an ellipse, a cone, etc., and the shape of the capsule is a cylinder with rounded ends, etc. The drugs are classified according to the drug shape to form a plurality of shape categories, and the drug shapes in each category are the same; Step S02: Based on the data preprocessed by the data preprocessing module, the shape of each drug to be identified in the drug package to be identified is obtained; the purpose is to first obtain each drug to be identified in the drug package to be identified, and then perform shape analysis on each drug to be identified to obtain the shape, which can be obtained through edge detection algorithm. Through edge detection, we can extract the outline of the object and then determine its shape; Step S03: Based on the shapes of the drugs to be identified obtained in step S02, they are divided into corresponding categories. The purpose is to divide the drugs to be identified in the drug packages to be identified into corresponding shape categories, so that in the subsequent drug identification process, further feature identification can be performed only on drugs of the same shape category. While ensuring the same shape, since no two drugs can have the same four parameters of imprint, color, size and shape, there must be some different parameter features, thereby ensuring the accuracy of drug identification.

[0024] In this embodiment, it should be specifically explained that the classification of drugs according to their shapes in step S01 may be performed using a clustering algorithm or a statistical method. The clustering algorithm divides the data set into several clusters, and the data points in each cluster have a high degree of similarity, while the data points between different clusters have a low degree of similarity. The drug shape information exists in the drug information stored in the database. Therefore, the drug shape corresponding to each drug name is used as feature data and clustered. Then, the drugs corresponding to the feature data in each cluster are all drugs of the same shape. The statistical method performs statistics based on the drug shapes in the drug information stored in the database to obtain the drug names and other information corresponding to drugs of the same shape. By classifying the drug shapes, the required information can be quickly located and retrieved in subsequent drug identification, thereby reducing search time, improving overall efficiency, and making the data clearer and easier to analyze.

[0025] In this embodiment, it should be specifically explained that the shape of each drug to be identified in the drug package to be identified in step S02 is obtained by using an edge detection algorithm, which can be any one of the edge detection algorithms such as a first-order differential operator, a second-order differential operator, and a Canny operator. In this embodiment, the Sobel operator is used as an example; The Sobel operator is used to calculate the horizontal x and vertical y components of the image gradient respectively. The convolution kernel used is a multi-scale convolution kernel for multi-scale feature extraction. Different drugs have different features at different scales. The use of multi-scale convolution kernels can identify drugs from multiple dimensions and capture the spatial features of drugs, such as shape, edge, surface texture, etc., through convolution kernels of different scales. Convolution kernels of different sizes (such as 1x1, 3x3, 5x5) can be used in parallel in the same convolution layer to capture features of different scales. This implementation uses size, Size and Taking size as an example, the multi-scale convolution kernel is expressed as: ; in, represents the output of the multi-scale convolution kernel, represents the input feature map, represents splicing along the channel dimension, Indicates the size The convolution operation, Indicates the size The convolution operation, Indicates the size Convolution operation; This embodiment provides a common convolution kernel specific method: The convolution kernel used is: , ; Among them, g x is the convolution kernel of the horizontal gradient, g y is the convolution kernel of the vertical gradient; The convolution kernel is convolved with the image to obtain the horizontal gradient component G x and the vertical gradient component G y , the calculation formula is expressed as: ; ; Where I is the picture of the medicine package to be identified, i is the i-th pixel point in the x direction, and j is the j-th pixel point in the y direction; The gradient direction θ is expressed as: ; When calculating the gradient amplitude G, the Euclidean norm is used to accurately reflect the actual length of the gradient vector. The calculation formula is expressed as: ; After obtaining the gradient magnitude and gradient direction of each pixel, find the local maximum value to accurately locate the edge and ensure edge refinement; After completing the local maximum search, the sub-pixel edge points are calculated using the gradient values ​​of the current pixel and its adjacent pixels in the gradient direction by applying the parabolic interpolation method; After calculating the sub-pixel coordinates of all edge points, set a threshold to distinguish valid edges from noise; Two thresholds are set, namely the high threshold and the low threshold, which are used to process strong edges and weak edges respectively; pixels with grayscale values ​​higher than the high threshold are directly retained as edge points, which often represent the most significant edges in the image and have high reliability and accuracy; the low threshold is used to remove weak edge points that may be caused by noise or weak gradients. For pixels with grayscale values ​​between the high threshold and the low threshold, if the pixel is connected to one or more strong edge points higher than the high threshold, the point is retained as an edge point, otherwise the point is regarded as a pseudo-edge and removed; this method ensures the continuity of the edge while minimizing the impact of pseudo-edges; After all edge points are obtained, feature recognition is performed on all closed figures formed by continuous edge points to obtain the shape of the closed image, that is, the shape of the medicine in the medicine package to be identified; The shapes of all drugs to be identified in the drug package to be identified are obtained. A closed figure is regarded as a drug to be identified, corresponding to a drug shape.

[0026] In this embodiment, it should be specifically explained that the process of finding the local maximum is: when and When the horizontal gradient is greater than the vertical gradient, the gradient direction is horizontal and the current point is marked as D x =1; when and , when the vertical gradient is greater than the horizontal gradient, the gradient direction is vertical, and the current point is marked as D y =1.

[0027] In this embodiment, it should be specifically explained that the drug recognition module intercepts a separate picture of each closed figure based on the drug edge points detected by the drug classification module as input data of the drug recognition model; there is a corresponding drug recognition model for each drug shape, and the drug information of all drugs corresponding to the drug shape is used as training data; the input data is input into the corresponding drug recognition model; illustratively, if there is a closed figure that is a circle, the separate picture of the closed figure is used as input data and input into the drug recognition model corresponding to the circle, that is, the drug recognition model formed by training with the circular-shaped drug as training data; The constructed drug recognition model is specifically a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer, an activation function, and an output layer; The input layer is used to input data; the convolution layer is the core part and the basis for extracting feature image information. It is composed of many convolution kernels. The convolution kernel slides on the input feature map according to a given step size and traverses each pixel point. Each pixel point will cause the convolution kernel and the input feature map to have an overlap area, multiply the elements corresponding to the overlap area and add them, and then add a bias term to generate an output feature pixel point. The depth of the output feature map determines the depth of the current convolution kernel, and the number of the current convolution kernels determines the depth of the output feature map of the current layer, that is, the number of channels of the output image; The pooling layer is also called a downsampling unit, which further extracts image features from the feature map after the convolution operation, reduces the parameters of the neural network and the amount of feature data while keeping the input data unchanged, improves the operation rate, and avoids overfitting. The pooling layer includes a maximum pooling layer and an average pooling layer; the maximum pooling layer retains the maximum value of a certain area, and the maximum pooling layer can extract its texture features from the image; the average pooling layer retains the average value of the area, which can be used to retain the background features of the image; The activation function introduces nonlinearity into the neural network, enabling the neural network to process data nonlinearly. The activation function is the nonlinear mapping part in the convolutional neural network. The addition of the activation function allows the neural network to better play the role of the hidden layer. The activation function is expressed as: ;in, Indicates input, and is an adjustable hyperparameter; when the input When larger, Approaching 1, alleviating the gradient vanishing problem, when the input When smaller, Functions provide nonlinearity and enhance the expressiveness of the model; The fully connected layer performs nonlinear combination on the extracted features to obtain output; the output layer is used to output data, namely, drug identification results.

[0028] In this embodiment, it should be specifically explained that the information acquisition module obtains the names of all drugs to be identified in the medicine packages to be identified, the drug pictures and the corresponding bed numbers, bed patient information and other related information; the display end displays the information through a display screen, and you can click on the drug name or drug picture to jump to the drug details interface to display the detailed information of the drug, including the drug's purpose, dosage, medication precautions, and drug properties.

[0029] In this embodiment, it should be specifically explained that the working principle of the system is as follows: the system is installed on an oral medicine cart. When distributing medicines, the medicine package to be identified is taken out from the placement grid of the oral medicine cart, the medicine inside is adjusted to be flat to ensure that there is no overlap or covering, the transparent side is facing the collection end, and the picture of the medicine package to be identified is collected, and the application end is used for identification and analysis. The medicine information acquisition module obtains the picture information of the collection end and transmits it to the data preprocessing module. The data is preprocessed and then transmitted to the medicine classification module. The edge detection algorithm is used to classify the medicine shape. After obtaining the medicine shape, a separate picture of each medicine shape is obtained as input data and input into the medicine recognition model of the corresponding shape. The final drug recognition result is identified and output to the information acquisition module to obtain related information, which is checked against the information displayed on the display screen. The opaque surface of the medicine package to be identified is printed with patient information and medication information, which can be directly checked. By clicking on the pill name or pill picture displayed on the display screen, more drug description information of the drug is further displayed to help patients understand the drug situation and enhance patients' trust in medical staff's medication. At the same time, relevant drug knowledge is popularized to patients and humanistic care is provided. The system is also applicable to other scenarios. For example, when medical staff put drugs into medicine packages to be identified, the system is used for identification and packaging after correct identification is performed to avoid incorrect drug distribution and improve patient medication safety.

[0030] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0031] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An artificial intelligence-based drug automatic identification system, characterized by: Including application end, acquisition end and display end; The application end includes a database, a drug information acquisition module, a data preprocessing module, a drug classification module, a drug identification module and an information acquisition module; The acquisition end is an image acquisition device, which is used to obtain the picture of the medicine package to be identified and input the picture of the medicine package to be identified; the acquisition end is also provided with a reading device, through which the coding information of the medicine package to be identified is obtained; The display terminal is used for performing human-computer interactive display through a display screen; The database is used to store all drug information; the drug information acquisition module is used to acquire the picture of the drug package to be identified collected by the acquisition terminal; The data preprocessing module is used to preprocess the data and then transmit it to the drug classification module; The drug classification module is used to classify the drugs to be identified in the drug packages to be identified, and divide the drugs to be identified into corresponding categories; the drug identification module is used to identify the classified drugs to be identified through the constructed drug identification model; the information acquisition module is used to obtain the associated information of the identification result and transmit it to the display end; at the same time, when the identification result is inconsistent with the coding information of the drug obtained by the acquisition end, an early warning is issued.

2. The artificial intelligence-based automatic drug identification system according to claim 1, characterized in that: The drug information includes drug name, drug picture, drug shape, drug use, drug properties and drug dosage; when the database stores drug information, the drug pictures are taken from multiple angles, the drug-related description information is all the drug description content contained in the drug instructions, and the data preprocessing module preprocesses the data including image grayscale processing, image enhancement processing and filtering processing; the reading device includes a barcode scanner and an RFID reader, and the coded information is the information of the drug barcode, QR code and RFID tag.

3. The artificial intelligence-based automatic drug identification system according to claim 2, characterized in that: The image grayscale processing is any one of a component method, a maximum method, an average method and a weighted average method. The component method uses the brightness of the red, green and blue components in the color image as the grayscale values ​​of the three grayscale images respectively. The maximum method uses the value with the largest brightness among the three components of R, G and B in the color image as the grayscale value of the grayscale image. The average value method averages the brightness of the three components R, G, and B in the color image to obtain a relatively balanced grayscale image; the weighted average method performs weighted average of the three components R, G, and B with different weights to obtain a grayscale value; The weights of the three components R, G, and B are dynamically adjusted using sliders. Three sliders are created to correspond to the three components R, G, and B respectively. The value range of the slider is set to the standard RGB component range. The slider is moved based on different drugs to be identified to obtain different weights.

4. The artificial intelligence-based automatic drug identification system according to claim 3, characterized in that: The image enhancement process adopts any one of an algorithm based on the spatial domain and an algorithm based on the frequency domain. The algorithm based on the spatial domain directly operates on the grayscale of the image, including a point operation algorithm and a domain enhancement algorithm. The point operation algorithm includes grayscale correction, grayscale transformation and histogram correction. The domain enhancement algorithm includes an image smoothing algorithm and a sharpening algorithm. The sharpening algorithm includes a gradient method, a second-order derivative operator method, a high-pass filter and a mask matching method. The frequency domain-based algorithm regards the image as a two-dimensional signal, performs two-dimensional Fourier transform signal enhancement on it, uses low-pass filtering to remove noise in the image, and uses high-pass filtering to enhance edge high-frequency signals.

5. The artificial intelligence-based automatic drug identification system according to claim 4, characterized in that: The drug classification module classifies each drug to be identified in the drug package to be identified, including the following steps: Step S01: Based on the information of each drug in the database, the drug shape of each drug is obtained, wherein the drug shape includes the shape of a tablet and the shape of a capsule, and the drugs are classified according to the drug shape to form a plurality of shape categories, and the drug shapes in each category are the same; Step S02: based on the data preprocessed by the data preprocessing module, obtaining the shape of each drug to be identified in the drug package to be identified; Step S03: Based on the shapes of the drugs to be identified obtained in step S02, classify them into corresponding categories.

6. The artificial intelligence-based automatic drug identification system according to claim 5, characterized in that: In step S02, the shape of each drug to be identified in the drug package to be identified is obtained by using the Sobel operator: The horizontal x and vertical y components of the image gradient are calculated respectively. The convolution kernel used is a multi-scale convolution kernel for multi-scale feature extraction. Different drugs have different characteristics at different scales. The use of multi-scale convolution kernels can identify drugs from multiple dimensions. The spatial characteristics of drugs are captured by convolution kernels of different scales, which can be expressed as: ; in, represents the output of the multi-scale convolution kernel, represents the input feature map, represents splicing along the channel dimension, Indicates the size The convolution operation, Indicates the size The convolution operation, Indicates the size Convolution operation; The convolution kernel is convolved with the image to obtain the horizontal gradient component G x and the vertical gradient component G y , the calculation formula is expressed as: ； ； Where I is the picture of the medicine package to be identified, i is the i-th pixel point in the x direction, and j is the j-th pixel point in the y direction; The gradient direction θ is expressed as: ; When calculating the gradient amplitude G, the Euclidean norm is used to accurately reflect the actual length of the gradient vector. The calculation formula is expressed as: ; After obtaining the gradient magnitude and gradient direction of each pixel, find the local maximum; After completing the local maximum search, the sub-pixel edge points are calculated using the gradient values ​​of the current pixel and its adjacent pixels in the gradient direction by applying the parabolic interpolation method; After calculating the sub-pixel coordinates of all edge points, set a threshold to distinguish valid edges from noise; After all edge points are obtained, feature recognition is performed on all closed figures formed by continuous edge points to obtain the shape of the closed image, that is, the shape of the medicine in the medicine package to be identified; The shapes of all drugs to be identified in the drug package to be identified are obtained. A closed figure is regarded as a drug to be identified, corresponding to a drug shape.

7. The artificial intelligence-based automatic drug identification system according to claim 6, characterized in that: The specific method of setting the threshold to distinguish effective edges from noise is: Two thresholds are set, namely the high threshold and the low threshold, which are used to process strong edges and weak edges respectively; pixels with grayscale values ​​higher than the high threshold are directly retained as edge points, and the low threshold is used to eliminate weak edge points that may be caused by noise and weak gradients. For pixels with grayscale values ​​between the high threshold and the low threshold, if the pixel is connected to one or more strong edge points higher than the high threshold, the point is retained as an edge point, otherwise the point is regarded as a pseudo edge and is eliminated.

8. The artificial intelligence-based automatic drug identification system according to claim 7, characterized in that: The process of finding the local maximum is: when and When the horizontal gradient is greater than the vertical gradient, the gradient direction is horizontal and the current point is marked as D x =1; when and , when the vertical gradient is greater than the horizontal gradient, the gradient direction is vertical, and the current point is marked as D y =1.

9. The artificial intelligence-based automatic drug identification system according to claim 8, characterized in that: The drug recognition module captures a separate picture of each closed figure based on the drug edge points detected by the drug classification module as input data for the drug recognition model; there is a corresponding drug recognition model for each drug shape, and the drug information of all drugs corresponding to the drug shape is used as training data; the input data is input into the corresponding drug recognition model.

10. The artificial intelligence-based automatic drug identification system according to claim 9, characterized in that: The constructed drug recognition model is specifically a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer, an activation function and an output layer; The input layer is used to input data; the convolution layer extracts feature image information and is composed of a plurality of convolution kernels. The convolution kernel slides on the input feature map according to a given step size and traverses each pixel point. Each pixel point will cause the convolution kernel and the input feature map to have an overlap area, multiply the elements corresponding to the overlap area and add them, and then add a bias term to generate an output feature pixel point; The pooling layer further extracts image features from the feature map after the convolution operation, reduces the parameters of the neural network and the amount of feature data while keeping the amount of input data unchanged; the activation function is expressed as: ;in, Indicates input, and is an adjustable hyperparameter; when the input When larger, Approaching 1, alleviating the gradient vanishing problem, when the input When smaller, Functions provide nonlinearity and enhance the expressiveness of the model; The fully connected layer performs nonlinear combination on the extracted features to obtain output; the output layer is used to output data, namely, drug identification results.

Images