A method, apparatus, electronic device and storage medium for drug detection
By acquiring medication instructions and image information from the dispensing station, the system identifies medication dispensing actions and drug characteristics, generating early warning signals to standardize medication dispensing operations. This solves the problem of non-standard medication use by medical staff and improves the accuracy and safety of medication dispensing.
Patent Information
- Application Number
- CN202211565186.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-07
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-12-07
AI Technical Summary
During clinical diagnosis, medical staff may administer medication improperly when dispensing medication, and existing technologies have not been able to effectively verify and standardize the procedures.
By acquiring the patient's medication instructions and the image information of the dispensing station, the system identifies the medication dispensing action and drug characteristics, determines the medication dispensing result information, matches it with the medication information, and generates an early warning signal to standardize the medication dispensing operation.
This improved the standardization of medication dispensing procedures for medical staff, reduced the risk of improper medication use, and ensured the accuracy and safety of medication administration.
Smart Images

Figure CN116246753B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing, and in particular to a method, apparatus, electronic device and storage medium for drug detection. Background Technology
[0002] Clinical diagnosis refers to the process by which doctors conduct physical examinations on patients, classify and differentiate the causes and pathogenesis of the patient's disease based on the patient's clinical manifestations, and prescribe appropriate treatment based on the differentiation results in order to enable the patient to recover as soon as possible.
[0003] In related technologies, during clinical diagnosis, doctors typically examine patients' various physical indicators and, based on the examination results or clinical manifestations, determine the medications required. They then notify relevant medical staff to administer the injections. For example, after a preliminary examination, a doctor might instruct medical staff to inject 100ml of glucose into the patient for treatment. However, because different patients have different conditions and require different medications, some patients may even need multiple injections. Furthermore, given the large number of patients, and the lack of verification regarding the procedures for medication dispensing by medical staff in these technologies, improper medication administration may occur during patient treatment. Summary of the Invention
[0004] In order to verify the procedures performed by medical staff during the medication dispensing process and thereby improve the medication use standards of medical staff, this application provides a medication detection method, device, electronic device and storage medium.
[0005] Firstly, this application provides a method for drug detection, employing the following technical solution: A method for detecting drug use, comprising: Obtain the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient. The medication information includes the medication name and dosage. Obtain image information of the dispensing station corresponding to the medication instruction; Based on the image information, the corresponding medication retrieval result information is determined, including the name of the medication and the dosage. The medication information and the medication retrieval result information are matched, and if there is a mismatch, an early warning signal is generated.
[0006] By adopting the above technical solution, the operation during medication retrieval is recorded to determine the name of the medication and the dosage. The operation during retrieval is then compared with the medication name and dosage corresponding to the medication instruction to determine whether the retrieval operation conforms to the specifications of the medication instruction. When the retrieval operation does not conform to the specifications of the medication instruction, an early warning signal is generated to remind relevant staff to promptly standardize the medication retrieval operation. The medication information is used to verify the medication retrieval results of relevant medical staff, which helps to improve the standardization of the medication retrieval operation process.
[0007] In one possible implementation, obtaining the medication instruction includes: The system acquires the input text information, parses the text information, and obtains the corresponding medication instructions. or, The system acquires on-site audio, parses the audio, and obtains corresponding medication instructions. The on-site audio is the audio recording of the doctor diagnosing the patient's condition.
[0008] By adopting the above technical solutions, on the one hand, medication instructions can be determined through text information. After a doctor diagnoses a patient, the medication data can be directly input, or the medication prescription generated by the doctor based on the patient's condition can be uploaded, which facilitates the improvement of the accuracy of determining medication instructions. On the other hand, by directly converting the audio of the diagnosis into medication instructions, instead of having relevant staff input them, the workload of relevant staff is reduced, and the speed of generating medication instructions is improved.
[0009] In one possible implementation, determining the corresponding medication retrieval result information based on the image information includes: The image information is used to identify the action of taking medicine; When a drug-taking action is detected in the image information, a hand image is acquired; The hand image is subjected to drug feature recognition, and the drug retrieval result information is determined based on the recognized drug features.
[0010] By adopting the above technical solution, image information is filtered based on the drug-taking action to obtain image information containing the drug-taking action, and image information that does not contain the drug-taking action can be discarded. Then, by performing drug feature recognition on the acquired hand image, the drug-taking result information can be determined based on the drug features. By recognizing the operation contained in the image, the accuracy of determining the drug name and dosage can be improved.
[0011] In one possible implementation, the step of performing drug feature recognition on the hand image and determining the medication dispensing result information based on the drug features includes: The hand image is subjected to drug packaging recognition to obtain drug packaging features, which include packaging shape and packaging pattern; Based on the drug packaging characteristics, the drug packaging database is traversed to determine the drug name corresponding to the drug packaging characteristics; The medicine-retrieving tool is determined based on the hand image, and multiple images containing the medicine-retrieving tool are determined from multiple hand images within a preset time period. The dosage of medication is determined based on multiple images containing medication dispensing tools. The name of the medication and the dosage constitute the medication result information.
[0012] By adopting the above technical solution, since there are many types of drugs, and similar drug shapes correspond to different therapeutic effects and names, if the name of the drug is determined solely by its shape, errors may occur. However, determining the name of the drug by using the shape and pattern of the drug packaging can improve the accuracy of determining the name of the drug. Furthermore, by varying multiple images containing the drug dispensing tool, the accuracy of determining the dosage can be improved.
[0013] In one possible implementation, determining the dosage based on multiple images containing the medication dispensing tool further includes: Based on the image acquisition time, multiple images containing medicine-retrieving tools are sorted to obtain a medicine-retrieving image sequence; From the drug-taking image sequence, a first image containing the drug-taking tool and a second image containing the drug-taking tool are determined. The first image is the image corresponding to the drug-taking process before the drug is taken, and the second image is the image corresponding to the drug-taking process after the drug is taken. Among these methods, the dosage is determined based on multiple images containing medication dispensing tools, including: The dosage of the medication is determined based on the first image and the second image.
[0014] By adopting the above technical solution, multiple images containing medication dispensing tools are sorted according to the acquisition time of each image to form a medication dispensing image sequence. This facilitates the supervision of the operation process of relevant personnel during medication dispensing based on the medication dispensing image sequence. From multiple images containing medication dispensing tools, the tool image before and after medication dispensing can be determined. Based on the change of medication dispensing tools in the two images, the medication dosage can be determined, thereby improving the accuracy of determining the medication dosage.
[0015] In one possible implementation, determining the drug dosage based on the first tool image and the first image includes: When the medication corresponding to the medication instruction is liquid, the dosage is determined based on the capacity of the dispensing tool in the first image and the capacity of the dispensing tool in the second image. When the medication corresponding to the medication instruction is solid, the dosage is determined based on the amount of medication in the medication dispensing tool in the previous working image, the amount of medication in the medication dispensing tool in the post-dispensing tool image, and the content standard corresponding to the medication instruction.
[0016] By adopting the above technical solution, when determining the dosage, the drug state can be determined first. Different drug states correspond to different methods for calculating the dosage. The dosage can be determined based on the drug state, which helps to improve the accuracy of determining the dosage.
[0017] One possible implementation also includes: Calculate the usage and remaining quantity of each drug in the current cycle; Based on the dosage of each drug in the current cycle, determine the predicted dosage of each drug in the next cycle; Based on the remaining quantity of each drug in the current cycle and the predicted usage of each drug in the next cycle, determine the replenishment amount corresponding to each drug. Supply information is generated based on the supply amount and sent to the terminal devices of relevant personnel.
[0018] By adopting the above technical solution, the dosage of each medicine in the current cycle is used to predict the dosage of medicine in the next cycle, and the amount of medicine to be replenished is determined based on the prediction results and the remaining amount of medicine, so that the medicine can meet the needs of the next cycle, thereby reducing the probability of insufficient medicine at the medicine dispensing station during the rescue of patients.
[0019] Secondly, this application provides a drug detection device, which adopts the following technical solution: A drug detection device, comprising: The instruction acquisition module is used to acquire the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient. The medication information includes the medication name and the medication dosage. The image information acquisition module is used to acquire image information of the dispensing station corresponding to the medication instruction; The medication retrieval result determination module is used to determine the corresponding medication retrieval result information based on the image information, wherein the medication retrieval result information includes the name of the medication and the dosage of the medication. The early warning generation module matches the medication information with the medication retrieval result information; if they do not match, it generates an early warning signal.
[0020] By adopting the above technical solution, the operation during medication retrieval is recorded to determine the name of the medication and the dosage. The operation during retrieval is then compared with the medication name and dosage corresponding to the medication instruction to determine whether the retrieval operation conforms to the specifications of the medication instruction. When the retrieval operation does not conform to the specifications of the medication instruction, an early warning signal is generated to remind relevant staff to promptly standardize the medication retrieval operation. The medication information is used to verify the medication retrieval results of relevant medical staff, which helps to improve the standardization of the medication retrieval operation process.
[0021] In one possible implementation, the instruction acquisition module, when acquiring a medication instruction, specifically performs the following: The system acquires the input text information, parses the text information, and obtains the corresponding medication instructions. or, The system acquires on-site audio, parses the audio, and obtains corresponding medication instructions. The on-site audio is the audio recording of the doctor diagnosing the patient's condition.
[0022] In one possible implementation, when determining the corresponding medication retrieval result information based on the image information, the medication retrieval result determination module is specifically used for: The image information is used to identify the action of taking medicine; When a drug-taking action is detected in the image information, a hand image is acquired; The hand image is subjected to drug feature recognition, and the drug retrieval result information is determined based on the recognized drug features.
[0023] In one possible implementation, when the medication retrieval result determination module performs drug feature recognition on the hand image and determines the medication retrieval result information based on the drug features, it is specifically used for: The hand image is subjected to drug packaging recognition to obtain drug packaging features, which include packaging shape and packaging pattern; Based on the drug packaging characteristics, the drug packaging database is traversed to determine the drug name corresponding to the drug packaging characteristics; The medicine-retrieving tool is determined based on the hand image, and multiple images containing the medicine-retrieving tool are determined from multiple hand images within a preset time period. The dosage of medication is determined based on multiple images containing medication dispensing tools. The name of the medication and the dosage constitute the medication result information.
[0024] In one possible implementation, the device further includes: The image sorting module is used to sort multiple images containing medicine-retrieving tools according to the image acquisition time to obtain a medicine-retrieving image sequence; An image determination module is used to determine a first image containing the medicine-retrieving tool and a second image containing the medicine-retrieving tool from the medicine-retrieving image sequence, wherein the first image is the image corresponding to the medicine-retrieving process before the medicine is retrieved, and the second image is the image corresponding to the medicine-retrieving process after the medicine is retrieved. Specifically, the module for determining the medication dispensing result, when determining the medication dosage based on multiple images containing the medication dispensing tool, is used for: The dosage is determined based on the first image containing the medication dispensing tool and the second image containing the medication dispensing tool.
[0025] In one possible implementation, when determining the medication dosage based on the first image and the second image, the medication result determination module is specifically used for: When the medication corresponding to the medication instruction is liquid, the dosage is determined based on the capacity of the dispensing tool in the first image and the capacity of the dispensing tool in the second image. When the medication corresponding to the medication instruction is solid, the dosage is determined based on the amount of medication in the medication dispensing tool in the first image, the amount of medication in the medication dispensing tool in the second image, and the content standard corresponding to the medication instruction.
[0026] In one possible implementation, the device further includes: The statistics module is used to calculate the usage and remaining quantity of each drug in the current cycle. The prediction module is used to determine the predicted dosage of each drug in the next cycle based on the dosage of each drug in the current cycle. The replenishment quantity determination module is used to determine the replenishment quantity corresponding to each drug based on the remaining quantity of each drug in the current cycle and the predicted usage value of each drug in the next cycle. The replenishment information generation module is used to generate replenishment information based on the replenishment amount and send it to the terminal devices of relevant personnel.
[0027] Thirdly, this application provides an electronic device that adopts the following technical solution: An electronic device comprising: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform the above-described method for drug detection.
[0028] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program stored thereon that can be loaded by a processor and execute the above-described drug detection method.
[0029] In summary, this application includes at least one of the following beneficial technical effects: 1. By recording the medication retrieval process, the system identifies the medication name and dosage. It then compares the retrieval process with the medication name and dosage specified in the medication instructions to determine if the retrieval procedure conforms to the instructions. An alert signal is generated when discrepancies are found, reminding staff to ensure proper medication retrieval procedures are followed. This information verifies the medication retrieval results from healthcare professionals, thereby improving the standardization of medication retrieval procedures.
[0030] 2. Since there are many types of drugs, and similar drug shapes correspond to different therapeutic effects and names, it is easy to make mistakes if the name of the drug is determined solely by its shape. Using the shape and pattern of the drug packaging to determine the name of the drug can improve the accuracy of determining the name of the drug. Using multiple images containing the drug dispensing tools can improve the accuracy of determining the dosage. Attached Figure Description
[0031] Figure 1 This is a schematic flowchart of a drug detection method in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a drug detection device according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0032] The following is in conjunction with the appendix Figure 1-3 This application will be described in further detail.
[0033] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.
[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0035] To improve the medication administration standards of medical staff, this embodiment records the medication retrieval process to determine the name and dosage of the medication. The retrieval process is then compared with the medication name and dosage corresponding to the medication instruction to determine if the retrieval operation conforms to the instructions. If the retrieval operation does not conform to the instructions, a warning signal is generated to remind relevant staff to standardize the retrieval process, preventing unknowingly administering medication to patients without their knowledge, which could negatively impact patient treatment.
[0036] Specifically, this application provides a drug detection method executed by an electronic device, which can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this.
[0037] refer to Figure 1 , Figure 1 This is a flowchart illustrating a drug detection method according to an embodiment of this application. The method includes steps S110, S120, S130, and S140, wherein: Step S110: Obtain the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient.
[0038] The medication information includes the name of the medication and the dosage.
[0039] Specifically, the medication instruction is determined by the medication data input by the doctor. This medication data can be audio data of the doctor's verbal instructions or text information uploaded by the doctor after diagnosing the patient, i.e., a medication order. The form of the medication data is not specifically limited in this embodiment, as long as it can form a medication instruction. When determining the medication instruction based on the medication data, semantic recognition can be performed on the data to discard modal particles and conjunctions, resulting in simplified medication data. This allows relevant staff to retrieve medication based on the simplified data, thereby improving efficiency. The medication data includes the medication name and dosage. The medication instruction is determined based on the medication data, and medication information is obtained based on the medication instruction; that is, the medication information also includes the medication name and dosage.
[0040] After the doctor uploads the medication information for each patient, the generated medication instructions are used to remind relevant medical staff to pick up the medication for the patient according to the instructions. Since there are a large number of patients and a large number of corresponding medication instructions, a medical staff member may need to pick up medication for multiple patients simultaneously based on the medication instructions of multiple patients. Medical staff can select the target medication instruction from multiple medication instructions through condition selection and pick up the medication for the target patient according to the target medication instruction. The selection conditions can be the patient's name or the generation time of the medication instruction.
[0041] Step S120: Obtain image information of the dispensing station corresponding to the medication information.
[0042] The medication dispensing station is an area for storing medications. When relevant personnel receive a medication instruction, they will retrieve the medication from the dispensing station according to the instruction. The image information of the dispensing station is acquired by an image acquisition device located in the dispensing station area, and the acquired image information is sent to an electronic device. The image acquisition device can be located above or to the side of the dispensing station; the specific location is not specifically limited in this embodiment, as long as the dispensing station can be captured in the image.
[0043] Once the target medication instruction selected by the relevant medical staff is detected, the image information of the dispensing station corresponding to the target medication instruction is retrieved.
[0044] Step S130: Determine the corresponding medication retrieval result information based on the image information. The medication retrieval result information includes the name of the medication and the dosage.
[0045] Specifically, by recognizing the image information at the dispensing station, the medication packaging can be identified, and the medication name can be determined. The dosage can be determined based on the dispensing tool. For example, when the medication instruction corresponds to a liquid and the required amount is small, the dispensing tool may be a syringe, such as 1 ml of adrenaline. In this case, the dosage can be determined by recognizing the volume of the medication in the dispensing tool. When the medication instruction corresponds to a liquid and the required amount is large, the dispensing tool may be a medication bag containing the medication. Each medication bag corresponds to a fixed volume, such as 500 ml of glucose per bag.
[0046] Step S140: Match the medication information with the medication collection result information. If they do not match, generate an early warning signal.
[0047] Specifically, when matching medication information and medication retrieval results, the medication name included in the medication information is matched with the medication name included in the medication retrieval results, and the medication dosage included in the medication information is matched with the medication dosage included in the medication retrieval results. If the medication retrieval result corresponding to the image information of the medication retrieval station does not match the medication information in the medication instruction, an early warning message is generated to remind relevant staff to retrieve medication in accordance with regulations.
[0048] In addition, due to the progression of a patient's condition, a patient's medication information may include multiple medications for treatment. The image information of the medication dispensing station makes it easier to match the medication dispensing operations of relevant medical staff with the medication information, thus avoiding the phenomenon of medical staff missing medications during the medication dispensing process.
[0049] In this embodiment of the application, the name of the drug and the dosage are determined by recording the operation when the medication is picked up. The operation is then compared with the name and dosage of the drug corresponding to the medication instruction to determine whether the operation conforms to the specifications of the medication instruction. If the operation does not conform to the medication instruction, an early warning signal is generated to remind relevant staff to standardize the drug picking operation in a timely manner. The medication information is used to verify the drug picking results of relevant medical staff, which helps to improve the standardization of the drug picking operation process of relevant medical staff.
[0050] Further, obtaining medication instructions and determining corresponding medication information based on the medication instructions may specifically include step S1101 (not shown in the attached figures) or step S1102 (not shown in the attached figures), wherein: Step S1101: Obtain the input text information, parse the text information, and obtain the corresponding medication instructions.
[0051] Specifically, the text information is input by relevant staff. The input text information may be generated based on on-site audio data, or it may be generated and input by relevant doctors after diagnosing the patient's condition, or it may be retrieved from the information database in the electronic device.
[0052] When parsing text information, stop words can be removed through methods such as text clustering. Based on the recognition results, the central words and central sentences in the text information can be determined. The central words and central sentences are used to represent the content of the text information. Medication instructions are generated based on the parsed text information, and the sentences in the medication instructions are more concise.
[0053] or, Step S1102: Obtain the on-site audio, parse the on-site audio to obtain the corresponding medication instructions. The on-site audio is the audio of the doctor diagnosing the patient's condition.
[0054] Specifically, the on-site audio consists of the doctor's spoken words during patient treatment, collected by voice acquisition equipment set up at the treatment site and uploaded to electronic devices. When analyzing the on-site audio, semantic recognition can be performed to remove background noise and determine the corresponding medication information, thereby generating medication instructions.
[0055] In this embodiment of the application, on the one hand, medication instructions can be determined through text information. After a doctor diagnoses a patient, the medication data can be directly input, or the medication prescription generated by the doctor based on the patient's condition can be uploaded, which can improve the accuracy of determining medication instructions. On the other hand, by directly converting the audio of the diagnosis into medication instructions, instead of having relevant staff input them, the workload of relevant staff is reduced, and the speed of generating medication instructions can be improved.
[0056] Further, based on the image information, the corresponding medication retrieval result information is determined. This result information includes the medication name and dosage, and may specifically include steps S1301 (not shown in the attached figures), S1302 (not shown in the attached figures), and S1303 (not shown in the attached figures), wherein: Step S1301: Recognize the drug-taking action from the image information.
[0057] Specifically, when recognizing medication-taking actions in image information, the image information can be imported into a human skeletal feature recognition model to identify preset important skeletal points. Based on the distance between these preset important skeletal points, it can be determined whether a medication-taking action exists in the image information. These preset important skeletal points are determined based on historical medication-taking operations, which are the medication-taking actions performed by relevant individuals during past medication-taking processes. For example, the action of drawing medication from a medicine box with a syringe. Since the actions performed by relevant individuals are different, the corresponding medication-taking actions are also different, and thus the distance between the bones corresponding to each action is also different. Historical medication-taking operations include various medication-taking actions. The preset important skeletal points are the wrist, elbow, and facial bone points in medication-taking actions. Based on the distance between these skeletal points, it is easy to determine whether a medication-taking action exists in the image information.
[0058] Step S1302: When a drug-taking action is detected in the image information, acquire a hand image.
[0059] Specifically, images that do not contain the action of taking medicine are discarded, and regions of interest are extracted from images that contain the action of taking medicine. Since the most crucial part of the action of taking medicine is the hand movement, the region of interest is the hand image. When obtaining the hand image from the image information, the head features are first identified, and then the hand image is obtained based on the identification results. Alternatively, the hand image can be obtained by expanding the edges based on the position of the wrist. The method of obtaining the hand image is not specifically limited in this embodiment.
[0060] Step S1303: Perform drug feature recognition on the hand image, and determine the drug retrieval result information based on the recognized drug features.
[0061] Specifically, drug characteristics can include the shape, size, and packaging of the container used to hold the drug, as well as the shape of the drug itself. For example, the shape of a drug can include tablets, granules, powders, capsules, syrups, aqueous solutions, suppositories, ointments, and creams. The name and dosage of the drug can be determined based on its shape and the container used to hold it.
[0062] In addition, hand images can be used to determine whether a drug has deteriorated. For example, tablets include ordinary tablets and sugar-coated tablets. Ordinary tablets (plain tablets) are considered to be non-deteriorated when they are uniform in color, free from moisture absorption, spots, fragments, and deformation. Sugar-coated tablets are considered to be non-deteriorated when they are free from discoloration, sticking, and cracking. Powders, granules, and powders are considered to be non-deteriorated when they are dry, loose, uniform in color and particle size, free from moisture absorption and clumping, mold growth, or discoloration. Capsules are considered to be non-deteriorated when they are free from sticking, breakage, leakage, moisture absorption, sticking, and breakage. Syrups and aqueous solutions are considered to be non-deteriorated when they are free from precipitation, mold, and discoloration. Suppositories, ointments, and creams are considered to be non-deteriorated when they are free from lumps, discoloration, layering, and particle precipitation.
[0063] In this embodiment of the application, image information is filtered by the drug-taking action to obtain image information containing the drug-taking action, and image information that does not contain the drug-taking action can be discarded. Then, drug feature recognition is performed on the acquired hand image to determine the drug-taking result information based on the drug features. By recognizing the operation contained in the image, the accuracy of determining the drug name and dosage can be improved.
[0064] Further, drug feature recognition is performed on the hand image, and based on the drug features, the drug retrieval result information is determined. Specifically, this may include steps S1303a (not shown in the figure), S1303b (not shown in the figure), S1303c (not shown in the figure), S1303d (not shown in the figure), and S1303e (not shown in the figure), wherein: Step S1303a: Perform drug packaging recognition on the hand image to obtain drug packaging features, wherein the drug features include packaging shape and packaging pattern.
[0065] Step S1303b: Based on the drug packaging characteristics, traverse the drug packaging database to determine the drug name corresponding to the drug packaging characteristics.
[0066] Specifically, each drug has a different production site, and to facilitate drug differentiation, different drugs have different drug packaging. By identifying the drug packaging and then traversing the drug database based on the identification results, the name of the drug corresponding to the packaging can be determined. The drug database stores the names of all drugs and their corresponding packaging.
[0067] When identifying drug packaging, the shape of the container holding the drug, as well as the text and patterns marked on the container, can be used for identification. Each pattern corresponds to a different drug, and the drug name can be easily determined based on the drug characteristics.
[0068] Step S1303c: Determine the medicine-retrieving tool based on the hand image, and determine multiple images containing the medicine-retrieving tool from multiple hand images within a preset time period.
[0069] Specifically, when there is a drug-taking action in the image, the hand image may contain the drug-taking tool. For some reagents or solvents, a specific drug-taking tool may be required, such as a syringe. In addition, when it is detected that the required drug is individually packaged, the hand of the relevant staff member is assumed to be the drug-taking tool.
[0070] Multiple images containing medication dispensing tools within a preset time period can be obtained by extracting frames from the medication dispensing video at the dispensing station. The preset time period is the medication dispensing process, from the moment the person arrives at the dispensing station to the moment they take the medication and leave.
[0071] Step S1303d: Determine the dosage of the medication based on multiple images containing the medication dispensing tool.
[0072] Specifically, the dosage is used to characterize the quantity of medication taken. When the required medication is liquid, the dosage is the volume taken; when the required medication is solid, the dosage is the weight taken. Using multiple images containing the medication taking tool helps improve the accuracy of determining the dosage.
[0073] Step S1303e: The name of the medicine and the dosage constitute the medicine dispensing result information.
[0074] Specifically, the name and dosage of the medication are integrated to obtain the medication result information.
[0075] In the embodiments of this application, since there are many types of drugs, and similar drug shapes correspond to different therapeutic effects and names, if the name of the drug is determined solely by the shape of the drug, errors may occur. Determining the name of the drug by the shape and pattern of the drug packaging can improve the accuracy of determining the name of the drug. Furthermore, by varying multiple images containing the drug dispensing tool, the accuracy of determining the dosage of the drug can be improved.
[0076] Further, step S1303d determines the dosage based on multiple images containing the medication dispensing tool, preceded by steps S1 (not shown in the figures) and S2 (not shown in the figures), wherein: Step S1: Sort multiple images containing medicine-retrieving tools according to the image acquisition time to obtain a medicine-retrieving image sequence.
[0077] Step S2: Determine a first image containing the medicine-retrieving tool and a second image containing the medicine-retrieving tool from the medicine-retrieving image sequence. The first image is the image corresponding to the medicine-retrieving process before the medicine is retrieved, and the second image is the image corresponding to the medicine-retrieving process after the medicine is retrieved.
[0078] Specifically, the image acquisition time is the same as the image generation time. By sorting multiple images according to their generation time, it is easier to determine the sequence of images for drug retrieval, which in turn helps to determine the drug retrieval process. Based on the drug retrieval process, it is easier to determine the results of drug retrieval. The sequence of images for drug retrieval facilitates the supervision of the drug retrieval process by relevant personnel, thereby improving the standardization of drug retrieval.
[0079] By analyzing the medication retrieval process, the tool images before and after medication retrieval are determined. The tool image before medication retrieval, i.e., the first image, represents the image of the medication retrieval tool when the relevant staff member moves to the medication retrieval station to retrieve the medication after receiving the medication instruction. At this time, the medication retrieval tool is empty. The tool image after medication retrieval, i.e., the second image, represents the image of the medication retrieval tool when the relevant staff member finishes retrieving the medication according to the medication instruction. At this time, the medication retrieval tool is not empty.
[0080] Among these methods, the dosage is determined based on multiple images containing images of medication dispensing tools, specifically including: The dosage of the medicine is determined based on the first and second images.
[0081] Specifically, by comparing the medication in the dispensing tool in the first image with the medication in the dispensing tool in the second image, the dosage can be easily determined. For example, if the required medication is liquid, then in the first image, there is no liquid in the dispensing tool, i.e., the volume is 0, while in the second image, there is liquid in the dispensing tool, i.e., the required medication has a volume of 2 ml. Using the volume difference between the two images, the dosage can be easily determined.
[0082] In this embodiment of the application, multiple images containing medication dispensing tools are sorted to form a medication dispensing image sequence based on the acquisition time of each image. This facilitates the supervision of the operation process of relevant personnel during medication dispensing based on the medication dispensing image sequence. From multiple images containing medication dispensing tools, the tool image corresponding to the period before and after medication dispensing is determined, and the medication dosage is determined based on the change of medication dispensing tools in the two images, thereby improving the accuracy of determining the medication dosage.
[0083] To further improve the accuracy of determining the medication dosage, the dosage is determined based on the first and second images, specifically including steps Sa1 (not shown in the attached figures) and Sa2 (not shown in the attached figures), wherein: Step Sa1: When the medication corresponding to the medication instruction is liquid, determine the dosage based on the capacity of the dispensing tool in the first image and the capacity of the dispensing tool in the image after dispensing.
[0084] Specifically, when determining the dosage, it is necessary to first determine the state of the drug corresponding to the medication instruction. The state of the drug mainly includes liquid and solid. The dosage calculation methods for liquid and solid drugs are different. When the drug is liquid, the volume change of the sampling tool before and after sampling can be recorded, and the dosage can be determined based on the volume change.
[0085] Step Sa2: When the drug corresponding to the medication instruction is solid, determine the dosage based on the amount of drug in the dispensing tool in the second image, the amount of drug in the dispensing tool in the image after dispensing, and the content standard corresponding to the medication instruction.
[0086] Specifically, when the medication is solid, the dosage can be determined by the quantity of medication. To determine the dosage, the individual dose corresponding to each medication is determined based on its name. The medication name can be obtained from the dosage instructions or identified by identifying the packaging characteristics of the medication held at the dispensing station. For example, ibuprofen tablets are solid medications, each containing 100 mg. Since each medication has different therapeutic effects and therefore different effects on the human body, the dosage for each individual medication can be determined based on its name, thus determining the appropriate dosage.
[0087] In the embodiments of this application, when determining the dosage, the drug state can be determined first. Different drug states correspond to different methods for calculating the dosage. The dosage can be determined based on the drug state, which helps to improve the accuracy of determining the dosage.
[0088] Furthermore, the method also includes steps Sb1 (not shown in the attached figures), Sb2 (not shown in the attached figures), Sb3 (not shown in the attached figures), and Sb4 (not shown in the attached figures), wherein: Step Sb1: Calculate the usage and remaining quantity of each drug in the current cycle.
[0089] Specifically, the current cycle can be one day, three days, or one week, and the user can input the specific duration. The dosage of each medicine within the current cycle can be determined based on the weight of the medicine boxes. By recording the weight of each medicine box on the first day of the current cycle, the weight of each medicine box on the last day of the current cycle, and the density of each medicine, the change in the weight of each medicine box can be determined. Based on this change, the dosage of each medicine within the current cycle can be determined. The remaining amount of each medicine in the current cycle can be determined by the total weight of all medicines on the first day of the current cycle and the dosage of each medicine within the current cycle.
[0090] Step Sb2: Based on the usage of each drug in the current cycle, determine the predicted usage value of each drug in the next cycle.
[0091] Specifically, since patients' illnesses in hospitals may be related to seasonal changes, such as during seasonal transitions or periods of sudden temperature drops, patients' symptoms may be similar. Therefore, the dosage of each drug in the next cycle can be predicted based on the dosage of each drug in the current cycle.
[0092] Step Sb3: Determine the replenishment amount for each drug based on the remaining amount of each drug in the current cycle and the predicted usage of each drug in the next cycle.
[0093] Specifically, the amount of medicine to be replenished is the minimum amount required to meet the needs of the medicine in the next cycle. For example, if there is 400 ml of drug A remaining at the end of the current cycle, and the predicted amount for the next cycle is 1000 ml based on the amount used in the current cycle, then an additional 600 ml of replenishment is required to meet the minimum usage standard for drug A in the next cycle.
[0094] Step Sb4: Generate supply information based on the supply amount and send it to the terminal devices of relevant personnel.
[0095] Specifically, the replenishment information is used to remind relevant staff to replenish the medication at the dispensing station in a timely manner, and to replenish the medication at the dispensing station according to the replenishment information, so as to reduce the probability of insufficient medication during the rescue of patients.
[0096] In addition, the timing of drug replenishment can be recorded, and drugs whose replenishment time exceeds a preset threshold can be replaced to reduce the chance of drug deterioration.
[0097] In this embodiment of the application, the dosage of each drug in the current cycle is used to predict the dosage of the drug in the next cycle, and the amount of drug to be replenished is determined based on the prediction results and the remaining drug quantity, so that the drugs can meet the needs of the next cycle, thereby reducing the probability of insufficient drug quantity at the dispensing station during the rescue of patients.
[0098] The above embodiments describe a method for drug detection from the perspective of process flow. The following embodiments describe a device for drug detection from the perspective of virtual module or virtual unit. For details, please refer to the following embodiments.
[0099] This application provides a device for drug detection, such as... Figure 2 As shown, the device may specifically include an instruction acquisition module 210, an image information acquisition module 220, a drug dispensing result determination module 230, and an early warning generation module 240, wherein: The instruction acquisition module 210 is used to acquire the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient. The medication information includes the medication name and the medication dosage. Image information acquisition module 220 is used to acquire image information of the dispensing station corresponding to the medication instruction; The medication result determination module 230 is used to determine the corresponding medication result information based on the image information. The medication result information includes the name of the medication and the dosage. The early warning generation module 240 matches the medication information with the medication collection result information. If they do not match, an early warning signal is generated.
[0100] In one possible implementation, when acquiring a medication instruction, the instruction acquisition module 210 is specifically used for: The system acquires the input text information, parses the text information, and obtains the corresponding medication instructions. or, The system acquires and analyzes on-site audio to obtain corresponding medication instructions. The on-site audio is the audio recording of the doctor diagnosing the patient's condition.
[0101] In one possible implementation, when determining the corresponding medication retrieval result information based on the image information, the medication retrieval result determination module 230 is specifically used for: Recognize the action of taking medicine from image information; When a drug-taking action is detected in the image information, an image of the hand is acquired; Drug feature recognition is performed on hand images, and the drug retrieval result information is determined based on the recognized drug features.
[0102] In one possible implementation, when determining the medication dispensing result module 230 performs drug feature recognition on the hand image and determines the medication dispensing result information based on the drug features, it is specifically used for: Drug packaging recognition is performed on hand images to obtain drug packaging features, which include packaging shape and packaging pattern. Based on the characteristics of drug packaging, the drug packaging database is traversed to determine the drug name corresponding to the drug packaging characteristics; The tool for taking medicine is determined based on the hand image, and multiple images containing the tool for taking medicine are determined from multiple hand images within a preset time period. The dosage of medication is determined based on multiple images containing medication dispensing tools. The name and dosage of the medication constitute the medication dispensing result information.
[0103] In one possible implementation, the device further includes: The image sorting module is used to sort multiple images containing medicine-retrieving tools according to the image acquisition time to obtain a medicine-retrieving image sequence; The image determination module is used to determine a first image containing the medicine-retrieving tool and a second image containing the medicine-retrieving tool from the medicine-retrieving image sequence. The first image is the image corresponding to the medicine-retrieving process before the medicine is retrieved, and the second image is the image corresponding to the medicine-retrieving process after the medicine is retrieved. Specifically, the module 230 for determining the medication dispensing result, when determining the medication dosage based on multiple images containing the medication dispensing tool, is used for: The dosage of the medicine is determined based on the first and second images.
[0104] In one possible implementation, when determining the medication dosage based on the first image and the second image, the medication determination module 230 is specifically used for: When the medication corresponding to the medication instruction is liquid, the dosage is determined based on the capacity of the dispensing tool in the first image and the capacity of the dispensing tool in the second image. When the medication corresponding to the medication instruction is solid, the dosage is determined based on the amount of medication in the medication dispensing tool in the previous working image, the amount of medication in the medication dispensing tool in the post-dispensing tool image, and the content standard corresponding to the medication instruction.
[0105] In one possible implementation, the device further includes: The statistics module is used to calculate the usage and remaining quantity of each drug in the current cycle. The prediction module is used to determine the predicted dosage of each drug in the next cycle based on the dosage of each drug in the current cycle. The replenishment quantity determination module is used to determine the replenishment quantity for each drug based on the remaining quantity of each drug in the current cycle and the predicted usage value of each drug in the next cycle. The supply information generation module is used to generate supply information based on the supply amount and send it to the terminal devices of relevant personnel.
[0106] This application provides an electronic device, such as... Figure 3 As shown, Figure 3 The illustrated electronic device 300 includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.
[0107] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0108] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0109] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.
[0110] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.
[0111] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0112] This application provides a computer-readable storage medium storing a computer program. When the program is run on a computer, it enables the computer to execute the corresponding content in the aforementioned method embodiments. Compared with related technologies, this application's embodiments determine the name and dosage of the medication by recording the operation during medication retrieval. The operation is compared with the medication name and dosage corresponding to the medication instruction to determine whether the retrieval operation conforms to the specifications of the medication instruction. If the retrieval operation does not conform to the medication instruction, a warning signal is generated to remind relevant staff to standardize the medication retrieval operation. The medication information is used to verify the medication retrieval results of relevant medical personnel, facilitating the standardization of medication retrieval operations by relevant medical personnel.
[0113] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0114] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method of medication detection, the method comprising: include: Obtain the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient. The medication information includes the medication name and dosage. Obtain image information of the dispensing station corresponding to the medication instruction; Based on the image information, the corresponding medication retrieval result information is determined, including the name of the medication and the dosage. The medication information and the medication collection result information are matched; if they do not match, an early warning signal is generated. The step of determining the corresponding medication retrieval result information based on the image information includes: The image information is used to identify the action of taking medicine; When a drug-taking action is detected in the image information, the position of the wrist point is identified, and the edge is expanded according to the position of the wrist point to obtain a hand image; The hand image is subjected to drug packaging recognition to obtain drug packaging features, which include packaging shape and packaging pattern; Based on the drug packaging characteristics, the drug packaging database is traversed to determine the drug name corresponding to the drug packaging characteristics; The medicine-retrieving tool is determined based on the hand image, and multiple images containing the medicine-retrieving tool are determined from multiple hand images within a preset time period. The dosage of medication is determined based on multiple images containing medication dispensing tools. The process of determining the dosage based on multiple images containing the medication dispensing tool also includes: Based on the image acquisition time, multiple images containing medicine-retrieving tools are sorted to obtain a medicine-retrieving image sequence; From the drug-taking image sequence, a first image containing the drug-taking tool and a second image containing the drug-taking tool are determined. The first image is the image corresponding to the drug-taking process before the drug is taken, and the second image is the image corresponding to the drug-taking process after the drug is taken. Determine the dosage of the medication based on the first image and the second image; When the medication corresponding to the medication instruction is a liquid, the dosage is determined based on the capacity of the medication dispensing tool in the first image and the capacity of the medication dispensing tool in the image after medication dispensing. The medication dispensing tool is a syringe. Calculate the usage and remaining quantity of each drug in the current cycle; Based on the dosage of each drug in the current cycle, determine the predicted dosage of each drug in the next cycle; Based on the remaining quantity of each drug in the current cycle and the predicted usage of each drug in the next cycle, determine the replenishment amount corresponding to each drug. Supply information is generated based on the supply amount and sent to the terminal devices of relevant personnel.
2. The method for detecting medication use according to claim 1, characterized in that, The method for obtaining medication instructions includes: The system acquires the input text information, parses the text information, and obtains the corresponding medication instructions. Alternatively, acquire on-site audio, parse the on-site audio to obtain corresponding medication instructions, wherein the on-site audio is the audio of the doctor diagnosing the patient's condition.
3. A medication detection device, characterized by, include: The instruction acquisition module is used to acquire the medication instructions for each patient and determine the corresponding medication information based on the medication instructions for each patient. The medication information includes the medication name and the medication dosage. The image information acquisition module is used to acquire image information of the dispensing station corresponding to the medication instruction; The module for determining the medication retrieval result is used to determine the corresponding medication retrieval result information based on the image information. The medication retrieval result information includes the name of the medication and the dosage. The early warning generation module is used to match the medication information and the medication collection result information. If they do not match, an early warning signal is generated. When determining the corresponding medication retrieval result information based on the image information, the module for determining the medication retrieval result is specifically used for: The image information is used to identify the action of taking medicine; When a drug-taking action is detected in the image information, the position of the wrist point is identified, and the edge is expanded according to the position of the wrist point to obtain a hand image; The hand image is subjected to drug packaging recognition to obtain drug packaging features, which include packaging shape and packaging pattern; Based on the drug packaging characteristics, the drug packaging database is traversed to determine the drug name corresponding to the drug packaging characteristics; The medicine-retrieving tool is determined based on the hand image, and multiple images containing the medicine-retrieving tool are determined from multiple hand images within a preset time period. The dosage of medication is determined based on multiple images containing medication dispensing tools. The image sorting module is used to sort multiple images containing medicine-retrieving tools according to the image acquisition time to obtain a medicine-retrieving image sequence; An image determination module is used to determine a first image containing the medicine-retrieving tool and a second image containing the medicine-retrieving tool from the medicine-retrieving image sequence, wherein the first image is the image corresponding to the medicine-retrieving process before the medicine is retrieved, and the second image is the image corresponding to the medicine-retrieving process after the medicine is retrieved. Determine the dosage of the medication based on the first image and the second image; When the medication corresponding to the medication instruction is a liquid, the dosage is determined based on the capacity of the medication dispensing tool in the first image and the capacity of the medication dispensing tool in the image after medication dispensing. The medication dispensing tool is a syringe. The statistics module is used to calculate the usage and remaining quantity of each drug in the current cycle. The prediction module is used to determine the predicted dosage of each drug in the next cycle based on the dosage of each drug in the current cycle. The replenishment quantity determination module is used to determine the replenishment quantity corresponding to each drug based on the remaining quantity of each drug in the current cycle and the predicted usage value of each drug in the next cycle. The replenishment information generation module is used to generate replenishment information based on the replenishment amount and send it to the terminal devices of relevant personnel.
4. An electronic device, comprising: The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform the method for drug detection according to any one of claims 1-2.
5. A computer-readable storage medium, characterized in that, include: The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1-2.
Citation Information
Patent Citations
Pharmacy inventory management system and method based on image processing
CN108229888A
Drug automatic check system and method
CN108538356A
Dispensing inspection device and method, and program
JP2017127407A
