Drug identification method based on AI visual identification and medication guidance system

Abstract

The embodiment of the invention provides a medicine identification method based on AI visual identification and a medication guidance system, and the method comprises the steps: carrying out the multi-angle image collection of a passing medicine through a plurality of image collection devices disposed on the side surface, the top surface and the bottom surface of a target transmission region when the medicine passes through the target transmission region of a conveyor belt, the conveyor belt is provided with a transparent belt; carrying out drug traceability code identification on the collected image; and if the identification of the traceability code of a certain drug fails, extracting the pictures of the target drug in the images at all angles, performing similarity comparison with the drug image database through the AI image identification model, determining the drug with the highest similarity as the target drug, and outputting the drug traceability information of the target drug. And the integrity of image information is improved through multi-angle image acquisition. Traditional bar code recognition and AI image recognition are combined, the robustness and the fault-tolerant capability of the system are enhanced, and the medication safety and the reliability of the tracing process are guaranteed.

Description

AI-based visual recognition drug identification method and medication guidance system Technical Field

[0001] This application relates to the field of drug identification technology, and in particular to a drug identification method and medication guidance system based on AI visual recognition. Background Technology

[0002] With the development of science and technology and medical technology, drug traceability code recognition has become an important identification method to ensure patient medication safety and improve the efficiency of synchronous scanning of drug traceability codes. Current drug traceability identification equipment, such as drug traceability scanners, primarily relies on barcode and QR code technologies to read information from drug packaging.

[0003] In drug traceability scanning scenarios, drugs may be stacked or placed side by side during the transportation process, which may lead to inaccurate identification of drug traceability codes or even missed detections. This requires manual intervention and cannot be achieved automatically. The above phenomena may lead to reduced drug traceability efficiency and lower identification accuracy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a drug identification method based on AI visual recognition. One or more embodiments of this application also relate to a drug identification device based on AI visual recognition, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] In a first aspect, embodiments of this application provide a drug identification method based on AI visual recognition, comprising: when a drug passes through a target transmission area of ​​a conveyor belt, acquiring multi-angle images of the passing drug using multiple image acquisition devices disposed on the side, top, and bottom surfaces of the target transmission area, wherein the conveyor belt is configured with a transparent material belt; identifying drug traceability codes on the acquired images; if the traceability code identification of a certain drug fails, extracting images of the target drug from the images at various angles, comparing the similarity between the AI ​​image recognition model and a drug image database, determining the drug with the highest similarity as the target drug, and outputting its drug traceability information.

[0006] In one possible implementation, the plurality of image acquisition devices disposed on the side of the target transmission area include at least two image acquisition devices in two acquisition directions.

[0007] In one possible implementation, the method further includes: adaptively adjusting the shooting height of multiple image acquisition devices on the side according to the size of the drug packaging and the stacking posture, so as to completely capture the entire contents of the corresponding surface of the drug.

[0008] In one possible implementation, the method further includes: after completing drug traceability, verifying the drug traceability information with the corresponding prescription information; if there is a discrepancy between the drug type or quantity and the prescription information, triggering an alarm.

[0009] In one possible implementation, the method further includes: uploading the drug traceability information to drug management software on a user terminal for drug usage records or medication reminders.

[0010] In one possible implementation, the AI ​​image recognition model extracts drug image features through a convolutional neural network and performs similarity matching with drug image features pre-stored in the database.

[0011] In one possible implementation, the plurality of image acquisition devices include at least one color camera and / or one depth camera for acquiring color images and three-dimensional morphological information of the drug.

[0012] In one possible implementation, the method further includes: deduplicating the successfully identified drug traceability codes to prevent the same drug from being recorded repeatedly.

[0013] Secondly, this application provides a drug identification device based on AI visual recognition, comprising: an image acquisition module configured to acquire multi-angle images of the passing drug by means of multiple image acquisition devices disposed on the side, top, and bottom surfaces of the target transmission area when the drug passes through the target transmission area of ​​the conveyor belt, wherein the conveyor belt is configured with a transparent material belt; and a drug identification module configured to: identify the drug traceability code on the acquired images; if the traceability code identification of a certain drug fails, extract images of the target drug from the images at various angles, compare the similarity between the AI ​​image recognition model and the drug image database, determine the drug with the highest similarity as the target drug, and output its drug traceability information.

[0014] Thirdly, embodiments of this application provide a computing device, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described drug identification method based on AI visual recognition.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the aforementioned drug identification method based on AI visual recognition.

[0016] Fifthly, embodiments of this application provide a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described AI-based visual recognition drug identification method.

[0017] Sixthly, this application also provides a medication guidance system, including: a conveyor belt equipped with a transparent material belt for transporting medications; multiple image acquisition devices disposed on the sides, top, and bottom of the target transmission area of ​​the conveyor belt, configured to acquire multi-angle images of the passing medications; and a controller configured to: perform medication traceability code recognition on the acquired images; wherein, if the traceability code recognition of a certain medication fails, images of the target medication are extracted from the images at various angles, and a similarity comparison is performed between an AI image recognition model and a medication image database to determine the medication with the highest similarity as the target medication, and its medication traceability information is output; the medication traceability information is uploaded to medication management software on a user terminal for medication usage records or medication administration. The system includes: a display screen configured to display the drug traceability information, the comparison results between the drug traceability information and the corresponding prescription information, the upload results of the drug traceability information, and medication guidance information, wherein the medication guidance information includes one or more of the following: high-alert drug warning information, refrigerated drug warning information, and warnings for pregnant women and drugs contraindicated during pregnancy; a storage module configured to store the drug image database and the comparison results between the drug traceability information and the corresponding prescription information within a target time range; and a height adjustment device including multiple slide rails and an adjuster, wherein multiple image acquisition devices on the side of the target transmission area are slidably mounted on corresponding slide rails, and the adjuster responds to the adjustment command of the controller to adjust the shooting height of the corresponding image acquisition device.

[0018] In one possible implementation, the controller is further configured to verify the drug traceability information against the corresponding prescription information after completing drug traceability; if there is a discrepancy between the drug type or quantity and the prescription information, an alarm will be triggered.

[0019] In the technical solution provided in this application, when the medicine passes through the target transmission area via a conveyor belt made of transparent material, multiple image acquisition devices arranged on the sides, top, and bottom of the area simultaneously acquire multi-angle images of the medicine, ensuring complete coverage of all surfaces of the medicine. Subsequently, the system automatically identifies the medicine traceability code from the acquired images. If the traceability code recognition fails due to obstruction, damage, or other reasons, the system activates an AI image recognition model as a supplementary identification method: extracting image features of the medicine from all angle images, performing a high-precision similarity comparison with a pre-built medicine image database, and finally determining the identity of the target medicine based on the highest similarity principle, successfully outputting its complete medicine traceability information. This solution, through multi-angle image acquisition and the design of the transparent belt, achieves complete coverage of the entire surface of the medicine without blind spots, significantly improving the integrity of image information. Combining traditional barcode recognition with AI image recognition provides a reliable backup identification path when barcode recognition fails, greatly enhancing the robustness and fault tolerance of the entire identification system. Ultimately, it ensures the accurate and complete output of medicine traceability information, effectively guaranteeing medication safety and the reliability of the traceability process.

[0020] This medication guidance system verifies the collected drug information against the prescription information provided by the HIS system. It has the function of verifying the dispensing of boxed drugs and can store the verification image results. The external display screen shows drug verification information such as correct information and dispensing errors, as well as the function of displaying data upload success information. The system has an interface to connect with the rational drug use module of the HIS system. Rational drug use information can be displayed on the all-in-one machine's LCD screen or sent to the patient's smartphone, such as medication guidance information. Important medication reminders are displayed, such as high-alert drugs, refrigerated drugs, reminders for pregnant women, and reminders for contraindications during pregnancy. It becomes a drug dispensing terminal that can intelligently replace some of the pharmacist's verification and medication guidance work, and performs intelligent drug verification, barcode scanning and uploading, storage of dispensing information images, display of medication guidance information, and reminders of important information. It is an all-in-one machine for medication guidance and drug dispensing. Attached Figure Description

[0021] Figure 1 is a scene diagram of a drug identification method based on AI visual recognition provided in an embodiment of this application; Figure 2 is a schematic diagram of a light shield setting provided in an embodiment of this application; Figure 3 is a schematic diagram of an oblique conveying provided in an embodiment of this application; Figure 4 is a flowchart of a drug identification method based on AI visual recognition provided in an embodiment of this application; Figure 5 is a structural schematic diagram of a drug identification device based on AI visual recognition provided in an embodiment of this application; Figure 6 is a structural block diagram of a computing device provided in an embodiment of this application. Detailed Implementation

[0022] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0023] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a” and “the” as used in one or more embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0024] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0025] This application provides a drug identification method based on AI visual recognition, a drug identification device based on AI visual recognition, a computing device, and a computer-readable storage medium. It also provides a medication guidance system, which will be described in detail in the following embodiments.

[0026] Referring to Figure 1, Figure 1 shows a scenario diagram of a drug identification method based on AI visual recognition according to an embodiment of this application.

[0027] In the application scenario shown in Figure 1, the system may include a conveyor belt S1, a conveyed medicine P1, multiple image acquisition devices, a height adjustment device L1, and a height adjustment device L2. The multiple image acquisition devices are respectively disposed on the side, top, and bottom surfaces of the target transmission area of ​​the conveyor belt. Specifically, the multiple image acquisition devices disposed on the side of the target transmission area of ​​the conveyor belt may include a first image acquisition device C1 disposed on one side of the conveyor belt S1 and a second image acquisition device C2 disposed on the other side of the conveyor belt S1. The angle between the transmission direction of the conveyor belt S1 and the horizontal line is within a preset angle range. For example, this angle can be freely adjusted between 0° and 45°, or it can be set to a fixed angle. The first image acquisition device C1 and the second image acquisition device C2 are respectively disposed on the height adjustment devices L1 and L2. Both the height adjustment devices L1 and L2 include a slide rail and an adjuster. The adjuster is used to control the image acquisition device to slide on the slide rail to adjust the shooting height of the image acquisition device L1 and the height adjustment device L2. The third image acquisition device C3 is installed on the top surface of the target transmission area of ​​the conveyor belt and is used to acquire images of the medicine being transported at the top of the conveyor belt. The fourth image acquisition device C4 is installed on the bottom surface of the target transmission area of ​​the conveyor belt and is used to acquire images of the medicine being transported at the bottom of the conveyor belt.

[0028] This application embodiment also provides a medication guidance system, including: a conveyor belt, configured with a transparent material for transporting medications; multiple image acquisition devices, disposed on the sides, top, and bottom of the target transport area of ​​the conveyor belt, configured to acquire multi-angle images of the passing medications; and a controller configured to: perform medication traceability code recognition on the acquired images; wherein, if the traceability code recognition of a certain medication fails, images of the target medication from various angles are extracted, and a similarity comparison is performed between an AI image recognition model and a medication image database to determine the medication with the highest similarity as the target medication, and its medication traceability information is output; the medication traceability information is uploaded to medication management software on a user terminal for medication usage records or medication reminders. The display screen is configured to display the drug traceability information, the comparison results between the drug traceability information and the corresponding prescription information, the upload results of the drug traceability information, and medication guidance information, wherein the medication guidance information includes one or more of the following: high-alert drug warning information, refrigerated drug warning information, and warning information for pregnant women and contraindications during pregnancy; the storage module is configured to store the drug image database and the comparison results between the drug traceability information and the corresponding prescription information within a target time range; the height adjustment device includes multiple slide rails and an adjuster, wherein multiple image acquisition devices on the side of the target transmission area are slidably mounted on the corresponding slide rails, and the adjuster responds to the adjustment command of the controller to adjust the shooting height of the corresponding image acquisition device.

[0029] In some embodiments, the specific architecture of the medication guidance system provided in this application may be the same as or similar to the architecture shown in the scenario diagram of Figure 1.

[0030] Figure 2 is a schematic diagram of a light shield setting provided in one embodiment of this application.

[0031] Referring to Figure 2, a light shield G1 can be added to the medication guidance system shown in Figure 1.

[0032] In some embodiments, the light shield G1 can cover all or part of the components of the medication guidance system shown in FIG1. ​​This part of the components includes at least multiple image acquisition devices. This prevents light spots from appearing in the images acquired by the multiple image acquisition devices when they capture images of the medicine P1 passing through the target transmission area, as external light sources illuminate the medicine's outer packaging. Furthermore, the light shield effectively blocks interference from ambient light and stray light, creating a relatively stable and uniform light field environment for the camera. This significantly reduces reflections, glare, and shadows in the images, ensuring clear product image features and consistent contrast. This greatly improves the accuracy and reliability of image recognition and detection algorithms, ultimately guaranteeing the stability and high efficiency of the sorting or quality inspection process.

[0033] In some embodiments, as shown in FIG2, the display screen B1 of the medication guidance system may be disposed on the side of the light shield G1.

[0034] Figure 3 is a schematic diagram of oblique transmission provided in one embodiment of this application.

[0035] Referring to Figure 3, in some embodiments, the conveyor belt S2 can be tilted at an angle of θ. Setting the conveyor belt to an inclined configuration, especially in the drug identification stage, allows for the automatic orientation and separation of drugs using gravity. On the inclined surface, drugs will naturally slide down and spread out in a single layer, effectively avoiding problems such as stacking, sticking, or inconsistent postures common on horizontal conveyor belts. This provides the downstream image acquisition device with an unobstructed and more uniform shooting perspective, significantly improving the accuracy and efficiency of identifying key features such as drug appearance and label text. Simultaneously, the inclined structure also facilitates rapid automatic sorting and flow guidance after identification.

[0036] Referring to Figure 3, the inclined conveyor belt S2 is surrounded by a protective frame G2 made of transparent material. The first image acquisition device C1, the second image acquisition device C2, the third image acquisition device C3, and the fourth image acquisition device C4 inside the outer casing 21 are all used to scan the medicine passing through the protective frame G2 to obtain image information of the medicine.

[0037] In some embodiments, the controller in the system can be a near-ground connected controller or a controller located at a remote location; this application does not impose any restrictions on this.

[0038] The specific functions of the medication guidance system provided in this application will be described in detail below through specific embodiments.

[0039] In Example 1, when the medicine to be identified is conveyed to the acquisition area of ​​the image acquisition device (i.e., the target transmission area of ​​the conveyor belt) via a conveyor belt, multiple image acquisition devices installed on the sides, top, and bottom of the target transmission area of ​​the conveyor belt acquire images of the passing medicine from multiple angles. If a particular image acquisition device cannot capture the entire contents of the medicine, the controller adjusts the height of the corresponding image acquisition device via a height adjustment device to ensure that the image acquisition device can capture the entire image content of the medicine.

[0040] The controller identifies the traceability code (such as a GS1 standard QR code) on the drug packaging based on the acquired images. If recognition fails due to wear and tear, the controller extracts images of the target drug (i.e., the drug to be identified) from various angles, calls an AI image recognition model (e.g., a deep learning model based on ResNet-50), and compares its similarity with a drug image database (containing a large number of drug image features) stored in the storage module. If the comparison result shows a 98.3% similarity to antihypertensive drug A, and similarities to other drugs are all below 60%, the controller can determine that the current drug's traceability result is antihypertensive drug A and outputs the drug traceability information.

[0041] Furthermore, the controller can also upload the drug traceability information to the user's terminal drug management software for drug usage records or medication reminders.

[0042] On the other hand, the controller can also display the drug traceability information on the display screen. For example, the drug traceability information is as follows: Drug name: Antihypertensive drug A; Production batch number: 00000000001; Manufacturer: Pharmaceutical Factory A; Expiry date: May 2028.

[0043] After the drug traceability information is successfully uploaded, the upload result can also be displayed on the screen.

[0044] Furthermore, the controller can also control the display screen to show the medication instructions for the antihypertensive drug A: Contraindicated in pregnant women and during pregnancy.

[0045] After obtaining the drug traceability information, the controller can also verify the drug traceability information against the patient's corresponding prescription information: Patient Zhang San's prescription drug is: antihypertensive drug A; actual traceable drug: antihypertensive drug A; comparison result: the traceability result is consistent with the prescription.

[0046] Furthermore, the controller can also display the comparison results of the drug traceability information and the corresponding prescription information on the display screen.

[0047] Once the traceability results are confirmed to be consistent with the prescription, the controller can also display medication reminder information on the screen: Patient Zhang San will take two courses of treatment starting from October 1, 2025; take one tablet once each morning and evening after meals.

[0048] Furthermore, the controller can also store the comparison results of the drug traceability information and the corresponding prescription information in the storage module and retain the target time range, such as 6 months, for easy viewing later.

[0049] This application also provides a drug identification method based on AI visual recognition, which can be applied to the medication guidance system.

[0050] Referring to Figure 4, Figure 4 shows a flowchart of a drug identification method based on AI visual recognition according to an embodiment of this application, which specifically includes the following steps: Step 401: When the drug passes through the target transmission area of ​​the conveyor belt, multiple image acquisition devices set on the side, top and bottom surfaces of the target transmission area are used to acquire multi-angle images of the passing drug.

[0051] The conveyor belt is made of transparent material.

[0052] Step 402: Perform drug traceability code recognition on the collected images; if the traceability code recognition of a certain drug fails, extract images of the target drug from various angles, compare the similarity between the AI ​​image recognition model and the drug image database, determine the drug with the highest similarity as the target drug, and output its drug traceability information.

[0053] In some embodiments, the plurality of image acquisition devices disposed on the side of the target transmission area include at least two image acquisition devices with acquisition directions. In one embodiment, the two image acquisition devices with acquisition directions can be the first image acquisition device C1 and the second image acquisition device C2 in the scenario shown in FIG1.

[0054] Based on the content of step 401, multi-angle image acquisition can be realized, in which an all-round image acquisition system can be constructed, as follows: Spatial layout: On the sides (including left, right, front, and rear), top, and bottom surfaces of the "target transmission area" (i.e., the dedicated recognition station) of the conveyor belt, multiple high-resolution industrial cameras or scanners are precisely deployed to form an image acquisition array covering the entire recognition area.

[0055] Transparent belt design: The conveyor belt is made of a highly transparent and strong material (such as transparent PVC or PU). This allows the bottom image acquisition device installed under the conveyor belt to penetrate the belt without obstruction and clearly capture images of the bottom of the medicine.

[0056] Collaborative workflow: When the medicine enters the target transmission area via the conveyor belt, the system triggers all image acquisition devices to perform synchronous or near-synchronous shooting. This ensures that the medicine's status is captured from different angles at the same moment (or within a very short time interval).

[0057] With this layout, regardless of the posture of the medicine on the conveyor belt (upright, sideways, or tilted) or which surface its traceability code is affixed to, at least one or more acquisition devices at different angles can capture clear images of the code or the appearance features of the medicine.

[0058] Traditional single-sided or top-side scanning methods are prone to having their codes obscured due to the random placement of the medicines. This method, through a six-sided encircling acquisition layout, theoretically achieves 100% image coverage of all outer surfaces of the medicines, fundamentally eliminating the "blind spots" in image acquisition.

[0059] By combining a transparent strap with a bottom-mounted camera, the bottom of pharmaceutical products has been brought into the realm of routine, automated identification for the first time. This is crucial for many pharmaceutical packages that print important markings (including traceability codes) on the bottom, solving a long-standing industry pain point of automating the collection of bottom information.

[0060] The collection of images acquired from multiple angles is not only used for scanning codes, but also constitutes a multi-perspective "identity profile" of the drug. This rich and complete data is the fundamental prerequisite for the successful execution of subsequent AI image recognition. If only a blurry or partial image is captured from one angle, the accuracy of AI recognition will drop significantly.

[0061] Based on the content of step 402, dual-mode recognition and AI-assisted traceability can be achieved. Traceability can be implemented through multiple levels, as follows: First level: Conventional traceability code recognition: The system first calls a high-efficiency barcode / QR code recognition algorithm to quickly parse all images collected in step 401 and extract the drug traceability code. As long as the code is successfully read from any angle of the image, it is considered a successful recognition, and traceability information is immediately output. This process is fast and efficient, making it the preferred solution for handling the vast majority of cases.

[0062] Second level: AI image recognition fault tolerance mechanism: Triggering condition: The system automatically switches to AI recognition mode when and only when the traceability code cannot be successfully recognized from all angles (possibly due to severe damage, tearing, complete obstruction, or printing quality issues).

[0063] The system will use object detection technology to select the image area of ​​the "target drug" from images from various angles.

[0064] Furthermore, these multi-angle images are fed into a pre-trained AI image recognition model (typically based on a deep convolutional neural network). This model is able to extract deep visual features of the pharmaceutical packaging (such as color, shape, logo, pattern, text layout, etc.).

[0065] Furthermore, the extracted features are compared with standard templates in the drug image database (a pre-built and continuously updated database containing multi-angle standard images of all drugs on sale).

[0066] The system identifies the standard drug with the highest similarity to the drug to be identified, recognizes it as the "target drug", and outputs the drug traceability information corresponding to the standard drug.

[0067] This technical solution creatively combines rule-driven code recognition with data-driven AI recognition, forming a "primary-backup collaborative" recognition architecture. When the primary path (code recognition) fails, the backup path (AI recognition) can immediately take over, ensuring that the system maintains a very high overall recognition success rate even in complex and non-ideal real-world scenarios (such as damaged codes). In the extreme case of completely unusable traceability codes, this solution offers a novel and effective solution. It does not rely on the code itself, but rather uses the "overall appearance" of the medicine as its identifier, achieving a conceptual upgrade from "code recognition" to "object recognition." The entire recognition and fault-tolerant process requires no manual intervention; the system can automatically determine the recognition status and switch recognition strategies. This significantly reduces the frequency of manual review due to recognition failures, improves overall efficiency, and is a key step towards intelligent management of the entire drug traceability process.

[0068] In some embodiments, the method further includes: adaptively adjusting the shooting height of the multiple image acquisition devices on the side according to the size of the drug packaging and the stacking posture, so as to completely capture the entire contents of the corresponding surface of the drug.

[0069] In some embodiments, visual sensors (such as 3D profilometers or binocular cameras) installed on the system can detect the packaging size and stacking posture of medicines on the conveyor belt in real time, and feed this information back to the system controller. The controller can calculate the optimal shooting height of each side image acquisition device according to a preset algorithm and drive its precision motion module to perform adaptive lifting and positioning. This allows the image acquisition device to always capture images from an ideal perspective that faces and completely covers the side surface of the medicine. This implementation fundamentally solves the industry problem of incomplete, distorted, or unreadable traceability codes caused by medicines of different sizes, disorderly stacking, or mutual obstruction. Its technical effect is to significantly improve the quality and consistency of image acquisition, ensuring the accuracy of subsequent traceability code recognition and AI visual recognition, and the overall reliability of the system.

[0070] In some embodiments, the method further includes: after completing drug traceability, verifying the drug traceability information with the corresponding prescription information; if there is a discrepancy between the drug type or quantity and the prescription information, triggering an alarm.

[0071] After the system successfully obtains the traceability information of a drug (such as drug name, specifications, and approval number) through barcode scanning or AI recognition, the system controller automatically compares this traceability information with the current prescription information to be processed retrieved from the Hospital Information System (HIS). Key items for verification include drug type and quantity. If the system's logical judgment indicates any inconsistency between the identification result and the prescription requirements (e.g., the presence of over-the-counter drugs, incorrect drug specifications, or quantity discrepancies), the controller immediately generates and executes an alarm signal, driving an audible and visual alarm to issue a warning, while simultaneously displaying error details on the software interface. The technical advantage of this process lies in constructing a fully automated closed-loop security verification mechanism of "identification-verification-interception," upgrading manual review to precise and efficient automated verification, completely eliminating medication dispensing errors at the end of the process, and greatly improving medication safety and prescription execution accuracy.

[0072] In some embodiments, the method further includes: uploading drug traceability information to drug management software on a user terminal for drug usage records or medication reminders.

[0073] After completing drug identification and traceability, the system can upload and synchronize the acquired drug traceability information (such as drug name, production batch number, expiration date, etc.) in real time to the drug management software running on the user terminal (such as the computer at the nurse station or mobile device) via a built-in network communication module (such as Wi-Fi or Ethernet). The software then automatically binds this information to specific patients or wards and records it in the electronic medication record. Simultaneously, based on the drug information and preset medical orders, it can automatically generate and push medication reminders. The technical effect of this process is that it bridges the "last mile" from drug identification to clinical management, achieving seamless integration of drug flow and information flow. It not only automatically generates accurate drug usage records, improving the integrity and traceability of medical data, but also proactively ensures the timeliness and accuracy of medication administration through intelligent reminder functions, thereby optimizing the overall medical process and enhancing patient medication safety.

[0074] In some embodiments, the AI ​​image recognition model extracts drug image features through a convolutional neural network and performs similarity matching with drug image features pre-stored in the database.

[0075] The image of the drug to be identified is input into a deep convolutional neural network model pre-trained with massive amounts of drug packaging data. This network, through its multi-layered convolutional and pooling structure, automatically and efficiently extracts highly discriminative deep visual features from the drug packaging and outputs a condensed high-dimensional feature vector. Subsequently, the system calculates the similarity between this feature vector and all standard drug feature vectors pre-stored in a drug feature database, ultimately selecting the drug with the highest similarity as the identification result. This deep learning-based approach achieves a leap from "barcode recognition" to "object recognition." Even if the traceability code on one side of the drug is missing, damaged, or obscured, the system can accurately identify it based on the overall appearance of the packaging, thereby greatly improving the robustness, fault tolerance, and overall recognition success rate of the drug traceability system.

[0076] In some embodiments, the plurality of image acquisition devices include at least one color camera and / or one depth camera for acquiring color images and three-dimensional morphological information of the drug.

[0077] In some embodiments, the method further includes: deduplicating the successfully identified drug traceability codes to prevent the same drug from being recorded repeatedly.

[0078] The system assigns a temporary, unique identifier to each successfully identified drug traceability code and compares it with a preset time-space deduplication window (e.g., based on the drug's physical size and conveyor belt speed, ensuring the same code cannot appear repeatedly within a specific time interval). When the same traceability code is identified and uploaded multiple times by different image acquisition devices within a short time window, the system quickly identifies and discards subsequent duplicate records using efficient hash comparison or cache matching algorithms, retaining only one valid traceability record for each unique drug. This method fundamentally eliminates data redundancy caused by multiple angles and cameras simultaneously capturing the same drug, ensuring a strict one-to-one correspondence between the backend database records and the physical drugs. This significantly improves the accuracy and reliability of inventory statistics, prescription verification, and traceability information, while avoiding unnecessary waste of storage space and computing resources.

[0079] Corresponding to the above method embodiments, this application also provides an embodiment of a drug identification device based on AI visual recognition. Figure 5 shows a schematic diagram of the structure of a drug identification device based on AI visual recognition provided in one embodiment of this application. As shown in Figure 5, the device includes: an image acquisition module 501, configured to acquire multi-angle images of the passing drug through multiple image acquisition devices set on the side, top, and bottom surfaces of the target transmission area when the drug passes through the target transmission area of ​​the conveyor belt, wherein the conveyor belt is made of transparent material; a drug identification module 502, configured to: identify the drug traceability code in the acquired images; if the traceability code identification of a certain drug fails, then extract the image of the target drug in the images from various angles, compare the similarity with the drug image database through an AI image recognition model, determine the drug with the highest similarity as the target drug, and output its drug traceability information.

[0080] The above is an illustrative scheme of a drug identification device based on AI visual recognition according to this embodiment. It should be noted that the technical solution of this drug identification device based on AI visual recognition and the technical solution of the drug identification method based on AI visual recognition described above belong to the same concept. For details not described in detail in the technical solution of the drug identification device based on AI visual recognition, please refer to the description of the technical solution of the drug identification method based on AI visual recognition described above.

[0081] Figure 6 shows a structural block diagram of a computing device 600 according to an embodiment of this application. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0082] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0083] In one embodiment of this application, the aforementioned components of the computing device 600, as well as other components not shown in FIG. 6, may be interconnected, for example, via a bus. It should be understood that the computing device structural block diagram shown in FIG. 6 is merely for illustrative purposes and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0084] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0085] The processor 620 executes computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned AI-based visual recognition-based drug identification method. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned AI-based visual recognition-based drug identification method belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the aforementioned AI-based visual recognition-based drug identification method.

[0086] An embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described AI-based visual recognition-based drug identification method.

[0087] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the AI-based visual recognition drug identification method described above. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the AI-based visual recognition drug identification method described above.

[0088] An embodiment of this application also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described drug identification method based on AI visual recognition.

[0089] The above is an illustrative example of a computer program in this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the aforementioned AI-based visual recognition-based drug identification method. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned AI-based visual recognition-based drug identification method.

[0090] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0091] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0092] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this application.

[0093] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0094] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of the embodiments of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A drug identification method based on AI visual recognition, characterized in that, include: When medicines pass through the target transmission area of ​​the conveyor belt, multiple image acquisition devices installed on the side, top, and bottom surfaces of the target transmission area capture multi-angle images of the passing medicines. The conveyor belt is equipped with a transparent material belt. The captured images are used to identify the medicine traceability code. If the traceability code of a certain medicine fails to be identified, the images of the target medicine from various angles are extracted, and the similarity is compared with the medicine image database using an AI image recognition model. The medicine with the highest similarity is determined as the target medicine, and its medicine traceability information is output.

2. The method according to claim 1, characterized in that, The multiple image acquisition devices disposed on the side of the target transmission area include at least two image acquisition devices in two acquisition directions.

3. The method according to claim 1, characterized in that, The method further includes: adaptively adjusting the shooting height of multiple image acquisition devices on the side according to the size of the drug packaging and the stacking posture, so as to completely capture the entire content of the corresponding surface of the drug.

4. The method according to claim 1, characterized in that, The method further includes: after completing drug traceability, verifying the drug traceability information with the corresponding prescription information; if there is a discrepancy between the drug type or quantity and the prescription information, triggering an alarm.

5. The method according to claim 1, characterized in that, The method further includes uploading the drug traceability information to the drug management software on the user terminal for drug usage records or medication reminders.

6. The method according to claim 1, characterized in that, The AI ​​image recognition model extracts drug image features through a convolutional neural network and performs similarity matching with drug image features pre-stored in the database.

7. The method according to claim 1, characterized in that, The plurality of image acquisition devices include at least one color camera and / or one depth camera, used to acquire color images and three-dimensional morphological information of the drug.

8. The method according to claim 1, characterized in that, The method also includes: deduplicating the successfully identified drug traceability codes to avoid the same drug being recorded repeatedly.

9. A medication guidance system, characterized in that, include: A conveyor belt, made of transparent material, is used to transport medicines. Multiple image acquisition devices are installed on the sides, top, and bottom of the target transport area of ​​the conveyor belt, configured to acquire multi-angle images of the passing medicines. A controller is configured to: identify the medicine traceability code in the acquired images; if the traceability code of a certain medicine fails to be identified, extract images of the target medicine from various angles, compare them with a medicine image database using an AI image recognition model, determine the medicine with the highest similarity as the target medicine, and output its medicine traceability information; upload the medicine traceability information to the user terminal's medicine management software for medicine usage records or medication reminders; and display screen is configured to display... The drug traceability information, the comparison results between the drug traceability information and the corresponding prescription information, the upload results of the drug traceability information, and the medication guidance information, wherein the medication guidance information includes one or more of the following: high-alert drug warning information, refrigerated drug warning information, and warning information for pregnant women and contraindications during pregnancy; a storage module configured to store the drug image database and the comparison results between the drug traceability information and the corresponding prescription information within a target time range; a height adjustment device including multiple slide rails and an adjuster, wherein multiple image acquisition devices on the side of the target transmission area are slidably mounted on corresponding slide rails, and the adjuster responds to the adjustment command of the controller to adjust the shooting height of the corresponding image acquisition device.

10. The medication guidance system according to claim 9, characterized in that, The controller is also configured to verify the drug traceability information against the corresponding prescription information after completing drug traceability; if there is a discrepancy between the drug type or quantity and the prescription information, an alarm will be triggered.

Images