Robot medicine identification method based on histogram feature identification
By using the combination of ORB features and HSV color histogram recognition method in the pharmacy automatic dispensing robot, the problem of high computing power requirements and insufficient robustness of the deep learning model on embedded devices is solved, and efficient and accurate drug recognition is achieved, suitable for complex scenarios and embedded environments.
Patent Information
- Application Number
- CN202510269648.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-07
- Publication Date
- 2025-06-24
AI Technical Summary
In pharmacy automatic dispensing robots, deep learning models are difficult to achieve fast and accurate drug recognition on embedded devices due to high computing power requirements, dependence on high-resolution images, real-time challenges, insufficient robustness and high data labeling costs.
The recognition method based on ORB local features and HSV three-channel color histogram is adopted, combined with the weighted matching strategy, to achieve feature extraction and matching of drugs, which is suitable for embedded devices with limited computing power.
It significantly improves the accuracy and robustness of drug identification in complex scenarios, reduces computing power requirements, improves real-timeness, and reduces data labeling costs, making it suitable for embedded environments.
Smart Images
Figure CN120198694A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer vision and automation technology, and particularly to a robot drug recognition method based on histogram feature recognition. Background Art
[0002] In recent years, with the rapid development of computer vision technology, drug recognition technology has been increasingly widely applied in the field of pharmacy automation. Especially in an automatic drug dispensing robot system, accurately and quickly recognizing drugs is the key to ensuring medication safety and improving the efficiency of drug dispensing. Currently, deep learning technology, especially convolutional neural networks (CNNs), has achieved remarkable results in the field of image recognition and is widely used in various image classification and detection tasks, and drug recognition is no exception.
[0003] Deep learning methods and their characteristics are as follows: Classification models based on CNN: Classical network architectures such as ResNet, VGG, and Inception extract high-dimensional features through multiple convolutional and pooling operations and can achieve high-precision classification on large-scale datasets. These models usually train each drug as a category and are suitable for recognition tasks with relatively fixed categories. Such models have good robustness under standard conditions and can classify the outer packaging of drugs with complex textures relatively accurately. Object detection models: Models such as Faster R-CNN, YOLO, and SSD can simultaneously complete drug category recognition and localization. They accurately mark the positions of drugs in images through region extraction or fully convolutional methods, especially suitable for scenarios where multiple drugs exist simultaneously. Among them, the YOLO series of models are famous for their real-time performance and are suitable for task scenarios that require quick responses. Lightweight deep learning models: To adapt to embedded and mobile devices, lightweight models such as MobileNet and ShuffleNet have been developed. These models improve the inference speed by reducing the complexity of convolutional operations while maintaining relatively high classification accuracy and are more suitable for resource-constrained environments.
[0004] Vision Models based on Transformers: Vision Transformers (ViT) utilize self-attention mechanisms to capture image features globally and have strong adaptability to complex backgrounds of drugs. They often outperform CNNs in terms of classification accuracy but have higher requirements for hardware resources.
[0005] Although deep learning technology has demonstrated excellent performance in drug recognition, its application to embedded devices (such as automatic drug dispensing robots in pharmacies) still faces the following limitations:
[0006] High computing power requirements: Deep learning models usually require GPUs or TPUs for efficient inference, while embedded devices typically only have low-power CPUs, making it difficult to meet the computing resource requirements of deep learning models.
[0007] Dependence on high-resolution images: Many deep learning models rely on high-definition images during training to capture more details, while the camera resolution on embedded devices is usually low, which can lead to a decrease in recognition accuracy.
[0008] Real-time challenges: In scenarios that require quick responses, the inference speed of deep learning models may be difficult to meet the requirements, and even lightweight models may experience a certain degree of latency.
[0009] Lack of robustness: In the presence of complex lighting conditions, occlusion, and background interference, the recognition accuracy of deep learning models may be significantly reduced.
[0010] High cost of data annotation: There is a wide variety of drug types and a fast update rate, and providing large-scale high-quality annotated data for deep learning models requires a large amount of manpower and time.
[0011] Based on this, the present invention proposes a robot drug recognition method based on histogram feature recognition. By fusing ORB local features and HSV three-channel color histograms and combining a weighted matching strategy, the recognition accuracy and robustness in complex scenarios are significantly improved. Summary of the Invention
[0012] The present invention provides a robot drug recognition method based on histogram feature recognition, aiming to solve the problem of quickly and accurately recognizing drugs by a pharmacy automatic dispensing robot under limited computing power and low camera resolution. This method achieves efficient recognition with low computing power consumption, can adapt to the embedded environment, and significantly improves the safety and accuracy of automatic drug dispensing.
[0013] To achieve the objectives of the present invention, the following technical solutions are adopted:
[0014] The technical solution of the present invention includes an image acquisition module, an image preprocessing module, a feature extraction and matching module, a result judgment module, and a terminal interaction module. The specific composition and implementation are as follows:
[0015] 1. Image acquisition module
[0016] Hardware device:
[0017] Equip a camera installed above the automatic drug dispensing robot to collect images of drugs in real time. The camera has the following characteristics:
[0018] The resolution is required to be above 640×480.
[0019] The frame rate is set to 10fps to meet the requirements of the robot's real-time operation.
[0020] Image capture method:
[0021] When the robot grasps the medicine, the camera captures two frames of images per second for subsequent feature matching and recognition tasks. The captured images are transmitted to the embedded processor via USB or serial port.
[0022] 2. Image preprocessing module
[0023] Denosing and adjustment: Convert to grayscale images to improve the efficiency of ORB feature extraction, while retaining the color images for color histogram calculation.
[0024] 3. Feature extraction and matching module
[0025] ORB feature extraction:
[0026] Use the ORB (Oriented FAST and Rotated BRIEF) algorithm to extract key points and their descriptors from grayscale images. The ORB algorithm has high computational efficiency and can run quickly on embedded devices.
[0027] ORB (Oriented FAST and Rotated BRIEF) is a commonly used feature extraction and description algorithm, designed specifically for high efficiency, combining the advantages of the FAST feature detector and the BRIEF feature descriptor, while optimizing and improving them.
[0028] The core of ORB lies in solving the deficiencies of traditional algorithms in terms of speed and robustness, while taking into account rotational invariance and scale invariance. In the feature extraction stage, ORB first detects corner points through an improved FAST (Features from Accelerated Segment Test), and then uses a Harris score method based on direction to sort the feature points and select points with higher quality. To improve the stability of the descriptor, ORB performs rotational correction on the neighborhood of each feature point before calculating the BRIEF descriptor, thus achieving robustness to rotation.
[0029] The feature descriptor of ORB is based on BRIEF, but to enhance performance, ORB introduces a learning mechanism and optimizes the binary pattern using a process called weighted learning, making the descriptor more sensitive and accurate to image changes. In addition, ORB uses the Hamming distance as a metric in the matching stage, greatly improving the speed and efficiency of feature matching. The calculation method of the Hamming distance matches the form of binary vectors and is very suitable for low-resource environments.
[0030] As an all-round algorithm, ORB does not rely on patents and can replace some more complex feature extraction algorithms (such as SIFT and SURF). It performs excellently in scenarios with high real-time requirements and limited resources (such as mobile devices and embedded systems) and is widely used in fields such as image recognition, object tracking, and 3D reconstruction. Generally speaking, ORB has become an important tool in the field of computer vision with its high efficiency and openness.
[0031] Color histogram calculation:
[0032] Calculate the three-channel histograms of the drug image in the HSV color space respectively and normalize them to reduce the influence of illumination changes.
[0033] The color histogram is used to match the overall color distribution characteristics of the drug, thus making up for the deficiencies of ORB feature point matching on flat-packaged drugs with less texture.
[0034] Feature matching:
[0035] Use the Hamming distance for feature matching of ORB descriptors, adopt the KNN (k-nearest neighbor) algorithm to select two optimal matching points, and combine the ratio test to filter out low-quality matches.
[0036] Use the Bhattacharyya distance to calculate the similarity score of two images for color histogram matching.
[0037] Matching fusion:
[0038] Weightedly fuse the matching results of ORB and color histogram, and finally output the candidate drug category with the highest matching score. This weighted fusion can be optimized through experimental data to adapt to specific drug recognition requirements.
[0039] 4. Result judgment module
[0040] Judgment of matching results: Select the one with the highest matching score as the output of the matching result.
[0041] 5. Terminal interaction module
[0042] Data transmission: Transmit the recognition result to the terminal through the UART or wireless module for displaying the drug name and other relevant information.
[0043] User interaction: Display the comparison between the drug image and the recognition result on the terminal interface, and at the same time record the recognized drug name, time, and matching score for subsequent traceability.
[0044] To further improve the system adaptability, the present invention can introduce the following alternative solutions:
[0045] 1. Feature extraction optimization
[0046] Replace the ORB feature extraction with AKAZE feature extraction to improve the matching accuracy for drug packages with less texture.
[0047] 2. Improvement of histogram fusion
[0048] Introduce the LAB color space histogram or improve it to a local color histogram to enhance the ability to distinguish complex backgrounds.
[0049] 3. Upgrade of the matching algorithm
[0050] Use the RANSAC method in graph matching to eliminate incorrect matching point pairs and further improve the recognition accuracy.
[0051] 4. Hardware expansion
[0052] Adopt a higher-resolution camera or a dual-camera solution, combined with a stereo matching algorithm to enhance the recognition accuracy.
[0053] Compared with related technologies, the robot drug recognition method based on histogram feature recognition provided by the present invention has the following beneficial effects:
[0054] 1. Low computing power requirement: The present invention adopts ORB feature matching and color histogram matching. These algorithms are all traditional methods with low computational overhead and can operate efficiently on embedded devices with limited computing power.
[0055] 2. Strong robustness: By combining the color histogram with local features, the present invention can still maintain a high recognition accuracy under low-resolution images and complex lighting conditions.
[0056] 3. Excellent real-time performance: The running speed of the algorithm of the present invention is faster than that of deep learning models, and it can achieve real-time processing of two frames of images per second, which is suitable for scenarios where automatic drug dispensing robots need to respond quickly.
[0057] 4. Low data annotation requirement: The present invention does not need to rely on large-scale dataset training, reduces the workload of data annotation, and at the same time avoids the cost of model retraining.
[0058] 5. Strong stability: In scenarios where the camera resolution is low and the light changes greatly, the present invention effectively improves the stability through multi-feature matching methods and adapts to the drug recognition task in the embedded environment.
[0059] Through this technical solution, the present invention realizes the rapid and accurate recognition of drugs, and can adapt to the limited computing power and resolution conditions in the embedded environment, providing an efficient intelligent solution for pharmacy automatic drug dispensing robots. Brief description of the drawings
[0060] Figure 1Flowchart of a robot drug recognition method based on histogram feature recognition provided by the present invention;
[0061] Figure 2 Schematic diagram of drug images taken under different angles and lighting conditions in the solution of the present invention. Detailed implementation manners
[0062] The present invention will be further described below in conjunction with the accompanying drawings and implementation manners.
[0063] 1. Implementation environment
[0064] The implementation of the present invention mainly relies on the collaborative work of hardware and software. The typical implementation environment is as follows:
[0065] Hardware device: A low-resolution (640×480) CMOS camera for capturing drug pictures.
[0066] Software environment: Developed based on Python, using the OpenCV library (version 4.10.0) to implement image processing and feature matching.
[0067] 2. Method steps
[0068] (1) Image acquisition module
[0069] Function description: Real-time collect drug images through the camera; capture two frames of images per second and send them to the feature matching module.
[0070] Implementation method: Use the cv2.VideoCapture method of OpenCV to open the camera, set the frame rate to 2fps; intercept the image frame, perform size adjustment (normalize to 320×240) to improve processing efficiency; convert to the HSV color space for color histogram calculation.
[0071] (2) Database initialization module
[0072] Function description: Build a drug image database, extract and store the feature information of each picture.
[0073] Implementation method:
[0074] Extract ORB features: Use the cv2.ORB_create() method of OpenCV to generate an instance of the ORB algorithm; calculate keypoints and descriptors for each image.
[0075] Extract color histogram: Convert the image to the HSV color space; use cv2.calcHist to calculate the histograms of the H, S, and V channels respectively and normalize them.
[0076] Serialize the above feature data and store it as a local file.
[0077] (3) Feature matching module
[0078] Function description: Compare the image features collected in real time with the features stored in the database to determine the matching result.
[0079] Implementation method:
[0080] ORB feature matching: Extract ORB features from the real-time image; Use cv2.BFMatcher to implement brute-force matching and calculate the similarity of descriptors based on the Hamming distance.
[0081] Color histogram matching: Use the Bhattacharyya Distance to calculate the similarity between the real-time image and the database image histograms; Convert the distance value to a matching score, ranging from 0 to 1.
[0082] Fuse the matching results: Use a weighted formula to synthesize the matching scores of ORB and color histograms.
[0083] (4) Matching result judgment module
[0084] Function description: Determine the drug category corresponding to the real-time image based on the comprehensive matching score.
[0085] Implementation method: If the comprehensive matching score is lower than the set threshold (such as 0.5), output "matching failed"; If the matching is successful, return the drug name and display it on the terminal screen.
[0086] 4. Effects and performance
[0087] Recognition effect: When the camera resolution is relatively low (640×480), the accuracy rate reaches 83%.
[0088] Operating efficiency: The average matching time per frame does not exceed 500 milliseconds.
[0089] System robustness: It has good anti-interference ability against light changes and angle changes.
[0090] Through the above implementation methods, the present invention can achieve efficient and accurate drug recognition in an embedded environment with limited computing power, ensuring the safety and reliability of the automatic drug dispensing robot in actual scenarios.
[0091] The robot drug recognition method based on histogram feature recognition provided by the present invention has the following technical effects:
[0092] The present invention realizes the efficient and accurate recognition of drugs under different lighting conditions and shooting angles through an identification method based on the fusion of ORB features and color histograms. The following is the specific performance of this technical solution in actual tests:
[0093] 1. Test platform
[0094] Hardware platform: The processor uses AMD Ryzen 77840HS.
[0095] Camera resolution: 640×480.
[0096] Test environment: A complex scene including different lighting intensities, shooting angles, and occlusion degrees.
[0097] 2. Performance test results
[0098] Test samples: Input 32 drug images taken under different angles and lighting conditions, Figure 2 Two of the drug images are shown below.
[0099] Average matching time: 0.0166 seconds per image, with extremely fast processing speed, fully meeting the real-time requirements of embedded devices.
[0100] Matching accuracy: 100.00%, and all test images are correctly matched to the corresponding drug categories in the database.
[0101] Output results:
[0102] All sample images are correctly predicted for their categories;
[0103] The matching time is distributed between 0.011 seconds and 0.021 seconds, showing good stability.
[0104] 3. Comparison and analysis
[0105] Compared with traditional deep learning models, the present invention shows significant advantages in the following aspects:
[0106] Real-time performance: The average processing time is 0.0166 seconds, significantly better than the running time of deep learning models on embedded platforms (usually exceeding 0.1 seconds).
[0107] Low computing power dependence: It occupies extremely low resources when running on the Ryzen 77840HS platform and can be ported to embedded platforms with more limited computing power.
[0108] Robustness: It is insensitive to changes in lighting, angle, and image quality, with an accuracy of 100%, suitable for reliable recognition in the complex environment of pharmacy robots.
[0109] 4. Actual application effect
[0110] Through the technical solution of the present invention, the pharmacy automatic drug dispensing robot can quickly identify the grabbed drugs and feedback the recognition results to the terminal in real time, ensuring the consistency between the retrieved drugs and the target drugs, and greatly improving the medication safety. At the same time, since this method has low computing power requirements and can be adapted to embedded devices, it has broad practical value. In summary, the present invention combines high efficiency, low cost and high accuracy, and perfectly solves the actual problem of automatic drug recognition.
[0111] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.
Claims
1. A robot drug recognition method based on histogram feature recognition, characterized in that: The following steps are involved: S1. Constructing a drug image database: the database stores images of several known drugs and their corresponding feature information, wherein the feature information includes ORB features and color histograms; S2. Real-time image capture and preprocessing: The image of the drug to be identified is captured in real time by a camera installed on the robot, and the real-time captured drug image is preprocessed, including denoising, grayscale conversion and color space conversion; S3, extracting RB features and color histogram: extracting ORB features of the preprocessed image and a three-channel histogram of the HSV color space; S4, perform feature matching and determine the results: match the extracted features with the features stored in the database, and determine the final matching score by weighted fusion based on the matching results, and select the drug category with the highest matching score as the recognition result; S5. Output recognition results: Output recognition results to the robot terminal for display or further processing.
2. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: The extraction of the ORB feature adopts the Oriented FAST and Rotated BRIEF algorithm, and the extraction of the color histogram includes respectively calculating the histograms of the three channels H, S and V and performing normalization processing.
3. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: In the feature matching step, the matching of ORB features uses the BFMatcher algorithm to achieve brute force matching, and the matching of color histograms calculates the similarity scores of the two images based on the Bhattacharyya distance.
4. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: The pre-processing step also includes resizing the captured image to improve processing efficiency.
5. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: The method further includes a database initialization step for constructing a drug image database and extracting and storing feature information of each image.
6. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: The method also includes the function of displaying the matching results through the robot terminal and issuing a warning prompt when the matching fails or the recognition result is abnormal.
7. The robot drug recognition method based on histogram feature recognition as claimed in claim 1, characterized in that: The camera is a low-resolution CMOS camera with a resolution of 640×480 and captures two frames of images per second.