Intelligent medicine chest management system based on multi-modal perception and adaptive learning

Abstract

The invention discloses an intelligent medicine chest management system based on multi-modal perception and adaptive learning, which obtains medicine visual information and millimeter wave feature codes through a rotary scanning platform, analyzes taking rules in combination with an edge side large model, and dynamically optimizes a reminding strategy based on user behavior learning. The system innovatively comprises a non-standard medicine multi-mode fusion recognition technology, a context-aware self-adaptive reminding engine and a hardware-level privacy protection architecture, can effectively improve family medicine taking compliance, obviously reduces the medicine taking error rate of a nursing home, and has great social value.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical and health Internet of Things, and specifically relates to an intelligent medicine box system integrating visual recognition, millimeter-wave radar and edge computing, which is used for drug filing, taking behavior learning and personalized voice reminder. Background Art

[0002] Existing intelligent medicine boxes have the following defects: weak drug recognition ability: relying on preset barcodes or RFID tags, unable to handle unlabeled drugs (such as dispensed tablets, traditional Chinese medicine powders); rigid reminder: only supporting fixed-time reminders and unable to adapt to individual taking habits (such as users often delaying taking medicine by 30 minutes); data security risk: uploading health data to the cloud is prone to privacy leakage.

[0003] Existing technologies (such as the patent CN119551341A for the medicine storage method and device of a medicine storage device) scan the text on the medicine box through a camera, but do not solve the problems of non-standard packaging recognition and behavior learning. Summary of the Invention

[0004] The present invention proposes an intelligent medicine box management system based on multi-modal perception and adaptive learning. The medicine box obtains drug visual information and millimeter-wave feature codes through a rotating scanning platform, combines a large model on the edge side to analyze the taking rules, and dynamically optimizes the reminder strategy based on user behavior learning. The system architecture of the present invention is as Figure 1 shown, including core modules such as a multi-modal filing unit, an adaptive learning engine, and a privacy protection architecture.

[0005] The multi-modal filing unit is composed of a rotating scanning platform and an edge processing chip.

[0006] After the user places the drug on the rotating scanning platform and manually rotates it 360°, the following scans are triggered: AI vision module (4K camera + ring-shaped fill light): takes images of all sides of the drug, and extracts taking instructions (dose, frequency, taboos) through OCR and a large model (GPT-4 medical version); millimeter-wave radar (60GHz band): obtains a three-dimensional point cloud model of the drug and generates a unique feature code (size + material reflection fingerprint).

[0007] Edge processing chip: binds the text description and the feature code to establish an encrypted drug file.

[0008] This system is equipped with an adaptive learning engine. The first week is the learning period, and the second week enters the execution period. First week learning period: records the actual taking time (through the weight sensor / infrared induction of the taking action) and constructs a personalized time pattern model. Second week execution period: dynamically adjusts the reminder time based on the learning model, and supports a progressive reminder strategy (first voice → vibration after 10 minutes → emergency contact notification after 30 minutes).

[0009] The system is designed with a privacy protection architecture: data is processed entirely at the edge (such as Rockchip RK3588 chip), and only the encrypted digest is output to the mobile APP; the drug feature code and user identity are stored separately, so even if the device is stolen, the specific patient cannot be associated.

[0010] The innovations of the present invention are reflected in three aspects: precise identification of non-standard drugs, context-aware reminders, and self-healing medicine cabinets.

[0011] Precise identification of non-standard drugs: Integrate visual semantic understanding (extracting unstructured text such as "twice a day, after breakfast and dinner") and radar physical features (able to identify even if the pill is transferred to a self-made bag); support recognition of more than 200 languages / handwritten scripts, with a misreading rate < 0.1%.

[0012] Context-aware reminders: Determine the user's status through environmental sensors (microphone, millimeter-wave presence detection): when a cough sound is detected, remind to take cough medicine in advance; automatically lower the reminder volume when getting up at night.

[0013] Self-healing medicine cabinet: When the stock of a certain drug is detected to be below the threshold (weight sensor + visual counting), automatically push a replenishment link or generate an e-prescription application; when reminding of expired drugs, link to the location navigation of the community recycling station. Description of the Drawings

[0014] Figure 1 : System structure block diagram (interaction of scanning, learning, and reminder modules).

[0015] Figure 2 : Mechanical design drawing of the rotating scanning platform (layout of torque sensor, radar / camera).

[0016] Figure 3 : Algorithm flow chart of the adaptive learning engine (LSTM time series prediction).

[0017] Figure 4 : Workflow chart of the home medicine cabinet. Detailed Implementation Modes

[0018] Example 1: Home medicine cabinet. Hardware configuration: The diameter of the scanning platform is 15 cm, with a load-bearing capacity of 5 kg, and a built-in torque sensor (to prevent damage to drugs caused by excessive rotation); the millimeter-wave radar has a resolution of 0.5 mm³ and can identify subtle deformations such as missing corners of pills; the voice module supports voiceprint recognition (only responds to preset user instructions). The work process is shown in Figure 4 , Scene example: The user puts in the sub-packaged antihypertensive medicine (without a label). After scanning, the AI recognizes the handwritten "1 pill in the morning and 1 pill in the evening", and the radar records the capsule feature code; during the learning period, it is found that the user takes the medicine 47 minutes later on average in the evening, and in the second week, the reminder time is automatically postponed to 8:47 AM / 7:47 PM.

[0019] Example 2: Centralized management of medicine cabinets in nursing homes. Hardware upgrade: Support batch scanning (simultaneously rotating and creating files for 30 boxes of drugs); add NFC wristband to identify the elderly to ensure targeted push of reminders; the administrator interface displays the heat map of the medication compliance rate of the whole hospital. Differentiated functions: When it is detected that a certain elderly person fails to take medicine on time for 3 consecutive times, it triggers a patrol by the caregiver; real-time detection of drug incompatibility taboos (such as locking the medicine cabinet when antihypertensive drugs conflict with grapefruit juice).

Claims

1. An intelligent medicine box management system based on multimodal perception and adaptive learning, characterized in that include: Rotating scanning platform, integrating AI vision module and millimeter wave radar for contactless drug feature extraction; The adaptive learning engine dynamically optimizes reminder time strategies through historical data of medication behavior; the privacy protection architecture achieves physical isolation and storage of drug feature codes and user identities.

2. The system according to claim 1, characterized in that: The AI ​​vision module calls the medical big model to parse the instructions for use and extract the dosage, frequency, and contraindications triplet; the feature code generated by the millimeter wave radar includes a combination of surface roughness (Ra), volume (V), and dielectric constant (ε).

3. The system according to claim 1, characterized in that: The adaptive learning engine adopts a time series prediction algorithm (LSTM network), and its input includes context features of ambient light and sound events; it supports progressive reminders: a four-level response of voice → device vibration → mobile phone notification → emergency contact alert.

4. The system according to claim 1, characterized in that: The privacy protection architecture controls the data export through a physical switch and runs completely offline when disconnected; the drug files are encrypted and stored separately in the edge chip and the pluggable SD card, and double authentication is required for complete decryption.

Images