Intelligent medicine box medicine verification method and system based on visual assistance

Abstract

The application relates to the field of medical care, and discloses a visual aid-based intelligent medicine box medicine taking verification method and system, which comprises the following steps: calling medicine taking instructions containing visual attributes of medicines, acquiring an initial weight of a medicine storage compartment and a coordinate reference of a medicine box relative to a medicine taking area; when the weight reduction is greater than a preset threshold, driving a camera to capture a hand movement path, calculating a path curvature, a height change rate and a residence time in the medicine taking area; when the path curvature is within a preset envelope range, the height change rate points to the medicine taking area and the residence time reaches a threshold, determining that a medicine taking action is valid and outputting a signal; the application utilizes a spatial trajectory and multi-dimensional feature cooperative judgment mechanism to effectively distinguish a medicine taking action from a real medicine taking behavior, accurately filters limb motion interference, eliminates a medicine taking and non-taking monitoring blind area, and ensures that a medicine taking record is real and objective.

Description

Technical Field

[0001] This invention relates to a vision-assisted intelligent pillbox medication verification method and system, belonging to the field of medical monitoring technology. Background Technology

[0002] In the field of home-based elderly care monitoring, medication containers specifically designed for medical purposes are essential tools for ensuring medication adherence among the elderly. These containers typically employ a compartmentalized storage structure and are equipped with timed reminder devices and compartment opening / closing monitoring modules to guide users to remove medications from the corresponding compartments at preset times.

[0003] However, existing technologies for monitoring the medication dispensing process generally suffer from a logical disconnect between the delivery status and the actual administration behavior. Current monitoring logic primarily relies on the opening and closing signals of the medication storage compartment to characterize the completion of the medication dispensing process. In reality, the medication leaving the container does not equate to its ingestion by the body, and cases of users forgetting to take the medication, discarding it, or taking it incorrectly are frequent. Attempting to cover these monitoring blind spots by introducing wide-area video surveillance faces the dual constraints of excessive data processing load and intrusion into personal privacy. Even by increasing the sensitivity of physical sensors, it is difficult to achieve automatic comparison of medication properties and monitoring of medication administration. The lack of synchronous verification of the dispensing path results in an open-loop process for medication supervision after the container delivery point. Existing technologies have several shortcomings: 1. The lack of a medication delivery verification mechanism prevents automatic verification of the medication's color, shape, and surface markings at the moment of removal, making it difficult to avoid the risk of incorrect dispensing or removal; 2. The incomplete logic for verifying medication use behavior limits monitoring to the static result of medication removal from the container, lacking dynamic tracking of the hand's movement path from the dispensing point to the mouth; 3. Insufficient mechanical intervention capabilities in abnormal situations, failing to immediately activate the dispensing mechanism into a physically locked state when duplicate dispensing or medication mismatch is detected.

[0004] Therefore, the technical problem to be solved by this invention is how to provide a vision-assisted smart pillbox medication verification method to solve the problem of the lack of automated property verification in the drug delivery process and the difficulty in determining the authenticity of medication behavior due to the lack of dynamic path tracking, which leads to the inability to stop abnormal medication in time. Summary of the Invention

[0005] To address the problems mentioned in the background art, the technical solution of the present invention is as follows: A vision-assisted intelligent pillbox medication verification method, comprising the following steps: Step 101: Retrieve the preset medication instructions from the memory. The medication instructions include the visual attribute features of the target drug and the time of administration. Also, obtain the initial weight of the drug corresponding to the storage compartment inside the medicine box and the spatial coordinate reference of the medicine box body relative to the preset medication target area. Step 102: When the monitoring system clock reaches the medication time, the weight signal of each medicine storage compartment is acquired in real time. When the decrease in weight signal relative to the initial weight is greater than the preset weight change threshold, the camera is driven to capture a continuous image sequence of the hand moving from the medicine box dispensing port to the preset medication target area. Step 103: Based on the pixel displacement mapping of hand joints in the continuous image sequence, the motion path in three-dimensional space is mapped, and the three-dimensional coordinates of the hand are extracted using the joint detection algorithm. The path curvature in the horizontal reference plane and the rate of change of height in the vertical gravitational direction over time are calculated. Step 104: The residence time of the hand holding the drug in the space corresponding to the preset drug administration target area is monitored in real time. Step 105: When the path curvature is within the preset medication movement trajectory envelope and there is no obvious retreat trajectory away from the face, the height change rate over time shows a unidirectional trend pointing towards the preset medication target area and the dwell time reaches the preset duration threshold, and the time of spatial displacement of the hand does not exceed the preset time threshold, it is determined that the hand has completed the medication action. Step 106: Extract the drug image features of the object held by the hand from the continuous image sequence. When the drug image features match the visual attribute features in the medication instruction and the medication action is determined to be valid, output a confirmation signal containing the device identification code, medication timestamp, and medication success indicator.

[0006] Preferably, step 106 further includes the following sub-steps: Step 1061, when it is determined that the hand is lingering in the preset medication target area, the color information, edge contour and surface texture of the hand holding the drug are acquired through the camera, and a lightweight convolutional neural network containing 8 bneck modules is used to extract the 128-dimensional feature vector corresponding to the multi-dimensional features; Step 1062, the cosine similarity between the 128-dimensional feature vector and the standard drug template pre-stored in the medication instruction is calculated using the Siamese network structure, and when the highest similarity is greater than or equal to the preset recognition threshold of 0.85, it is confirmed that the drug image features match the medication instruction, and the association between the drug image features and the medication success identifier is established.

[0007] Preferably, after determining that the hand has completed the medication taking action, the method further includes: continuously monitoring the return path of the hand after completing the medication taking action through a camera; calculating the trajectory overlap between the return path and the movement path; determining that the trajectory overlap is lower than a preset interference threshold and that there is no infrared trigger signal of the hand approaching the dispensing port within a 2-second cooling time; confirming that non-medication-related limb shaking interference has been eliminated; and locking the current verification result.

[0008] Preferably, after outputting the confirmation signal, the method further includes: encapsulating the confirmation signal into a custom format data frame using a star-shaped local LoRa self-organizing network protocol with a communication frequency of 470MHz and a transmission rate of 5.4kbps. The custom format data frame includes, in sequence, a 2-byte 0xAA55 frame header, a 2-byte device ID, a 1-byte alarm type, a 4-byte Unix timestamp, and a 1-byte CRC checksum, and sending it to the linkage broadcasting device outside the medicine box; driving the linkage broadcasting device to parse the data frame, outputting a voice prompt corresponding to the confirmation signal, and updating the medication record form stored locally on the linkage broadcasting device for the day.

[0009] Preferably, after determining that the reduction amount is greater than the weight change threshold, the method further includes: comparing the timestamp of the current reduction amount with the historical medication records stored in the medication instruction; if the time interval between the timestamp and the most recent successful medication record is less than a preset safety period threshold, it is determined that there is a risk of overdose, and the servo drive mechanism corresponding to the medication storage compartment is put into a mechanical lock state, and the opening path of the medication storage compartment is cut off.

[0010] Preferably, after the servo drive mechanism corresponding to the drug storage compartment is in a mechanically locked state, the method further includes: pushing a duplicate medication alarm message to the associated mobile terminal and entering an authorization waiting state; after receiving the remote authorization command fed back by the mobile terminal through the network, driving the servo drive mechanism to move the locking pin and release the mechanically locked state of the drug storage compartment.

[0011] Preferably, before monitoring the real-time weight of each medicine compartment in step 102, the method further includes: monitoring the connection status between the communication module integrated in the medicine box and the external server; when the connection status is determined to be abnormally interrupted, switching to offline monitoring mode; in offline monitoring mode, using locally stored verification algorithms and criteria to continuously complete medication verification, and temporarily storing the generated verification results in local non-volatile cache space.

[0012] Preferably, during the operation of the offline monitoring mode, the method further includes: responding to the communication module detecting that the connection status has returned to normal, asynchronously synchronizing the verification results stored in the non-volatile cache space to the cloud database; and performing incremental updates and calibrations on the locally stored medication instructions according to the synchronization feedback instructions issued by the cloud database.

[0013] Preferably, after step 106, the method further includes: when the number of times the decrease in amount exceeds the weight change threshold but the medication action determination is invalid exceeds a preset abnormal frequency threshold, the current state is confirmed as an abnormal medication state; the buzzer integrated into the medicine box body is driven to output a high-frequency sound and light prompt, and the linkage broadcasting device is triggered to output an emergency help call command.

[0014] A vision-assisted intelligent pillbox medication verification system, characterized in that it includes: The storage unit is used to store medication instructions, the initial weight of each medicine compartment, the spatial coordinate reference of the medicine box body relative to the preset medication target area, and the judgment criteria. The weight monitoring unit is used to acquire the real-time weight signal of each medicine storage compartment, calculate the decrease in the real-time weight signal relative to the initial weight, and send a trigger signal when the decrease is greater than a preset weight change threshold. The image acquisition unit includes a camera for capturing a continuous sequence of images of a hand moving from the dispensing port of a medicine box to a preset medication target area in response to a trigger signal. The analysis and judgment unit is used to map the motion path in three-dimensional space based on the pixel displacement of the hand joints in a continuous image sequence, calculate the path curvature in the horizontal reference plane and the rate of change of the height in the vertical direction of gravity over time, and determine whether the motion path conforms to the drug administration trajectory envelope; the analysis and judgment unit is also used to monitor the duration of hand stay in the preset drug administration target area. The recognition output unit is used to extract drug image features of objects held by hands in a continuous image sequence, and when the drug image features match the medication instructions and the retention time reaches a preset time threshold, it outputs a confirmation signal containing a device identification code, a medication timestamp, and a medication success indicator.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. By integrating visual feature comparison with the physical linkage mechanism of the dispensing mechanism in the medicine box delivery process, this invention establishes a delivery approval process driven by drug feature recognition. This process uses the collected images of the drug's appearance to extract color, shape, and surface marking features, and compares the extracted feature vectors with a standard feature library in a preset medication plan in real time. When the similarity is lower than 0.85, the system directly controls the servo motor to lock the dispensing mechanism and trigger an alarm. This mechanism moves the medication safety control point from the storage stage to the moment of delivery, directly intervening in the mechanical state through visual verification results, avoiding the risk of accidental ingestion due to incorrect drug packaging or dispensing, and significantly improving the accuracy of medicine container delivery in non-medical environments.

[0016] 2. This invention constructs a closed-loop verification system for medication-taking behavior by logically coupling weight sensing signals and hand spatial motion trajectory signals. This system requires that while monitoring the reduction in the weight of the medication at the dispensing port is within a preset deviation range of 15%, it also needs to identify the complete path of the user's key hand moving from the dispensing port area to the mouth area with a dwell time of not less than 0.5 seconds. This collaborative judgment mechanism based on multi-dimensional physical features effectively distinguishes between simple medication-taking actions and actual medication-taking behavior, eliminates monitoring blind spots caused by taking medication but not eating it, and ensures the authenticity and objectivity of medication log records.

[0017] 3. Relying on the spatial and temporal constraints of medication-taking behavior, this invention has the ability to accurately filter out daily non-medication-taking actions. By real-time monitoring of the curvature radius and vertical height change trend of the hand movement path, and combined with constraints such as the movement time not exceeding 3 seconds and no repeated approaching actions within 2 seconds after medication, the system can accurately identify and eliminate interfering behaviors such as scratching and wiping the mouth. This filtering logic, based on the behavioral characteristics of specific medical scenarios, reduces the false alarm rate of the system, making the monitoring feedback of the medicine container more in line with the actual engineering application needs of home-based elderly care. Attached Figure Description

[0018] Fig. 1 This is a flowchart illustrating the drug-taking behavior verification process based on the multi-dimensional feature collaborative determination method of this invention. Fig. 2 This is a system architecture diagram of the present invention that integrates mechanical blocking and multi-terminal linkage.

[0019] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] A vision-assisted smart pillbox medication verification method includes the following steps: Step 101: Retrieve the preset medication instructions from the memory. The medication instructions include the visual attribute features of the target drug and the time of administration. Also, obtain the initial weight of the drug corresponding to the storage compartment inside the medicine box and the spatial coordinate reference of the medicine box body relative to the preset medication target area. Step 102: When the monitoring system clock reaches the medication time, the weight signal of each medicine storage compartment is acquired in real time. When the decrease in weight signal relative to the initial weight is greater than the preset weight change threshold, the camera is driven to capture a continuous image sequence of the hand moving from the medicine box dispensing port to the preset medication target area. Step 103: Based on the pixel displacement mapping of hand joints in the continuous image sequence, the motion path in three-dimensional space is mapped, and the three-dimensional coordinates of the hand are extracted using the joint detection algorithm. The path curvature in the horizontal reference plane and the rate of change of height in the vertical gravitational direction over time are calculated. Step 104: The residence time of the hand holding the drug in the space corresponding to the preset drug administration target area is monitored in real time. Step 105: When the path curvature is within the preset medication movement trajectory envelope and there is no obvious retreat trajectory away from the face, the height change rate over time shows a unidirectional trend pointing towards the preset medication target area and the dwell time reaches the preset duration threshold, and the time of spatial displacement of the hand does not exceed the preset time threshold, it is determined that the hand has completed the medication action. Step 106: Extract the drug image features of the object held by the hand from the continuous image sequence. When the drug image features match the visual attribute features in the medication instruction and the medication action is determined to be valid, output a confirmation signal containing the device identification code, medication timestamp, and medication success indicator.

[0022] Preferably, step 106 further includes the following sub-steps: Step 1061, when it is determined that the hand is lingering in the preset medication target area, the color information, edge contour and surface texture of the hand holding the drug are acquired through the camera, and a lightweight convolutional neural network containing 8 bneck modules is used to extract the 128-dimensional feature vector corresponding to the multi-dimensional features; Step 1062, the cosine similarity between the 128-dimensional feature vector and the standard drug template pre-stored in the medication instruction is calculated using the Siamese network structure, and when the highest similarity is greater than or equal to the preset recognition threshold of 0.85, it is confirmed that the drug image features match the medication instruction, and the association between the drug image features and the medication success identifier is established.

[0023] Preferably, after determining that the hand has completed the medication taking action, the method further includes: continuously monitoring the return path of the hand after completing the medication taking action through a camera; calculating the trajectory overlap between the return path and the movement path; determining that the trajectory overlap is lower than a preset interference threshold and that there is no infrared trigger signal of the hand approaching the dispensing port within a 2-second cooling time; confirming that non-medication-related limb shaking interference has been eliminated; and locking the current verification result.

[0024] Preferably, after outputting the confirmation signal, the method further includes: encapsulating the confirmation signal into a custom format data frame using a star-shaped local LoRa self-organizing network protocol with a communication frequency of 470MHz and a transmission rate of 5.4kbps. The custom format data frame includes, in sequence, a 2-byte 0xAA55 frame header, a 2-byte device ID, a 1-byte alarm type, a 4-byte Unix timestamp, and a 1-byte CRC checksum, and sending it to the linkage broadcasting device outside the medicine box; driving the linkage broadcasting device to parse the data frame, outputting a voice prompt corresponding to the confirmation signal, and updating the medication record form stored locally on the linkage broadcasting device for the day.

[0025] Preferably, after determining that the reduction amount is greater than the weight change threshold, the method further includes: comparing the timestamp of the current reduction amount with the historical medication records stored in the medication instruction; if the time interval between the timestamp and the most recent successful medication record is less than a preset safety period threshold, it is determined that there is a risk of overdose, and the servo drive mechanism corresponding to the medication storage compartment is put into a mechanical lock state, and the opening path of the medication storage compartment is cut off.

[0026] Preferably, after the servo drive mechanism corresponding to the drug storage compartment is in a mechanically locked state, the method further includes: pushing a duplicate medication alarm message to the associated mobile terminal and entering an authorization waiting state; after receiving the remote authorization command fed back by the mobile terminal through the network, driving the servo drive mechanism to move the locking pin and release the mechanically locked state of the drug storage compartment.

[0027] Preferably, before monitoring the real-time weight of each medicine compartment in step 102, the method further includes: monitoring the connection status between the communication module integrated in the medicine box and the external server; when the connection status is determined to be abnormally interrupted, switching to offline monitoring mode; in offline monitoring mode, using locally stored verification algorithms and criteria to continuously complete medication verification, and temporarily storing the generated verification results in local non-volatile cache space.

[0028] Preferably, during the operation of the offline monitoring mode, the method further includes: responding to the communication module detecting that the connection status has returned to normal, asynchronously synchronizing the verification results stored in the non-volatile cache space to the cloud database; and performing incremental updates and calibrations on the locally stored medication instructions according to the synchronization feedback instructions issued by the cloud database.

[0029] Preferably, after step 106, the method further includes: when the number of times the decrease in amount exceeds the weight change threshold but the medication action determination is invalid exceeds a preset abnormal frequency threshold, the current state is confirmed as an abnormal medication state; the buzzer integrated into the medicine box body is driven to output a high-frequency sound and light prompt, and the linkage broadcasting device is triggered to output an emergency help call command.

[0030] A vision-assisted intelligent pillbox medication verification system, characterized in that it includes: The storage unit is used to store medication instructions, the initial weight of each medicine compartment, the spatial coordinate reference of the medicine box body relative to the preset medication target area, and the judgment criteria. The weight monitoring unit is used to acquire the real-time weight signal of each medicine storage compartment, calculate the decrease in the real-time weight signal relative to the initial weight, and send a trigger signal when the decrease is greater than a preset weight change threshold. The image acquisition unit includes a camera for capturing a continuous sequence of images of a hand moving from the dispensing port of a medicine box to a preset medication target area in response to a trigger signal. The analysis and judgment unit is used to map the motion path in three-dimensional space based on the pixel displacement of the hand joints in a continuous image sequence, calculate the path curvature in the horizontal reference plane and the rate of change of the height in the vertical direction of gravity over time, and determine whether the motion path conforms to the drug administration trajectory envelope; the analysis and judgment unit is also used to monitor the duration of hand stay in the preset drug administration target area. The recognition output unit is used to extract drug image features of objects held by hands in a continuous image sequence, and when the drug image features match the medication instructions and the retention time reaches a preset time threshold, it outputs a confirmation signal containing a device identification code, a medication timestamp, and a medication success indicator.

[0031] Example 1: This example combines Figs. 1-2 This document describes a vision-assisted smart pillbox medication verification method and system. Fig. 1 As shown, step 101 retrieves a pre-set medication instruction containing the visual attributes of the target drug and the medication time from the memory, obtains the initial weight of the drug corresponding to the storage compartment inside the medicine box, and the spatial coordinate reference of the medicine box body relative to the pre-set medication target area. In step 102, the system clock is monitored to reach the medication time and the weight signal of each storage compartment is acquired in real time. When the decrease in weight signal relative to the initial weight is greater than the pre-set weight change threshold, the camera is driven to capture a continuous image sequence of the hand moving from the medicine box dispensing port to the pre-set medication target area. Then, in step 103, the motion path in three-dimensional space is mapped based on the pixel displacement of the hand joints in the continuous image sequence, and the motion path in the horizontal reference plane is calculated. The system monitors the path curvature and the rate of change of height in the vertical direction of gravity over time. In step 104, it monitors the duration of the hand holding the drug within the spatial range corresponding to the preset drug administration target area. In step 105, it determines that the hand has completed the drug administration action when the path curvature is within the preset drug administration trajectory envelope, the rate of change of height over time shows a unidirectional trend pointing towards the target area, and the duration of the stay reaches the preset duration threshold. Finally, step 106 is executed to extract the drug image features of the hand holding the object from the continuous image sequence. When the drug image features match the visual attribute features and the drug administration action is determined to be valid, a confirmation signal containing the device identification code, drug administration timestamp, and drug administration success indicator is output.

[0032] like Fig. 2As shown, the external server has a cloud database to receive asynchronous synchronization signals from the medicine box and send back synchronization feedback instructions. The mobile terminal is used to receive repeated medication alarm messages and send remote authorization instructions to the medicine box. The linkage broadcasting device communicates with the medicine box through the local LoRa self-organizing network protocol. The medicine box integrates a camera, memory, communication module, buzzer, medicine storage compartment and servo drive mechanism. The continuous image sequence captured by the camera is transmitted to the memory for analysis. The communication module is responsible for maintaining operation and processing various data transmission tasks in offline monitoring mode. When the memory detects an abnormal medication status, it triggers the buzzer to output a prompt. After receiving authorization or control instructions, the servo drive mechanism drives the locking pin to move, thereby achieving precise control of the mechanical status of the medicine storage compartment.

[0033] Example 2: In a specific application scenario of this invention, the core hardware of the system uses an STM32F407 as the main control chip. It interacts with the cloud server and the user-side mini-program via an integrated 4G module. When a medication reminder is triggered, the main control chip drives a servo motor to push the medication to the dispensing port. A miniature camera above captures a 224x224 pixel image and inputs it into a preset MobileNetV3-small network model. This model consists of eight bridge modules composed of depthwise separable convolutions, attention mechanisms, and residual connections. The number of network parameters is controlled within 3.8M. The main control chip performs real-time inference in less than or equal to 150ms. The extracted 128-dimensional feature vector is fed into the Siamese network logic and compared with the locally stored standard drug template image using cosine similarity. If the highest similarity obtained is above 0.85, it is determined that the characteristics of the currently dispensed medication meet the medication instructions. Simultaneously, the system starts the MediaPipe. The Hand algorithm locks the spatial coordinates of 21 joints on the hand in real time, and uses a spherical space with a radius of 10 cm at the center of the medicine dispensing port as the medicine dispensing area A, and a spherical space with a radius of 5 cm at the center of the key point of the user's mouth as the target area B.

[0034] When determining the authenticity of medication administration, the system requires that the single displacement process of the key hand points from area A to area B must be completed within 3 seconds, and the spatial dwell time of the hand in area B must reach 0.5 seconds. Through dynamic monitoring of the movement trajectory, the system further analyzes the radius of curvature of the movement path to ensure that the hand movement trajectory is smooth and there is no obvious backward movement away from the face. Combined with the cooling period restriction of no secondary movement close to the mouth within 2 seconds after the medication administration is completed, the system can accurately eliminate interference behaviors such as wiping the mouth or scratching. In terms of communication, the medicine box body uses a LoRa star topology structure to network with the linkage broadcasting equipment, and the communication frequency is set at 470 MHz. When duplicate medication or medication retrieval errors are detected, the system encapsulates a custom data frame containing a 0xAA55 start frame header, device identification code, alarm type, timestamp, and checksum. The system then links the broadcasting device to provide a stepped voice prompt at a volume ranging from 60 to 85 decibels based on the alarm type. In the event of a network outage, the system automatically switches to local monitoring mode and temporarily stores the verification results. Once the network is restored, the results are asynchronously synchronized to the cloud platform to ensure the integrity of the medication supervision process.

[0035] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A vision-assisted smart pillbox medication verification method, characterized in that, Includes the following steps: Step 101: Retrieve the preset medication instructions from the memory. The medication instructions include the visual attribute features of the target drug and the time of administration. Also, obtain the initial weight of the drug corresponding to the storage compartment inside the medicine box and the spatial coordinate reference of the medicine box body relative to the preset medication target area. Step 102: When the monitoring system clock reaches the medication time, the weight signal of each medicine storage compartment is acquired in real time. When the decrease in weight signal relative to the initial weight is greater than the preset weight change threshold, the camera is driven to capture a continuous image sequence of the hand moving from the medicine box dispensing port to the preset medication target area. Step 103: Based on the pixel displacement mapping of hand joints in the continuous image sequence, the motion path in three-dimensional space is mapped, the three-dimensional coordinates of the hand are extracted using the joint detection algorithm, and the path curvature in the horizontal reference plane and the rate of change of height in the vertical gravitational direction over time are calculated. Step 104: Monitor in real time the duration of time the hand holds the medication within the space corresponding to the preset medication target area; Step 105: When the path curvature is within the preset medication movement trajectory envelope and there is no obvious retreat trajectory away from the face, the height change rate over time shows a unidirectional trend pointing towards the preset medication target area and the dwell time reaches the preset duration threshold, and the time of spatial displacement of the hand does not exceed the preset time threshold, it is determined that the hand has completed the medication action. Step 106: Extract the drug image features of the object held by the hand from the continuous image sequence. When the drug image features match the visual attribute features in the medication instruction and the medication action is determined to be valid, output a confirmation signal containing the device identification code, medication timestamp, and medication success indicator.

2. The method for verifying medication use in a vision-assisted smart pillbox according to claim 1, characterized in that, Step 106 further includes the following sub-steps: Step 1061, when it is determined that the hand is stuck in the preset drug administration target area, the color information, edge contour and surface texture of the hand holding the drug are obtained through the camera, and a lightweight convolutional neural network containing 8 bneck modules is used to extract the 128-dimensional feature vector corresponding to the multi-dimensional features. Step 1062: The Siamese network structure is used to calculate the cosine similarity between the 128-dimensional feature vector and the standard drug template pre-stored in the medication instruction. When the highest similarity is greater than or equal to the preset recognition threshold of 0.85, the drug image features are confirmed to match the medication instruction, and the association between the drug image features and the medication success identifier is established.

3. The method for verifying medication use in a vision-assisted smart pillbox according to claim 2, characterized in that, After determining that the hand has completed the medication taking action, the process also includes: continuously monitoring the return path of the hand after completing the medication taking action through a camera; calculating the trajectory overlap between the return path and the movement path; determining that the trajectory overlap is lower than a preset interference threshold and there is no infrared trigger signal of the hand approaching the dispensing port within a 2-second cooling time; confirming that non-medication-related limb shaking interference has been eliminated; and locking the current verification result.

4. The method for verifying medication use in a vision-assisted smart pillbox according to claim 1, characterized in that, After outputting the confirmation signal, the process also includes: encapsulating the confirmation signal into a custom-formatted data frame using a star-shaped local LoRa self-organizing network protocol with a communication frequency of 470MHz and a transmission rate of 5.4kbps. This custom-formatted data frame includes, in sequence, a 2-byte 0xAA55 frame header, a 2-byte device ID, a 1-byte alarm type, a 4-byte Unix timestamp, and a 1-byte CRC checksum, and then sending it to the external linkage broadcasting device. The linkage broadcasting device is then driven to parse the data frame, output a voice prompt corresponding to the confirmation signal, and simultaneously update the daily medication record form stored locally on the linkage broadcasting device.

5. The method for verifying medication use in a vision-assisted smart pillbox according to claim 1, characterized in that, After determining that the reduction amount is greater than the weight change threshold, the process also includes: comparing the timestamp of the current reduction amount with the historical medication records stored in the medication instructions; if the time interval between the timestamp and the most recent successful medication record is less than the preset safety period threshold, it is determined that there is a risk of overdose, and the servo drive mechanism corresponding to the medication compartment is put into a mechanical lock state, and the opening path of the medication compartment is cut off.

6. The method for verifying medication use in a vision-assisted smart pillbox according to claim 5, characterized in that, After the servo drive mechanism corresponding to the drug storage compartment is in a mechanically locked state, the process also includes: pushing a duplicate medication alarm message to the associated mobile terminal and entering an authorization waiting state; after receiving the remote authorization command fed back by the mobile terminal through the network, driving the servo drive mechanism to move the locking pin and release the mechanically locked state of the drug storage compartment.

7. The method for verifying medication use in a vision-assisted smart pillbox according to claim 1, characterized in that, Before monitoring the real-time weight of each medicine compartment in step 102, the following steps are also included: monitoring the connection status between the communication module integrated in the medicine box and the external server, and switching to offline monitoring mode when the connection status is abnormally interrupted; in offline monitoring mode, the verification algorithm and criteria stored locally are used to continuously complete the medication verification, and the generated verification results are temporarily stored in the local non-volatile cache space.

8. The method for verifying medication use in a vision-assisted smart pillbox according to claim 7, characterized in that, During offline monitoring mode operation, the following steps are also included: in response to the communication module detecting that the connection status has returned to normal, the verification results stored in the non-volatile cache space are asynchronously synchronized to the cloud database; and based on the synchronization feedback instructions issued by the cloud database, the medication instructions stored locally are incrementally updated and calibrated.

9. A method for verifying medication use in a vision-assisted smart pillbox according to claim 1, characterized in that, After step 106, the process also includes: when the number of times the decrease in amount exceeds the weight change threshold but the medication action determination fails exceeds the preset abnormal frequency threshold, the current state is confirmed as an abnormal medication state; the buzzer integrated into the medicine box body is driven to output a high-frequency sound and light prompt, and the linkage broadcasting device is triggered to output an emergency help call command.

10. A vision-assisted intelligent pillbox medication verification system, used to implement the vision-assisted intelligent pillbox medication verification method as described in the claim, characterized in that, include: The storage unit is used to store medication instructions, the initial weight of each medicine compartment, the spatial coordinate reference of the medicine box body relative to the preset medication target area, and the judgment criteria. The weight monitoring unit is used to acquire the real-time weight signal of each medicine storage compartment, calculate the decrease in the real-time weight signal relative to the initial weight, and send a trigger signal when the decrease is greater than a preset weight change threshold. The image acquisition unit includes a camera for capturing a continuous sequence of images of a hand moving from the dispensing port of a medicine box to a preset medication target area in response to a trigger signal. The analysis and judgment unit is used to map the motion path in three-dimensional space based on the pixel displacement of the hand joints in a continuous image sequence, calculate the path curvature in the horizontal reference plane and the rate of change of the height in the vertical direction of gravity over time, and determine whether the motion path conforms to the drug administration trajectory envelope; the analysis and judgment unit is also used to monitor the duration of hand stay in the preset drug administration target area. The recognition output unit is used to extract drug image features of objects held by hands in a continuous image sequence, and when the drug image features match the medication instructions and the retention time reaches a preset time threshold, it outputs a confirmation signal containing a device identification code, a medication timestamp, and a medication success indicator.

Images