Computer vision-based automatic medicine feeding device and method

Abstract

The application provides an automatic medicine feeding device and method based on computer vision, wherein the device comprises: an image processing unit, which is used for collecting a dose image of a medicine bottle and identifying a real-time dose in the dose image; a judgment unit, which is used for judging whether to trigger an automatic medicine feeding process based on a collection time stamp of the dose image and the real-time dose; and a driving unit, which is used for generating an automatic medicine feeding signal and driving a mechanical arm through the automatic medicine feeding signal so that the mechanical arm feeds medicine to a specified target if the automatic medicine feeding signal is triggered. The technical scheme provided by the application can automatically feed medicine to a target object, thereby saving manpower and material resources.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an automatic drug feeding device and method based on computer vision. Background Technology

[0002] Currently, in some aspects of animal husbandry, professional caregivers are often needed to administer medication to sick animals on a regular schedule. However, in practice, employing professional caregivers consumes significant human resources, and as the number of animals increases, the caregivers' energy becomes insufficient, resulting in animals not receiving adequate care. Summary of the Invention

[0003] This invention provides an automatic drug feeding device and method based on computer vision, which can automatically feed drugs to target objects, thereby saving manpower and material resources.

[0004] In view of this, the present invention provides an automatic drug feeding device based on computer vision, the device comprising:

[0005] An image processing unit is used to acquire a dose image of the medicine bottle and identify the real-time dose in the dose image;

[0006] The judgment unit is used to determine whether to trigger the automatic drug delivery process based on the acquisition timestamp of the dose image and the real-time dose;

[0007] A drive unit is used to generate an automatic drug feeding signal when triggered, and to drive the robotic arm through the automatic drug feeding signal so that the robotic arm feeds the drug to the designated target.

[0008] In one embodiment, the image processing unit is specifically configured to: identify a detection frame of the medicine bottle from the dosage image and crop the dosage image based on the detection frame; identify boundary lines from the cropped image, the boundary lines being used to characterize the outline of the medicine bottle and the liquid plane of the medicine; identify the liquid plane of the medicine from the boundary lines based on the positional relationship between the boundary lines; read the position of the medicine corresponding to the liquid plane, and determine the real-time dose based on the position of the medicine.

[0009] In one embodiment, the image processing unit is further configured to identify the maximum circumscribed boundary from the boundary line, remove the maximum circumscribed boundary from the boundary line, and use the remaining boundary line as the liquid plane of the agent.

[0010] In one embodiment, the judgment unit is specifically used to: determine the drug feeding cycle of the target object and the initial drug dose in the drug bottle; determine the drug consumption based on the drug feeding cycle and the collection timestamp; determine the theoretical remaining drug amount based on the initial drug dose and the drug consumption; and compare the theoretical remaining drug amount with the real-time dose to determine whether to trigger the automatic drug feeding process.

[0011] In one embodiment, the judgment unit is specifically used to calculate the difference between the theoretical remaining amount of medicine and the real-time dose; if the difference is less than or equal to a specified threshold, it is determined that the automatic medication process is not triggered; if the difference is greater than the specified threshold, it is determined that the automatic medication process is triggered.

[0012] In one embodiment, the driving unit is specifically used to identify the identity of the target object and acquire drug feeding driving data related to the identity, wherein the drug feeding driving data at least characterizes the liquid injection rate during the drug feeding process; and generate an automatic drug feeding signal based on the drug feeding driving data.

[0013] Another aspect of the present invention provides an automatic medication delivery method based on computer vision, the method comprising:

[0014] Acquire a dose image from the medicine bottle and identify the real-time dose in the dose image;

[0015] Based on the acquisition timestamp of the dose image and the real-time dose, determine whether to trigger the automatic medication process;

[0016] If triggered, an automatic drug feeding signal is generated, and the robotic arm is driven by the automatic drug feeding signal so that the robotic arm feeds the drug to the designated target.

[0017] In one implementation, identifying the real-time dose in the dose image includes:

[0018] Identify the detection frame of the medicine bottle from the dosage image, and crop the dosage image based on the detection frame;

[0019] Identify boundary lines from the cropped image; these boundary lines characterize the outline of the medicine bottle and the liquid plane of the medicine.

[0020] Based on the positional relationship between the boundary lines, the liquid plane of the agent is identified from the boundary lines;

[0021] The position of the drug corresponding to the liquid plane is read, and the real-time dose is determined based on the position of the drug.

[0022] In one embodiment, identifying the liquid plane of the agent from the boundary line includes:

[0023] Identify the largest circumscribed boundary from the boundary lines, remove the largest circumscribed boundary from the boundary lines, and use the remaining boundary lines as the liquid plane of the agent.

[0024] In one embodiment, after the robotic arm is driven by the automatic medication feeding signal, the method further includes:

[0025] Step A1: The robotic arm generates a return signal based on the received signal using formula (1).

[0026]

[0027] Where F 16 This represents the hexadecimal form of the return signal; S 16 This represents the hexadecimal form of the automatic medication feeding signal; S 16 (1→4) represents the data S 16 The first four digits of the hexadecimal data; len() calculates the total number of digits within the parentheses; A 16 This represents the hexadecimal form of the standard data frame header; E 16 The hexadecimal representation of the standard data frame tail; S 16 {[len(S 16 )-4]→len(S 16 )} represents data S 16 The middle len(S) 16 )-4 bits to the len(S) 16 ) hexadecimal data in digit ; ∧ represents the logical relation AND; () 16 This indicates that the value within the parentheses will be converted to hexadecimal data; error1 represents error1 in ASCII code form; error2 represents error2 in ASCII code form; yes0 represents yes0 in ASCII code form.

[0028] Step A2: Using formula (2), control the number of times the automatic medication feeding signal is sent again based on the returned signal.

[0029] B = {F 16 -(F 16 >>2<<2)]-30 16} 10 (2)

[0030] Where B represents the number of times the automatic medication feeding signal is sent again; >>2 represents a right shift of 2 bits; <<2 represents a left shift of 2 bits; 30 16 Represents 30 in hexadecimal form; {} 10 This indicates that the value within the parentheses will be converted to decimal data;

[0031] Step A3: Use formula (3) to control the robot arm's fault indicator based on the historically received robot arm return signals.

[0032]

[0033] Where G represents the control value of the fault light of the robotic arm; B(i) represents the number of times the automatic drug feeding signal is resent after the i-th robotic arm return signal received in the past; P[] represents the judgment function, the function value is 1 if the formula in the parentheses is true, and the function value is 0 if the formula in the parentheses is false; n represents the total number of robotic arm return signals received in the past.

[0034] If G=1, the fault light controlling the robotic arm will turn on;

[0035] If G=0, the fault light controlling the robotic arm will turn off.

[0036] The technical solution provided by this invention can drive a robotic arm to automatically feed medicine to a target object by identifying the real-time dosage in the medicine bottle, thereby saving manpower and material resources and realizing an automated medicine feeding process.

[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0039] Figure 1 This is a schematic diagram of the functional modules of an automatic drug feeding device based on computer vision in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram illustrating the steps of an automatic drug delivery method based on computer vision in an embodiment of the present invention. Detailed Implementation

[0041] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0042] Please see Figure 1 One embodiment of this application provides an automatic drug delivery device based on computer vision, the device comprising:

[0043] An image processing unit is used to acquire a dose image of the medicine bottle and identify the real-time dose in the dose image;

[0044] The judgment unit is used to determine whether to trigger the automatic drug delivery process based on the acquisition timestamp of the dose image and the real-time dose;

[0045] A drive unit is used to generate an automatic drug feeding signal when triggered, and to drive the robotic arm through the automatic drug feeding signal so that the robotic arm feeds the drug to the designated target.

[0046] In one embodiment, the image processing unit is specifically configured to: identify a detection frame of the medicine bottle from the dosage image and crop the dosage image based on the detection frame; identify boundary lines from the cropped image, the boundary lines being used to characterize the outline of the medicine bottle and the liquid plane of the medicine; identify the liquid plane of the medicine from the boundary lines based on the positional relationship between the boundary lines; read the position of the medicine corresponding to the liquid plane, and determine the real-time dose based on the position of the medicine.

[0047] Specifically, an object detection algorithm can be used to detect the bounding boxes of each object in the dose image. Each bounding box can be labeled with the object type. By identifying the object type label, the bounding box of the medicine bottle can be identified from among many bounding boxes. Based on the bounding box, the area image of the medicine bottle can be cropped, thereby reducing the amount of data to be processed.

[0048] In the cropped area image, boundary lines can be identified using a boundary recognition algorithm. These boundary lines can be the boundary between the medicine bottle and its surrounding environment, or the boundary between the liquid plane of the medicine and the air.

[0049] In one embodiment, the image processing unit is further configured to identify the maximum circumscribed boundary from the boundary line, remove the maximum circumscribed boundary from the boundary line, and use the remaining boundary line as the liquid plane of the agent.

[0050] Generally speaking, the boundary line that forms the outline of a medicine bottle is usually the outermost of all boundary lines. Therefore, by identifying the largest circumscribed boundary, the outline of the medicine bottle can be effectively removed. What remains is the boundary between the liquid plane of the medicine and the space. The scale corresponding to this boundary in the medicine bottle can indicate the real-time dosage.

[0051] In one embodiment, the judgment unit is specifically used to: determine the drug feeding cycle of the target object and the initial drug dose in the drug bottle; determine the drug consumption based on the drug feeding cycle and the collection timestamp; determine the theoretical remaining drug amount based on the initial drug dose and the drug consumption; and compare the theoretical remaining drug amount with the real-time dose to determine whether to trigger the automatic drug feeding process.

[0052] The drug administration cycle can be the dosage at each administration. The product of the drug administration cycle and the data collection timestamp can be used as the drug consumption. Subtracting the drug consumption from the initial drug dosage gives the remaining drug amount.

[0053] In one embodiment, the judgment unit is specifically used to calculate the difference between the theoretical remaining amount of medicine and the real-time dose; if the difference is less than or equal to a specified threshold, it is determined that the automatic medication process is not triggered; if the difference is greater than the specified threshold, it is determined that the automatic medication process is triggered.

[0054] In one embodiment, the driving unit is specifically used to identify the identity of the target object and acquire drug feeding driving data related to the identity, wherein the drug feeding driving data at least characterizes the liquid injection rate during the drug feeding process; and generate an automatic drug feeding signal based on the drug feeding driving data.

[0055] Please see Figure 2 In another aspect, the present invention provides an automatic medication delivery method based on computer vision, the method comprising:

[0056] S1: Acquire a dose image of the medicine bottle and identify the real-time dose in the dose image;

[0057] S2: Based on the acquisition timestamp of the dose image and the real-time dose, determine whether to trigger the automatic drug delivery process;

[0058] S3: If triggered, an automatic medication feeding signal is generated, and the robotic arm is driven by the automatic medication feeding signal to feed medication to the designated target. The specific steps include...

[0059] Step A1: After the automatic drug feeding signal is sent to the robotic arm, the robotic arm will generate a return signal based on the received signal using formula (1).

[0060]

[0061] Where F 16 This represents the hexadecimal form of the return signal; S 16 This represents the hexadecimal form of the automatic medication feeding signal; S 16 (1→4) represents the data S 16 The first four digits of the hexadecimal data; len() calculates the total number of digits within the parentheses; A 16 This represents the hexadecimal form of the standard data frame header; E 16 The hexadecimal representation of the standard data frame tail; S 16 {[len(S 16 )-4]→len(S 16 )} represents data S16 The middle len(S) 16 )-4 bits to the len(S) 16 ) hexadecimal data in digit ; ∧ represents the logical relation AND; () 16 This indicates that the value within the parentheses will be converted to hexadecimal data; error1 represents error1 in ASCII code form; error2 represents error2 in ASCII code form; yes0 represents yes0 in ASCII code form.

[0062] Step A2: Using formula (2), control the number of times the automatic medication feeding signal is sent again based on the returned signal.

[0063] B = {F 16 -(F 16 >>2<<2)]-30 16} 10 (2)

[0064] Where B represents the number of times the automatic medication feeding signal is sent again; >>2 represents a right shift of 2 bits; <<2 represents a left shift of 2 bits; 30 16 Represents 30 in hexadecimal form; {} 10 This indicates that the value within the parentheses will be converted to decimal data;

[0065] Step A3: Use formula (3) to control the robot arm's fault indicator based on the historically received robot arm return signals.

[0066]

[0067] Where G represents the control value of the fault light of the robotic arm; B(i) represents the number of times the automatic drug feeding signal is resent after the i-th robotic arm return signal received in the past; P[] represents the judgment function, the function value is 1 if the formula in the parentheses is true, and the function value is 0 if the formula in the parentheses is false; n represents the total number of robotic arm return signals received in the past.

[0068] If G=1, the fault light controlling the robotic arm will turn on;

[0069] If G=0, the fault light controlling the robotic arm will turn off.

[0070] The beneficial effects of the above technical solution are as follows: using formula (1) in step A1 to generate a return signal based on the received signal, the state of the robotic arm receiving the signal can be known based on the return signal, which facilitates subsequent control; then using formula (2) in step A2 to control the number of times the automatic feeding signal is sent again based on the return signal, so that a new automatic feeding signal is repeatedly sent when an erroneous signal is received, ensuring the reliability of the robotic arm's normal operation; then using formula (3) in step A3 to control the fault light of the robotic arm based on the historically received robotic arm return signals, so that the fault light is lit up after multiple fault signals are received to remind the staff to carry out maintenance.

[0071] In one implementation, identifying the real-time dose in the dose image includes:

[0072] Identify the detection frame of the medicine bottle from the dosage image, and crop the dosage image based on the detection frame;

[0073] Identify boundary lines from the cropped image; these boundary lines characterize the outline of the medicine bottle and the liquid plane of the medicine.

[0074] Based on the positional relationship between the boundary lines, the liquid plane of the agent is identified from the boundary lines;

[0075] The position of the drug corresponding to the liquid plane is read, and the real-time dose is determined based on the position of the drug.

[0076] In one embodiment, identifying the liquid plane of the agent from the boundary line includes:

[0077] Identify the largest circumscribed boundary from the boundary lines, remove the largest circumscribed boundary from the boundary lines, and use the remaining boundary lines as the liquid plane of the agent.

[0078] In one implementation, generating an automatic medication feeding signal includes:

[0079] Determine the drug feeding cycle for the target object and the initial drug dosage in the drug bottle; determine the drug consumption based on the drug feeding cycle and the collection timestamp.

[0080] The theoretical remaining amount of drug is determined based on the initial drug dosage and the amount of drug consumed;

[0081] By comparing the theoretical remaining amount of medicine with the real-time dose, it is determined whether to trigger the automatic medication process.

[0082] The technical solution provided by this invention can drive a robotic arm to automatically feed medicine to a target object by identifying the real-time dosage in the medicine bottle, thereby saving manpower and material resources and realizing an automated medicine feeding process.

[0083] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An automatic medication feeding device based on computer vision, characterized in that, The device includes: An image processing unit is used to acquire a dose image of the medicine bottle and identify the real-time dose in the dose image; The judgment unit is used to determine whether to trigger the automatic drug delivery process based on the acquisition timestamp of the dose image and the real-time dose; A drive unit is used to generate an automatic drug feeding signal when triggered, and drive the robotic arm through the automatic drug feeding signal so that the robotic arm feeds the drug to the designated target; Specifically, the image processing unit is used to: identify the detection frame of the medicine bottle from the dosage image and crop the dosage image based on the detection frame; identify boundary lines from the cropped image, the boundary lines being used to characterize the outline of the medicine bottle and the liquid plane of the medicine; identify the liquid plane of the medicine from the boundary lines based on the positional relationship between the boundary lines; read the position of the medicine corresponding to the liquid plane, and determine the real-time dosage based on the position of the medicine; The system includes, after driving the robotic arm via the automatic medication feeding signal, the following: Step A1: The robotic arm generates a return signal based on the received signal using formula (1). （1） in The hexadecimal representation of the return signal; The hexadecimal form representing the automatic medication feeding signal; Representing data The hexadecimal data from the first to the fourth digit; This indicates the total number of digits in the data within the parentheses; The hexadecimal representation of the standard data frame header; The hexadecimal representation of the end of a standard data frame; Representing data The Middle Ranked first Hexadecimal data in bits; Indicates a logical relationship; This indicates that the value within the parentheses will be converted to hexadecimal data; Representing ASCII code form ; Representing ASCII code form ; Representing ASCII code form ; Step A2: Using formula (2), control the number of times the automatic medication feeding signal is sent again based on the return signal. （2） in This indicates the number of times the automatic medication feeding signal will be sent again; This indicates a right shift of 2 bits; This indicates a left shift of 2 bits; This represents 30 in hexadecimal form; This indicates that the value within the parentheses will be converted to decimal data; Step A3: Use formula (3) to control the fault light of the robotic arm based on the historically received robotic arm return signals. （3） in The control value of the fault indicator light for the robotic arm; Indicates the number of historically received [items / received]. The number of times the automatic medication feeding signal is resent is calculated from the return signal of the robotic arm; This represents a conditional function. If the expression within the parentheses is true, the function value is 1; if the expression within the parentheses is false, the function value is 0. This represents the total number of robotic arm return signals received in the past. like If so, the malfunction indicator light controlling the robotic arm will turn on; like If the fault indicator light on the robotic arm is turned off, then the fault indicator light will be turned off.

2. The apparatus according to claim 1, characterized in that, The image processing unit is further configured to identify the maximum circumscribed boundary from the boundary line, remove the maximum circumscribed boundary from the boundary line, and use the remaining boundary line as the liquid plane of the agent.

3. The apparatus according to claim 1, characterized in that, The judgment unit is specifically used to determine the drug feeding cycle of the target object and the initial drug dosage in the drug bottle, and to determine the drug consumption based on the drug feeding cycle and the collection timestamp; Based on the initial drug dosage and the drug consumption, determine the theoretical remaining drug amount; compare the theoretical remaining drug amount with the real-time dosage to determine whether to trigger the automatic drug delivery process.

4. The apparatus according to claim 3, characterized in that, The judgment unit is specifically used to calculate the difference between the theoretical remaining amount of medicine and the real-time dose. If the difference is less than or equal to a specified threshold, it is determined that the automatic medication process is not triggered. If the difference is greater than the specified threshold, the automatic medication feeding process is triggered.

5. The apparatus according to claim 1, characterized in that, The driving unit is specifically used to identify the identity of the target object and obtain drug feeding driving data related to the identity, wherein the drug feeding driving data at least characterizes the liquid injection rate during the drug feeding process; An automatic medication feeding signal is generated based on the medication driving data.

6. An automated medication delivery method based on computer vision, characterized in that, The method includes: Acquire a dose image from the medicine bottle and identify the real-time dose in the dose image; Based on the acquisition timestamp of the dose image and the real-time dose, determine whether to trigger the automatic medication process; If triggered, an automatic drug feeding signal is generated, and the robotic arm is driven by the automatic drug feeding signal so that the robotic arm feeds the drug to the designated target; The identification of the real-time dose in the dose image includes: Identify the detection frame of the medicine bottle from the dosage image, and crop the dosage image based on the detection frame; Identify boundary lines from the cropped image; these boundary lines characterize the outline of the medicine bottle and the liquid plane of the medicine. Based on the positional relationship between the boundary lines, the liquid plane of the agent is identified from the boundary lines; Read the position of the drug corresponding to the liquid plane, and determine the real-time dose based on the position of the drug; The method further includes, after driving the robotic arm via the automatic drug feeding signal: Step A1: The robotic arm generates a return signal based on the received signal using formula (1). （1） in The hexadecimal representation of the return signal; The hexadecimal form representing the automatic medication feeding signal; Representing data The hexadecimal data from the first to the fourth digit; This indicates the total number of digits in the data within the parentheses; The hexadecimal representation of the standard data frame header; The hexadecimal representation of the end of a standard data frame; Representing data The Middle Ranked first Hexadecimal data in bits; Indicates a logical relationship; This indicates that the value within the parentheses will be converted to hexadecimal data; Representing ASCII code form ; Representing ASCII code form ; Representing ASCII code form ; Step A2: Using formula (2), control the number of times the automatic medication feeding signal is sent again based on the return signal. （2） in This indicates the number of times the automatic medication feeding signal will be sent again; This indicates a right shift of 2 bits; This indicates a left shift of 2 bits; This represents 30 in hexadecimal form; This indicates that the value within the parentheses will be converted to decimal data; Step A3: Use formula (3) to control the fault light of the robotic arm based on the historically received robotic arm return signals. （3） in The control value of the fault indicator light for the robotic arm; Indicates the number of historically received [items / received]. The number of times the automatic medication feeding signal is resent is calculated from the return signal of the robotic arm; This represents a conditional function. If the expression within the parentheses is true, the function value is 1; if the expression within the parentheses is false, the function value is 0. This represents the total number of robotic arm return signals received in the past. like If so, the malfunction indicator light controlling the robotic arm will turn on; like If the fault indicator light on the robotic arm is turned off, then the fault indicator light will be turned off.

7. The method according to claim 6, characterized in that, Identifying the liquid plane of the agent from the boundary line includes: Identify the largest circumscribed boundary from the boundary lines, remove the largest circumscribed boundary from the boundary lines, and use the remaining boundary lines as the liquid plane of the agent.

Images