Drug use detection processing method and device

Abstract

The embodiment of the invention provides a drug use detection processing method and device.The drug use detection processing method comprises the steps that in the drug use detection process, after a drug image obtained by conducting image collection on a drug before drug use is obtained, drug use detection is conducted in combination with the drug image and at least one pre-configured drug use task, after medicine use detection is passed, medicine use state information, recognized based on the rotation image sequence of the medicine after medicine use, of the user for the medicine is obtained, medicine use feedback processing is conducted on the associated user in combination with the medicine use state information and the corresponding medicine use feedback level, and therefore medicine use state recognition is achieved through the rotation image sequence of the medicine after medicine use; therefore, medicine use feedback is carried out to the associated user through the medicine use state information obtained through identification.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a detection and processing method and apparatus for drug use. Background Technology

[0002] With the continuous development and promotion of internet technology, online services based on internet technology are also developing rapidly. At the same time, with the wide variety of medicines and their different packaging, the dosage of the same type of medicine may also be different. Therefore, online medicine services have emerged to provide users with convenient means of accessing medicines and meet their diverse needs. However, with the increasing number of providers of medicine-related services, certain pressures and challenges have also been brought to these providers. Summary of the Invention

[0003] This specification provides one or more embodiments of a drug use detection and processing method, comprising: acquiring a drug image obtained by image acquisition of the drug before administration; performing drug use detection based on the drug image and at least one pre-configured drug use task; if the drug use detection passes, acquiring user drug use status information uploaded by a user terminal regarding the drug; the drug use status information is obtained by drug use status recognition based on a sequence of rotated images of the drug after administration; and providing drug use feedback processing to the associated user based on the drug use status information and the corresponding drug use feedback level.

[0004] This specification provides one or more embodiments of another method for detecting and processing drug use, including: acquiring images of the drug before administration and uploading them to a server, to perform drug use detection based on the drug images and at least one pre-configured drug use task. If the drug use detection passes, acquiring rotated images of the drug after administration to obtain a sequence of rotated images of the drug. Recognizing the user's drug use status information based on the rotated image sequence. Uploading the drug use status information to the server to provide drug use feedback to associated users based on the drug use status information and the corresponding drug use feedback level.

[0005] This specification provides one or more embodiments of a drug use detection and processing device, comprising: an image acquisition module configured to acquire a drug image obtained by image acquisition of the drug before administration; a drug use detection module configured to perform drug use detection based on the drug image and at least one pre-configured drug use task; an information acquisition module configured to acquire, if the drug use detection passes, user drug use status information uploaded by a user terminal regarding the drug. The drug use status information is obtained by drug use status identification based on a sequence of rotated images of the drug after administration; and a drug use feedback module configured to provide drug use feedback processing to associated users based on the drug use status information and a corresponding drug use feedback level.

[0006] This specification provides one or more embodiments of another drug use detection and processing device, comprising: an image uploading module configured to acquire an image of the drug before administration and upload it to a server, for drug use detection based on the drug image and at least one pre-configured drug use task; an image acquisition module configured to acquire a sequence of rotated images of the drug after administration if the drug use detection is passed; a status recognition module configured to identify the user's drug use status information based on the rotated image sequence; and an information uploading module configured to upload the drug use status information to the server, for drug use feedback processing to the associated user based on the drug use status information and the corresponding drug use feedback level.

[0007] This specification provides one or more embodiments of a drug use detection and processing device, comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: acquire a drug image obtained by image acquisition of the drug before administration; perform drug use detection based on the drug image and at least one pre-configured drug use task; if the drug use detection passes, acquire user drug use status information uploaded by a user terminal regarding the drug; the drug use status information is obtained by drug use status identification based on a sequence of rotated images of the drug after administration; and provide drug use feedback processing to the associated user based on the drug use status information and the corresponding drug use feedback level.

[0008] This specification provides one or more embodiments of another drug use detection and processing device, comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: acquire an image of the drug before administration and upload it to a server, to perform drug use detection based on the drug image and at least one pre-configured drug use task; if the drug use detection passes, acquire a rotational image sequence of the drug after administration; identify the user's drug use status information based on the rotational image sequence; and upload the drug use status information to the server to provide drug use feedback to the associated user based on the drug use status information and a corresponding drug use feedback level.

[0009] This specification provides one or more embodiments of a computer-readable storage medium for storing computer-executable instructions that, when executed, perform the following steps: acquiring a drug image obtained by image acquisition of the drug before administration; performing medication detection based on the drug image and at least one pre-configured medication task; if the medication detection passes, acquiring user medication status information for the drug uploaded by a user terminal; the medication status information is obtained by medication status recognition based on a sequence of rotated images of the drug after administration; and providing medication feedback to the associated user based on the medication status information and the corresponding medication feedback level.

[0010] This specification provides one or more embodiments of another computer-readable storage medium for storing computer-executable instructions that, when executed, perform the following steps: acquiring an image of the drug before administration and uploading it to a server to perform medication detection based on the drug image and at least one pre-configured medication task; if the medication detection passes, acquiring a sequence of rotated images of the drug after administration; identifying the user's medication status information for the drug based on the sequence of rotated images; and uploading the medication status information to the server to provide medication feedback to the associated user based on the medication status information and the corresponding medication feedback level. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 A schematic diagram illustrating the implementation environment of a detection and processing method for drug use provided in one or more embodiments of this specification; Figure 2 A flowchart illustrating a detection and processing method for drug use provided in one or more embodiments of this specification; Figure 3 A schematic diagram of a drug identification page provided for one or more embodiments of this specification; Figure 4 This is a schematic diagram illustrating a page detection process provided for one or more embodiments of this specification. Figure 5 This diagram illustrates a method for detecting pages that fail to pass inspection, provided for one or more embodiments of this specification. Figure 6A clockwise rotation diagram of a medicine provided for one or more embodiments of this specification; Figure 7 A schematic diagram of counterclockwise rotation of a medicine provided in one or more embodiments of this specification; Figure 8 A flowchart illustrating a detection and processing method for drug use in a drug use scenario, provided by one or more embodiments of this specification. Figure 9 A flowchart of a detection and processing method for another drug provided in one or more embodiments of this specification; Figure 10 A schematic diagram of an embodiment of a drug use detection and processing device provided in one or more embodiments of this specification; Figure 11 A schematic diagram of another embodiment of a detection and processing device for use in pharmaceuticals provided in one or more embodiments of this specification; Figure 12 A schematic diagram of a detection and processing device for pharmaceutical use provided in one or more embodiments of this specification; Figure 13 This is a schematic diagram of the structure of another detection and processing device for a drug provided in one or more embodiments of this specification. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] The detection and processing methods for drug use provided in one or more embodiments of this specification are applicable to the implementation environment of drug use testing. (Refer to...) Figure 1 The implementation environment includes at least: Server 101; in addition, the implementation environment may also include user terminal 102; The server 101 is used to acquire drug images obtained by image acquisition of the drug before medication, and to perform medication detection by combining the drug images with at least one pre-configured medication task. After medication detection, medication feedback is processed to the associated user based on the user's medication status information and the corresponding medication feedback level. The server 101 can be one or more servers, a server cluster composed of several servers, or a cloud server of a cloud computing platform. User terminal 102 is used to acquire images of the medicine before administration and upload them to the server; to identify the medication status based on the rotational image sequence of the medicine after administration and obtain medication status information; and to upload the medication status information to the server. User terminal 102 can be a mobile phone, personal computer, tablet computer, e-book reader, wearable device, device for information interaction based on AR (Augmented Reality) / VR (Virtual Reality), and laptop computer, etc. In addition, the implementation environment may also include associated user terminal 103 of associated users, which can be used to cooperate with the server to process medication feedback; associated user terminal 103 may be a mobile phone, personal computer, tablet computer, e-book reader, wearable device, device for information interaction based on AR (Augmented Reality) / VR (Virtual Reality) and laptop computer, etc. In this implementation environment, server 101 acquires drug images uploaded by user terminal 102, obtained through image acquisition of the drug before administration. It then performs drug administration detection by combining the drug images with at least one pre-configured drug administration task. After the drug administration detection is passed, it provides drug administration feedback to the associated user based on the user's drug administration status information and the corresponding drug administration feedback level. In this way, drug administration status recognition is achieved through a sequence of rotated images of the drug after administration, thereby enabling drug administration feedback to be provided to the associated user based on the identified drug administration status information.

[0014] One or more embodiments of a detection and processing method for drug use provided in this specification are as follows: Reference Figure 2 The detection and processing method for drugs provided in this embodiment can be applied to a server, specifically including steps S202 to S208.

[0015] Step S202: Obtain the drug image obtained by image acquisition of the drug before administration.

[0016] The medicine described in this embodiment can be any type of medicine, such as capsule medicine, tablet medicine, granule medicine and / or liquid medicine; the medicine image may include an image of the medicine itself, an image of the medicine packaging and / or an image of the medicine packaging carrier, and the image of the medicine packaging carrier may include an image of the medicine box and / or an image of the medicine bottle.

[0017] In practice, the system acquires images of the medication before administration. Specifically, the user terminal can acquire images of the medication before administration and upload them to the server. Correspondingly, the server can acquire the uploaded images of the medication from the user terminal. Here, the medication before administration can refer to the medication that the user is taking before administration. More specifically, users can submit drug identification commands through the service program. The user terminal can then render and display a drug identification interface based on these commands. This interface captures an image of the drug before administration and uploads it to the server. The server can then retrieve the drug image. For example... Figure 3 The drug recognition page shows that the user holds the capsule and the medicine box for image capture. During the image capture process, the drug recognition page displays an image capture reminder, which reads "Camera is on, recognizing medicine. Please show a picture of a hand holding the medicine or a real picture of the medicine." The user then performs image capture based on the image capture reminder.

[0018] The service program can be an application, subprogram (mini-program), or web application that provides medical services; that is, the service program can be a medical service program, a medical application, a medical subprogram, or a medical web application. The user can be a user who has a need for medication, and the user can specifically be a patient. The associated user can be a user's related user, specifically a user who has a relationship with the user, such as a relative, friend, or colleague of the user. In addition, the user can be a patient, and the associated user can be a doctor who provides medical consultation or diagnosis to the user, or a doctor assigned to the user by the service program.

[0019] In addition, during the process of acquiring images of the medicine before administration, the user terminal can also acquire images of the medicine being held in the hand and / or the medicine packaging carrier before administration; or it can acquire images of the medicine being held in the hand and / or the medicine packaging carrier before administration; among these, the medicine image can also be an image of the hand holding the medicine and / or an image of the hand holding the medicine packaging carrier.

[0020] In practical applications, there may be a need for medication review through associated users. To address this, and to meet the need for associated users to review medication use, this embodiment provides an optional implementation where, after acquiring the drug image obtained from pre-administration image acquisition, the drug image can be synchronized to the associated user for medication review, or the drug image can be compressed and synchronized to the associated user for medication review. Furthermore, in response to a medication error alert sent by the associated user after medication review, a medication error voice message generated based on the associated user's voiceprint features can be sent to the user terminal. Further, a medication error warning can be generated based on medication misuse prompts and / or the associated user's contact information and sent to the user terminal. Specifically, after acquiring the drug image obtained from pre-administration image acquisition, the following operations can also be performed: The drug image is compressed and synchronized to the associated user; optionally, after the associated user reviews the medication based on the compressed drug image, the associated user terminal generates a medication error reminder based on the associated user's medication error instruction and sends it to the server; Send medication error voice messages generated based on the voiceprint features of associated users to the user terminal; generate medication error warnings based on drug misuse prompts and the contact information of associated users and send them to the user terminal.

[0021] Specifically, after medication review based on compressed drug images, associated users can submit medication error commands by triggering a medication error space. The associated user's terminal generates a medication error reminder based on the command and sends it to the server. The server can then send a medication error voice message generated based on the associated user's voiceprint features to the user's terminal. The user's terminal can play the medication error voice message, thus providing initial emergency handling of medication errors to the user. Furthermore, the server can generate a medication error warning based on the drug misuse prompt and / or the associated user's phone number and send it to the user's terminal, which can then render the warning. The medication error voice message can be generated in real time or in advance.

[0022] Among them, medication error voice messages refer to voice messages that indicate medication errors to users, such as "Mom, medication error"; medication ingestion prompts can be instructions on how to handle medication ingestion after it has occurred, such as "You may have accidentally ingested x1 hypoglycemic drug. If the time since the ingestion is short, please follow these initial steps: immediately consume sugary food, please rest quietly and wait for further assistance."

[0023] In addition, the operation of compressing the drug image and synchronizing the compressed drug image to the associated user can be replaced by synchronizing the drug image to the associated user; the operation of generating a medication error warning based on the drug misuse prompt and the contact information of the associated user can be replaced by generating a medication error warning based on the drug misuse prompt and / or the contact information of the associated user.

[0024] Step S204: Perform drug administration detection based on the drug image and at least one pre-configured drug administration task.

[0025] The above-mentioned acquisition of drug images obtained by image acquisition of drugs before medication. In this step, medication detection is performed by combining the drug images with at least one pre-configured medication task. This medication detection ensures that the user takes the medication correctly and avoids the user mistakenly taking drugs other than those assigned in the medication task.

[0026] In this embodiment, at least one medication task refers to one or more medication tasks or medication plans; a medication task refers to a user's task of taking medication or a user's plan of using medication; a medication task can be constructed based on the drug identifier, i.e., drug name, medication method, medication time, user identifier, medication start time and / or medication duration; medication method refers to the way of taking medication, such as medication method is three times a day; medication time refers to the time point when the user takes medication; at least one medication task can be configured for a user and can be obtained by associating the user with the medication task configuration.

[0027] In practice, medication detection can be performed based on drug images and at least one pre-configured medication task. Specifically, medication detection can be performed based on drug images and the task drugs included in at least one pre-configured user medication task, or the drug image can be input into a visual recognition engine for visual recognition to obtain visual recognition results, and the medication detection result can be determined based on the visual recognition results. Here, the task drugs can be the drugs recorded in each medication task in at least one medication task; the visual recognition results can be similarity, matching degree and / or matching index. If the medication test passes, a result containing the drug identifier can be returned to the user terminal. The user terminal can then render and display a "Test Passed" page based on this result. Conversely, if the medication test fails, a result containing the drug identifier can be returned to the user terminal. The user terminal can then render and display a "Test Failed" page based on this result. For example... Figure 4 The page shown indicates that the detection passed. This page includes the drug's name, image, and / or a successful match marker indicating that the drug matches any of the drugs included in the drug task. For example... Figure 5 The page shown is a "Detection Failed" page. This page contains the drug name, drug image, and / or a "Match Failed" flag indicating that the drug does not match any of the drugs included in the drug task.

[0028] In the specific implementation process, in order to improve the accuracy and effectiveness of medication detection, medication detection can be performed from a visual dimension. In one optional implementation method provided in this embodiment, the medication can be matched with the task medications included in each medication task based on the visual features of the medication extracted from the medication image. If the match is successful, the medication detection is determined to have passed; if the match fails, the medication detection is determined to have failed. Specifically, in the process of performing medication detection based on the medication image and at least one pre-configured medication task, the following operations can be performed: Visual features of drugs are obtained by extracting visual features from drug images; Based on the visual features of the drugs, the drugs are matched with the drugs included in each medication task. If the match is successful, the medication test is considered passed; if the match fails, the medication test is considered failed.

[0029] Among them, drug visual features refer to the visual features of drugs, which may include drug color features, drug outline features, drug texture features, drug pattern features, drug transparency features, drug suspension features, drug packaging text features and / or drug packaging style features.

[0030] Specifically, in the process of extracting visual features from drug images to obtain drug visual features, the drug category can be identified based on the drug image, and then visual features can be extracted from the drug image according to the feature extraction method corresponding to the drug category. In the process of extracting visual features from the drug image according to the feature extraction method corresponding to the drug category, if the drug category is a capsule, drug color features and / or drug outline features can be extracted from the drug image to obtain drug color features and / or drug outline features; if the drug category is an aluminum foil drug, aluminum foil pattern extraction and / or text extraction can be performed to obtain aluminum foil pattern features and / or aluminum foil text features; if the drug category is a granular drug, drug color features, drug shape features, and / or drug texture features can be extracted from the drug image to obtain drug color features, drug shape features, and / or drug texture features; if the drug category is a liquid drug, text features, transparency features, and / or suspension features can be extracted from the drug image to obtain text features, drug transparency features, and / or drug suspension features. Here, drug suspension features can be features that characterize the presence of suspended matter in the drug.

[0031] In the process of matching drugs with the drugs included in each medication task based on the visual features of the drugs, a matching index can be calculated based on the visual features of the drugs and the benchmark visual features of the drugs in the user's drug feature database. If the matching index is greater than the index threshold, the matching is determined to be successful. If the matching index is less than or equal to the index threshold, the matching is determined to be unsuccessful.

[0032] Based on this, in the process of matching drugs with the task drugs included in each medication task based on drug visual features, this embodiment provides an optional implementation method. In this process, similarity is calculated based on the drug's color features and the baseline drug color features in the user's drug color feature library. If the similarity is greater than a similarity threshold, a successful match is determined. Alternatively, a matching degree is calculated based on the drug's outline features and the baseline drug outline features. If the matching degree is greater than a matching degree threshold, a successful match is determined. Alternatively, a matching index can be calculated based on the drug's color features and outline features, and if the matching index is greater than an index threshold, a successful match is determined. Specifically, in the process of matching drugs with the task drugs included in each medication task based on drug visual features, the following operations can be performed: The similarity is calculated based on the drug color features and the baseline drug color features in the user's drug color feature library; optionally, the drug color feature library is constructed based on the drug color features of the drugs included in each of the user's medication tasks. The matching degree is calculated based on the drug's profile features and the benchmark drug's profile features. A matching index is also calculated based on similarity and matching degree. If the matching index is greater than the index threshold, the matching is considered successful; if the matching index is less than or equal to the index threshold, the matching is considered unsuccessful.

[0033] Among them, the color characteristics of a drug can be a color histogram of the drug, such as the HSV (Hue-Saturation-Value Histogram) color histogram of a drug, which is a three-dimensional color characteristic composed of H (hue), S (saturation), and V (brightness).

[0034] Specifically, the drug color feature library can be constructed based on the drug color features of the drugs included in each medication task after the associated user is configured with at least one medication task. In calculating the matching index based on similarity and matching degree, the matching index can be calculated based on similarity, matching degree, and their respective weight values. In calculating the matching degree based on the drug contour features and the baseline drug contour features, the Euclidean distance between the drug contour features and the baseline drug contour features can be calculated as the matching degree. The drug contour features can be drug contour geometric features, obtained by inputting the drug image into a contour extraction algorithm, which can be the Hu invariant moment algorithm or an invariant moment algorithm. In addition, the baseline drug color features and / or baseline drug contour features can also be stored in the user's drug feature library.

[0035] Furthermore, in the process of matching drugs with the task drugs included in each medication task based on the visual features of the drugs, standard drug identifiers can be obtained by standardizing the drugs based on the textual features of the drugs. Based on the standard drug identifiers, matching task drugs can be queried in the drug set composed of task drugs included in each medication task. If the query result is not empty and / or the matching index is greater than the index threshold, the matching can be determined to be successful. If the query result is empty and / or the matching index is less than or equal to the index threshold, the matching can be determined to be unsuccessful. In the case that the query result is not empty, the matching result of the drug dosage, medication prompts and / or standard packaging image of the matching task drugs can also be returned to the user terminal.

[0036] After the above-mentioned matching process based on the visual characteristics of the drugs with the task drugs included in each medication task is executed, if the matching fails, the system can also query the drug database for structured information containing drug dosage, medication instructions, and / or standard packaging images based on the standard drug identifier obtained by standardizing the drug's textual characteristics and return it to the user terminal. Since a matching failure indicates that the drug is not included in the user's medication task, there is a risk of incorrect medication use. To reduce the probability of incorrect medication use, the system can further allow the user to identify the drug using standard packaging images, drug dosages, etc., in the structured information, thus reducing the risk of incorrect medication use. In an optional implementation of this embodiment, after the matching process based on the visual characteristics of the drugs with the task drugs included in each medication task is executed, the following operation is also performed: If the matching fails, a standard drug identifier is obtained by standardizing the drug based on the drug text features; optionally, the drug text features are obtained by extracting text features from the drug packaging image. Based on the standard drug identifier, the system queries the drug database for structured information containing drug dosage, medication instructions, and / or standard packaging diagrams, and returns a matching failure result containing structured information to the user terminal.

[0037] Specifically, in the process of obtaining standard drug identification by standardizing drugs based on drug text features, the drug text features can be input into a named entity model in the drug domain for named entity recognition to obtain drug entities in the drug text features. The drug entities are then standardized to obtain standard drug names. The drug entities here may include drug name entities and / or drug batch number entities. Medication prompts may be prohibition prompts for drug use.

[0038] The drug text features can be obtained as follows: input the drug packaging image into a text detection model to detect text regions in the drug packaging image, segment the drug packaging image into image blocks according to the text regions to obtain text image blocks, and input the text image blocks into an optical character engine for optical character recognition to obtain drug text features; drug text features may include drug name and / or drug batch number; the model structure of the text detection model can be a neural network model, such as the text detection model being the EAST (Efficient and Accurate Scene Text Detector) model, the optical character engine being the Tesseract OCR (Tesseract Optical Character Recognition, a cross-platform optical character recognition engine), and the named entity model being the BiLSTM-CRF (Bidirectional Long Short-Term Memory - Conditional Random Field, a deep learning model architecture for sequence labeling tasks in natural language processing).

[0039] Furthermore, after the server performs matching processing on the drugs and the drugs included in each medication task, it may obtain a matching failure result. Based on this, the associated user may submit a medication error reminder after reviewing the medication based on the medication image. Therefore, in order to reduce the probability and risk of users taking medication errors, in addition to the server automatically performing matching processing and returning the matching failure result to the user terminal, a medication error warning can also be returned to the user terminal from the perspective of manual review by the associated user. This combines the manual review and automated review dimensions to provide medication warnings to users and reduce the risk of users taking medication errors. Specifically, in addition to returning the matching failure result containing structured information to the user terminal, this embodiment can also synchronize the drug image to the associated user for medication review or compress the drug image and synchronize the compressed drug image to the associated user for medication review. Based on this, in response to the medication error reminder sent by the associated user after reviewing the medication, a medication error voice generated based on the associated user's voiceprint features can be sent to the user terminal. Furthermore, a medication error warning can be generated based on the drug misuse prompt and / or the contact information of the associated user and sent to the user terminal. Specifically, the following operations can also be performed: The drug image is compressed and synchronized to the associated user; optionally, after the associated user reviews the medication based on the compressed drug image, the associated user terminal generates a medication error reminder based on the associated user's medication error instruction and sends it to the server; Send medication error voice messages generated based on the voiceprint features of associated users to the user terminal; generate medication error warnings based on drug misuse prompts and / or the contact information of associated users and send them to the user terminal.

[0040] Specifically, in the process of compressing drug images, an encoding compression method can be used to encode and compress the drug images to obtain compressed drug images; for example, H.265 encoding (High Efficiency Video Coding, a video compression standard) can be used to compress the drug images. In the process of synchronizing the compressed drug images to associated users, a preset power consumption protocol can be used to synchronize the compressed drug images to associated users, such as using the low-power LoRaWAN (Long Range Wide Area Network) protocol. The specific implementation process of this operation has been described above and will not be repeated here.

[0041] It should be noted that the above-mentioned drug detection operation based on drug images and at least one pre-configured drug use task can be replaced by drug detection based on drug images and / or at least one pre-configured drug use task, and together with other processing steps provided in this embodiment, form a new implementation method.

[0042] Step S206: If the medication test passes, obtain the user's medication status information for the drug uploaded by the user terminal.

[0043] The above-mentioned drug use detection is based on drug images and at least one pre-configured drug use task. In this step, if the drug use detection passes, the user's drug use status information uploaded by the user terminal is obtained. In practice, if the medication test passes, the user terminal can acquire a rotational image sequence of the medication after administration. Based on this sequence, the user's medication status is identified, and this information is uploaded to the server. Specifically, before performing this process, it can be verified whether the medication in the rotational image sequence matches the medication in the matching medication task. If so, the operation to obtain the user's medication status information based on the rotational image sequence can proceed, and the server can retrieve the user's medication status information uploaded by the user terminal. Optionally, the medication status information can be obtained through medication status identification based on the rotational image sequence of the medication after administration. The medication in the matching medication task can be any medication included in the medication task whose image was matched during the medication test.

[0044] The medication status information mentioned in this embodiment refers to the user's medication status information for the drug. The medication status information may include non-medication status information with reasons for not using the drug or medication status information. The rotating image sequence of the drug may be a rotating image sequence of the drug itself and / or a rotating image sequence of the drug packaging. The drug packaging may be a carrier for packaging the drug, such as a medicine box, medicine bottle and / or a drug aluminum-plastic blister pack.

[0045] In specific implementation, to reduce the power consumption of the user terminal and avoid invalid image acquisition, the rotation image sequence can optionally be acquired when the user's hand contact area on the user terminal is greater than a preset contact area. The hand contact area is predicted based on the capacitance change data of the user terminal's capacitance sensor. That is, the user can acquire rotation images of the medicine after taking the medication, meaning the user can rotate the medicine and / or medicine packaging after taking the medication. The user terminal can call an image acquisition component such as a camera to acquire a rotation image sequence of the medicine and / or medicine packaging. The rotation image sequence of the medicine after taking the medication can be acquired when the hand contact area on the user terminal is greater than a preset contact area. The capacitance change data here can be the amount of capacitance change.

[0046] Specifically, the user terminal can acquire a sequence of rotating images of the medication after administration, either clockwise or counterclockwise; for example... Figure 6The illustration shows a clockwise rotation of a medication. After administration, the user rotates the medication clockwise, and the user terminal captures a sequence of images showing this clockwise rotation. For example... Figure 7 The diagram shown illustrates counter-clockwise rotation. After administration, the user rotates the medication counter-clockwise, and the user terminal captures a sequence of rotating images of the medication.

[0047] After obtaining the rotated image sequence of the medication after administration, medication status recognition can be performed on the user, allowing the user to easily mark the medication status by rotating the medication, thus improving the flexibility and convenience of medication status marking. In one optional implementation of this embodiment, during the medication status recognition process based on the rotated image sequence of the medication after administration, the user's medication status information is obtained by recognizing the medication status based on the medication rotation parameters calculated based on the rotated image sequence. Specifically, the following operations can be performed: The rotation parameters of the drug are obtained by calculating the rotation parameters of the drug based on the rotated image sequence; Medication status information of users can be obtained by identifying medication status based on drug rotation parameters.

[0048] The drug rotation parameters may include the drug rotation angle and / or the drug rotation direction.

[0049] To ensure user privacy and security, the rotation parameters of the medicine can be calculated on the user terminal. Optionally, the rotation parameters of the medicine can be calculated based on the rotated image sequence and executed on a lightweight rotation recognition model deployed on the user terminal. The input of the lightweight rotation recognition model includes the rotated image sequence, and the output of the rotation recognition model includes the rotation parameters of the medicine. The model framework of the lightweight rotation recognition model can be TensorFlow Lite (a lightweight machine learning inference framework) running on an embedded MCU.

[0050] In the process of calculating the rotation parameters of the drug based on the rotated image sequence to obtain the drug rotation parameters, in order to improve the accuracy and comprehensiveness of subsequent drug use status recognition from multiple dimensions, this embodiment provides an optional implementation method in which the following operations are performed: Based on the drug feature points in each rotated image in the rotated image sequence, the rotation direction and rotation angle of the drug are identified to obtain the drug rotation direction and rotation angle.

[0051] Among them, drug feature points refer to key points in the rotated image.

[0052] Specifically, the rotation direction and / or rotation angle of the drug can be identified by the movement trajectory of the same feature points in each rotated image in the rotation image sequence, thereby obtaining the drug rotation direction and / or drug rotation angle.

[0053] Accordingly, in the first optional implementation provided in this embodiment, during the process of obtaining the user's medication status information based on the drug rotation parameters, the following operations are performed: If the drug is rotated clockwise and the rotation angle is greater than the preset angle, the medication status information is determined to be "medication has been used".

[0054] In addition, the operation of determining the medication status information as "medication used" if the drug rotates clockwise and the rotation angle is greater than the preset angle can be replaced by determining the medication status information as "medication used" if the drug rotates counterclockwise and the rotation angle is greater than the preset angle, or it can be replaced by determining the medication status information as "medication used" if the drug rotates clockwise or counterclockwise and / or the rotation angle is greater than the preset angle and / or the drug rotation duration is greater than or equal to the preset duration.

[0055] Furthermore, in an optional implementation of this embodiment, before determining the medication status information as "medication already used" before determining the medication status information if the drug rotation direction is clockwise and the drug rotation angle is greater than a preset angle, the dosage quantity is calculated based on the quantity before and after medication. If the dosage quantity matches the medication method in the medication task, the operation of determining the medication status information as "medication already used" is performed. This can be implemented in the following way: The quantity of drugs before administration is identified based on drug images, and the quantity of drugs after administration is identified based on rotated image sequences. The dosage is calculated based on the quantity before and after medication. If the dosage matches the medication administration method in the medication task, the medication status information is confirmed as "medication administered".

[0056] Among them, the method of drug administration in the drug administration task can be the way the drug is used, such as taking 3 tablets at a time; the matching of the drug quantity with the drug administration method in the drug administration task can include the drug quantity being the same as the quantity corresponding to the drug administration method or the drug quantity being greater than the quantity corresponding to the drug administration method.

[0057] Based on this, after calculating the dosage based on the pre-dosing and post-dosing quantities, if the dosage does not match the dosage method in the medication task, the mismatch result can be synchronized to the associated user terminal, and / or an incorrect medication prompt can be sent to the user terminal, and / or a voice call can be established between the associated user terminal and the user terminal based on the voice request submitted by the associated user terminal; in this embodiment, the voice call can be replaced by a video call.

[0058] Once the medication status information is confirmed as "medication taken," the user terminal can upload the medication status information to the server. The server can generate a medication record based on the medication status information and the medication timestamp. Furthermore, it can push an access link to the user terminal through the system push channel to start the service program. The user terminal can generate a medication message based on the access link. After the medication message is triggered, the user terminal can send a medication access request to the server. The server can send the medication record to the user terminal based on the medication access request. The user terminal can then render the medication taken interface within the service program based on the medication record.

[0059] Furthermore, in real-world applications, users may not have taken their medication. To address this, and in order to comprehensively understand the actual situation regarding medication non-use and improve the success rate of subsequent medication reminders, non-medication status information can be generated based on the reasons selected by the user in the list of reasons for non-medication. In the second optional implementation provided in this embodiment, during the process of obtaining the user's medication status information based on drug rotation parameters, the following operations are also performed: If the drug rotation direction is counterclockwise and the drug rotation angle is greater than the angle threshold, or the drug rotation duration is less than the preset duration, render and display the unused drug interface containing a list of reasons for not using the drug; The system generates non-medication status information based on the reasons for non-medication selected by the user in the list of reasons for non-medication.

[0060] The operation described above, which renders and displays an unused medication interface containing a list of reasons for not using medication, if the medication is rotated counterclockwise and the rotation angle is greater than an angle threshold, or if the rotation duration is less than a preset duration, can be replaced by rendering and displaying an unused medication interface containing a list of reasons for not using medication, if the medication is rotated clockwise or counterclockwise and / or the rotation angle is greater than an angle threshold and / or the rotation duration is less than a preset duration; the unused medication interface may also include the reminder time for the next medication reminder.

[0061] For example, the list of reasons for not taking medication includes reasons such as not having medication with you when you are out, being busy with other things, not wanting to take medication, and other reasons. The medication not-taking status information is generated based on the reasons for not taking medication selected by the user in the list of reasons for not taking medication.

[0062] It should be added that after the medication detection is performed based on the drug image and at least one pre-configured medication task, if the medication detection passes, a medication detection pass result containing the drug identifier can be returned to the user terminal. This allows the user terminal to obtain the user's medication status information for the drug and / or synchronize the medication detection pass result containing the drug identifier with associated users. Specifically, the medication detection pass result containing the drug identifier can be compressed and the compressed result can be sent to associated users and / or the user terminal. If the medication detection fails, a medication detection failure result containing the drug identifier can be returned to the user terminal and / or associated users, allowing the user terminal to re-render the drug recognition page and collect drug images through the drug recognition interface.

[0063] It should also be added that, in the process of identifying the medication status based on the rotated image sequence of the medication after administration, the following operations can also be performed: identify the quantity of medication before administration based on the medication image, and identify the quantity of medication after administration based on the rotated image sequence. The dosage is calculated based on the quantity before and after medication. If the dosage matches the medication administration method in the medication task, the medication status information is determined as the medication status information.

[0064] Based on this, after calculating the medication quantity according to the quantity before and after medication, if the medication quantity does not match the medication method in the medication task, non-medication status information can be generated based on the non-medication reason selected by the user in the non-medication reason list.

[0065] Step S208: Based on the medication status information and the corresponding medication feedback level, medication feedback is processed to the associated user.

[0066] The above-mentioned acquisition of user medication status information uploaded by user terminals. In this step, medication feedback is processed to associated users based on the medication status information and the corresponding medication feedback level, thereby achieving targeted medication feedback to associated users through the medication feedback level.

[0067] The medication feedback level mentioned in this embodiment refers to the level at which associated users provide medication feedback. The medication feedback level can be a level that characterizes the urgency of the medication feedback.

[0068] The medication status information mentioned above can be determined as medication-taken status information. Based on this, to ensure that associated users are promptly aware of the user's medication status and to avoid their concerns about whether the user is taking their medication on time, in the first optional implementation of this embodiment, the following operations are performed during the medication feedback processing to associated users based on the medication status information and the corresponding medication feedback level: The system pushes an access link to the associated user terminal to start the service program; Based on the medication access request submitted after the associated user terminal starts the service program, the medication record generated based on the medication status information is synchronized to the associated user terminal.

[0069] Among them, associated user terminal refers to the terminal equipment of associated users or the user terminal of associated users.

[0070] Specifically, the server can push an access link to the associated user terminal through the system push channel to start the service program. The associated user terminal can generate a medication message based on the access link. After the medication message is triggered, the associated user terminal can send a medication access request to the server. The server can synchronize the medication record generated based on the medication status information to the associated user terminal based on the medication access request. The associated user terminal can render the medication interface within the service program based on the medication record, thereby displaying the user's medication details to the associated user through the medication interface.

[0071] In practical applications, users may fail to take a certain medication despite multiple medication reminders. To address this, and to further strengthen the alerts regarding non-medication, making users aware of the urgency of medication use and improving the success rate of medication administration, the aforementioned medication status information may include non-medication status information with reasons for non-administration. Based on this, in the second optional implementation of this embodiment, during the process of providing medication feedback to associated users based on medication status information and corresponding medication feedback levels, the following operations are performed: Update the number of times a user has not used a drug based on the drug use status information; Feedback on non-medication is sent to the associated users based on the non-medication level corresponding to the frequency range in which the non-medication frequency falls.

[0072] Among them, the non-medication level can be the warning level for users who have not taken medication, that is, the non-medication warning level.

[0073] Based on this, in the first optional implementation provided in this embodiment, during the process of providing feedback on non-medication to the associated user according to the non-medication level corresponding to the frequency range of non-medication, the following operations are performed: The system pushes access links and / or non-medication levels to associated user terminals through the system push channel, so that the service program can be started after the associated user terminal is reminded of non-medication. Based on the medication access request sent by the associated user terminal after starting the service program, generate an unused medication record containing unused medication status information and / or drug identifier and send it to the associated user terminal.

[0074] Specifically, after receiving a record of no medication use, the associated user terminal can render and display the no-medication interface within the service program. The no-medication interface may include the medication identifier, the reason for no medication use, and / or the time for the next medication reminder.

[0075] In the process of activating the service procedure after a medication non-use reminder, the first optional implementation provided in this embodiment performs the following operations: Based on the level of non-medication use, a medication notification is generated and displayed using the access link. Once the medication notification is triggered, the service program is started based on the access link.

[0076] The "no medication" level can be the first "no medication" level, and the medication notification can be a system notification, such as FCM (Firebase Cloud Messaging) cloud messaging.

[0077] In the process of activating the service procedure after the unused medication reminder, the second optional implementation provided in this embodiment performs the following operations: The system renders and displays a medication pop-up window containing an access link according to the level of non-medication, and calls a vibration sensor to vibrate. After detecting that the medication pop-up window has been triggered, the service program is started based on the access link.

[0078] The level of non-medication here can be the second level of non-medication, which is higher than the first level of non-medication.

[0079] In addition, in the second optional implementation provided in this embodiment, during the process of providing feedback to the associated user on non-medication according to the non-medication level corresponding to the frequency range of non-medication, the following operations are performed: The system synchronizes the level of non-medication status with associated user terminals, and sends a call request to the user terminal through the signaling server based on the call instruction submitted by the associated user. Based on the call request, a voice call is established between the associated user terminal and the user terminal.

[0080] The level of no medication use here can be the third level of no medication use, which can be higher than the second level of no medication use and higher than the first level of no medication use. In the process of establishing a voice call between user terminals based on a call request, a voice call between user terminals can be established based on a call request and using the native WebRTC (Web Real-Time Communication) protocol.

[0081] Specifically, after the signaling server sends a call request to the user terminal, the user terminal can render a full-screen call interface based on the call request. After detecting the user terminal's answer instruction, the signaling server can establish a voice channel between the user terminal and the associated user terminal. If the voice channel fails to be established, the associated user terminal can conduct a voice call through the basic voice channel of the cellular network.

[0082] It should be noted that the user data obtained in this specification, such as voiceprint features, is authorized by the user and does not involve user privacy. In this embodiment, data can be encrypted before data transmission between the server and the user terminal or associated user terminal. For example, encrypted data can be obtained by encrypting data using AES-GCM-256 (Advanced Encryption Standard - Galois / Counter Mode - 256-bit, a security algorithm combination that combines encryption and authentication functions).

[0083] It should be added that each optional implementation method and each feasible execution method in steps S202 to S208 provided in this embodiment can be executed independently as needed, or they can be combined and referenced with each other. At the same time, each specific execution step in each optional implementation method or each feasible execution method can also be executed independently or combined as needed. The execution conditions of "if" or "under what circumstances" involved in each step or operation can be directly deleted, and subsequent operations can be executed. This embodiment does not make specific limitations on this.

[0084] It should also be added that, depending on the actual application scenario, step S202 and any of the subsequent steps S204 to S208 can be deleted, or any feature in any step can be deleted. For example, the "pre-medication" part of step S202 can be deleted, and the execution order of steps S202 to S208 can also be arbitrary.

[0085] The above-described detection and processing method for drug use can be executed by a server. The following method embodiment provides another method for drug use that can be executed by a user terminal. The two methods can cooperate with each other during execution. Therefore, when reading the above implementation process, you can refer to the corresponding content of the following other method embodiment for drug use. Similarly, when reading the following other method embodiment for drug use, you can also refer to the corresponding content of the above method embodiment.

[0086] The following description uses the application of a drug use detection and processing method provided in this embodiment in a drug use scenario as an example to further illustrate the drug use detection and processing method provided in this embodiment. (See also...) Figure 8The detection and processing method for drug use in drug use scenarios can be applied to servers and specifically includes the following steps.

[0087] Step S802: Obtain the drug image uploaded by the user terminal, which is obtained by image acquisition of the drug before administration.

[0088] Step S804: Based on the drug image, identify the drug category to obtain the drug category, and extract the visual features of the drug image according to the feature extraction method corresponding to the drug category to obtain the drug visual features.

[0089] Step S806: Based on the visual features of the drug, perform matching processing between the drug and the task drugs included in each medication task. If the matching is successful, return the matching success result to the user terminal.

[0090] Optionally, each medication task may include at least one medication task pre-configured to the user.

[0091] Step S808: Obtain the user's medication status information for the drug uploaded by the user terminal.

[0092] Optionally, medication status information can be obtained by identifying the medication status based on a sequence of rotated images of the medication after administration.

[0093] Step S810: If the medication status information includes non-medication status information with reasons for non-medication, update the number of times the user has not used the medication based on the non-medication status information.

[0094] Step S812: Provide feedback on non-medication to the associated user according to the non-medication level corresponding to the frequency range in which the non-medication frequency falls.

[0095] It should be noted that any one or more steps from S802 to S812 can be replaced by the corresponding technical means provided by steps S202 to S208 as needed for implementation and deployment. Any one or more steps from S802 to S812 can also be combined into a new implementation method as needed for implementation and deployment. Furthermore, any one or more steps from S802 to S812 can also be combined with one or more steps provided by steps S202 to S208 to form a new implementation method, or combined with one or more optional implementation methods provided by steps S202 to S208 to form a new implementation method, as needed for actual deployment. These will not be elaborated on here.

[0096] One or more embodiments of another detection and processing method for pharmaceuticals provided in this specification are as follows: Reference Figure 9The detection and processing method for drugs provided in this embodiment can be applied to user terminals, specifically including steps S902 to S908.

[0097] Step S902: Before medication, image acquisition is performed on the drug to obtain drug images and upload them to the server for medication detection based on the drug images and at least one pre-configured medication task.

[0098] The medicine described in this embodiment can be any type of medicine, such as capsule medicine, tablet medicine, granule medicine and / or liquid medicine; the medicine image may include an image of the medicine itself, an image of the medicine packaging and / or an image of the medicine packaging carrier, and the image of the medicine packaging carrier may include an image of the medicine box and / or an image of the medicine bottle.

[0099] In practice, images of the medication are acquired before administration and uploaded to the server. The server can perform medication detection based on the medication images and at least one pre-configured medication task. The medication to be administered before administration can be the medication that the user is using.

[0100] Specifically, users can submit drug identification commands through the service program. The user terminal can then render and display a drug identification interface based on these commands. This interface captures an image of the drug before administration and uploads it to the server. The server can then retrieve the drug image. For example... Figure 3 The drug recognition page shows that the user holds the capsule and the medicine box for image capture. During the image capture process, the drug recognition page displays an image capture reminder, which reads "Camera is on, recognizing medicine. Please show a picture of a hand holding the medicine or a real picture of the medicine." The user then performs image capture based on the image capture reminder.

[0101] The service program can be an application, subprogram (mini-program), or web application that provides medical services; that is, the service program can be a medical service program, a medical application, a medical subprogram, or a medical web application. The user can be a user who has a need for medication, and the user can specifically be a patient. The associated user can be a user's related user, specifically a user who has a relationship with the user, such as a relative, friend, or colleague of the user. In addition, the user can be a patient, and the associated user can be a doctor who provides medical consultation or diagnosis to the user, or a doctor assigned to the user by the service program.

[0102] In addition, during the process of acquiring images of the medicine before administration, the user terminal can also acquire images of the medicine being held in the hand and / or the medicine packaging carrier before administration; or it can acquire images of the medicine being held in the hand and / or the medicine packaging carrier before administration; among these, the medicine image can also be an image of the hand holding the medicine and / or an image of the hand holding the medicine packaging carrier.

[0103] It should be noted that the above-mentioned operation of acquiring images of the drug before administration and uploading them to the server, so as to perform drug detection based on the drug images and at least one pre-configured drug administration task, can be replaced by acquiring images of the drug before administration and uploading them to the server, and forming a new implementation method with other processing steps provided in this embodiment.

[0104] Step S904: If the drug test is passed, the drug after administration is rotated to obtain a rotational image sequence of the drug.

[0105] The rotational image sequence of the drug can be a rotational image sequence of the drug itself and / or a rotational image sequence of the drug packaging. The drug packaging can be a carrier for packaging the drug, such as a medicine box, medicine bottle, and / or a drug aluminum-plastic blister pack.

[0106] In practical implementation, if the medication test passes, a rotational image sequence of the medication can be obtained by rotating the medication after administration. To reduce the power consumption of the user terminal and avoid invalid image acquisition, the rotational image sequence can optionally be acquired when the user's hand contact area on the user terminal is greater than a preset contact area. The hand contact area is predicted based on the capacitance change data of the user terminal's capacitance sensor. That is, the user can perform rotational image acquisition of the medication after administration, meaning the user can rotate the medication and / or medication packaging after administration. The user terminal can call an image acquisition component, such as a camera, to acquire a rotational image sequence of the medication and / or medication packaging. The rotational image sequence of the medication after administration can be acquired when the user's hand contact area on the user terminal is greater than a preset contact area. The capacitance change data here can be the amount of capacitance change.

[0107] Specifically, the user terminal can acquire a sequence of rotating images of the medication after administration, either clockwise or counterclockwise; for example... Figure 6 The illustration shows a clockwise rotation of a medication. After administration, the user rotates the medication clockwise, and the user terminal captures a sequence of images showing this clockwise rotation. For example... Figure 7The diagram shown illustrates counter-clockwise rotation. After administration, the user rotates the medication counter-clockwise, and the user terminal captures a sequence of rotating images of the medication.

[0108] Step S906: Obtain the user's medication status information for the drug by performing medication status recognition based on the rotated image sequence.

[0109] The medication status information refers to the user's medication status information for the drug. The medication status information may include non-medication status information with a reason for not using the drug or medication status information. The rotating image sequence of the drug may be a rotating image sequence of the drug itself and / or a rotating image sequence of the drug packaging. The drug packaging may be a carrier for packaging the drug, such as a medicine box, medicine bottle and / or a drug aluminum-plastic blister pack.

[0110] In practical implementation, medication status recognition can be performed based on the rotated image sequence to obtain the user's medication status information. Specifically, after obtaining the rotated image sequence of the medication after administration, medication status recognition can be performed on the user, allowing the user to easily mark the medication status by rotating the medication, thus improving the flexibility and convenience of medication status marking. During the process of medication status recognition based on the rotated image sequence of the medication after administration, medication status recognition can be performed based on the medication rotation parameters calculated based on the rotated image sequence to obtain the user's medication status information. The user terminal can specifically use the following methods to perform medication status recognition: The rotation parameters of the drug are obtained by calculating the rotation parameters of the drug based on the rotated image sequence; Medication status information of users can be obtained by identifying medication status based on drug rotation parameters.

[0111] The drug rotation parameters may include the drug rotation angle and / or the drug rotation direction.

[0112] To ensure user privacy and security, the rotation parameters of the medicine can be calculated on the user terminal. Optionally, the rotation parameters of the medicine can be calculated based on the rotated image sequence and executed on a lightweight rotation recognition model deployed on the user terminal. The input of the lightweight rotation recognition model includes the rotated image sequence, and the output of the rotation recognition model includes the rotation parameters of the medicine. The model framework of the lightweight rotation recognition model can be TensorFlow Lite (a lightweight machine learning inference framework) running on an embedded MCU.

[0113] In the process of calculating the rotation parameters of the drug based on the rotated image sequence, to improve the accuracy and comprehensiveness of subsequent medication status recognition from multiple dimensions, the user terminal can perform the following operations: Based on the drug feature points in each rotated image in the rotated image sequence, the rotation direction and rotation angle of the drug are identified to obtain the drug rotation direction and rotation angle.

[0114] Among them, drug feature points refer to key points in the rotated image.

[0115] Specifically, the rotation direction and / or rotation angle of the drug can be identified by the movement trajectory of the same feature points in each rotated image in the rotation image sequence, thereby obtaining the drug rotation direction and / or drug rotation angle.

[0116] Accordingly, in the process of obtaining the user's medication status information based on the drug rotation parameters, the user terminal can perform the following operations: If the drug is rotated clockwise and the rotation angle is greater than the preset angle, the medication status information is determined to be "medication has been used".

[0117] In addition, the operation of determining the medication status information as "medication used" if the drug rotates clockwise and the rotation angle is greater than the preset angle can be replaced by determining the medication status information as "medication used" if the drug rotates counterclockwise and the rotation angle is greater than the preset angle, or it can be replaced by determining the medication status information as "medication used" if the drug rotates clockwise or counterclockwise and / or the rotation angle is greater than the preset angle and / or the drug rotation duration is greater than or equal to the preset duration.

[0118] Furthermore, in real-world applications, users may not have taken their medication. To address this, and in order to comprehensively understand the actual situation regarding medication non-use and improve the success rate of subsequent medication reminders, non-medication status information can be generated based on the reasons selected by the user in the list of reasons for non-medication. During the process of obtaining the user's medication status information based on drug rotation parameters, the user terminal can also perform the following operations: If the drug rotation direction is counterclockwise and the drug rotation angle is greater than the angle threshold, or the drug rotation duration is less than the preset duration, render and display the unused drug interface containing a list of reasons for not using the drug; The system generates non-medication status information based on the reasons for non-medication selected by the user in the list of reasons for non-medication.

[0119] The operation described above, which renders and displays an unused medication interface containing a list of reasons for not using medication, if the medication is rotated counterclockwise and the rotation angle is greater than an angle threshold, or if the rotation duration is less than a preset duration, can be replaced by rendering and displaying an unused medication interface containing a list of reasons for not using medication, if the medication is rotated clockwise or counterclockwise and / or the rotation angle is greater than an angle threshold and / or the rotation duration is less than a preset duration; the unused medication interface may also include the reminder time for the next medication reminder.

[0120] Step S908: Upload medication status information to the server so as to provide medication feedback to the associated user based on the medication status information and the corresponding medication feedback level.

[0121] The above-mentioned medication status recognition based on the rotated image sequence obtains the user's medication status information. In this step, the medication status information is uploaded to the server. The server can process medication feedback to the associated user based on the medication status information and the corresponding medication feedback level. The medication feedback level refers to the level at which medication feedback is given to the associated user. The medication feedback level can be a level that represents the urgency of the medication feedback.

[0122] It should be noted that the above-described embodiments of the detection and processing method for drug use applied to the server have already described in detail. The specific implementation process of the server providing drug use feedback to the associated user based on the drug use status information and the corresponding drug use feedback level can be referred to, and will not be repeated here.

[0123] The following description uses the application of a drug use detection and processing method provided in this embodiment in a drug use scenario as an example to further illustrate the drug use detection and processing method provided in this embodiment. It can be applied to the user's user terminal and specifically includes the following steps.

[0124] Step 1: Before medication, acquire images of the medication and upload them to the server.

[0125] The server can identify drug categories based on drug images, extract visual features from drug images according to the feature extraction method corresponding to drug categories, and match the drugs with the drugs included in each medication task based on the drug visual features. If the match is successful, the server returns a successful match result to the user terminal.

[0126] Step 2: Receive the successful matching result returned by the server, and acquire a rotational image sequence of the medicine after administration.

[0127] Step 3: Obtain the user's medication status information based on the rotating image sequence by performing medication status recognition.

[0128] Step 4: Upload the medication status information to the server so that medication feedback can be processed for the associated user based on the medication status information and the corresponding medication feedback level.

[0129] The drug use detection and processing method embodiment applicable to user terminals and drug use scenarios described here can be executed in conjunction with the drug use detection and processing method embodiment applicable to servers and drug use scenarios described above. Reading this embodiment can refer to the above embodiment, and reading the above embodiment can refer to this embodiment.

[0130] The following is an example of a drug detection and processing device provided in this manual: In the above embodiments, a method for detecting and processing drug use is provided, and correspondingly, a device for detecting and processing drug use is also provided, which will be described below with reference to the accompanying drawings.

[0131] Reference Figure 10 This illustration shows a schematic diagram of an embodiment of a drug use detection and processing device provided in this embodiment.

[0132] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0133] This embodiment provides a detection and processing device for drug use, including: Image acquisition module 1002 is configured to acquire drug images obtained by image acquisition of the drug before administration; The medication detection module 1004 is configured to perform medication detection based on the drug image and at least one pre-configured medication task; The information acquisition module 1006 is configured to acquire the user's medication status information for the drug uploaded by the user terminal if the medication test is passed; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after medication. The medication feedback module 1008 is configured to provide medication feedback to associated users based on the medication status information and the corresponding medication feedback level.

[0134] Another example of a detection and processing device for pharmaceuticals provided in this instruction manual is as follows: In the above embodiments, another detection and processing method for drug use is provided, and correspondingly, another detection and processing device for drug use is also provided, which will be described below with reference to the accompanying drawings.

[0135] Reference Figure 11This illustration shows a schematic diagram of an embodiment of a drug use detection and processing device provided in this embodiment.

[0136] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0137] This embodiment provides a detection and processing device for drug use, including: The image upload module 1102 is configured to acquire images of the drug before administration and upload them to the server, so as to perform drug administration detection based on the drug images and at least one pre-configured drug administration task. The image acquisition module 1104 is configured to acquire a rotational image sequence of the drug after administration if the drug administration test is passed. The status recognition module 1106 is configured to perform medication status recognition based on the rotated image sequence to obtain the user's medication status information for the drug; The information upload module 1108 is configured to upload the medication status information to the server so as to provide medication feedback to the associated user based on the medication status information and the corresponding medication feedback level.

[0138] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0139] The following is an example of a drug testing and processing device provided in this manual: Corresponding to the above-described method for detecting and processing a drug, based on the same technical concept, one or more embodiments of this specification also provide a drug detection and processing device for performing the above-described method for detecting and processing a drug. Figure 12 This is a schematic diagram of the structure of a detection and processing device for pharmaceutical use provided in one or more embodiments of this specification.

[0140] This embodiment provides a detection and processing device for pharmaceutical use, comprising: like Figure 12As shown, device 1200 mainly consists of a communication interface 1202, a user interface 1204, a processor 1206, and a data storage 1208. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1210. The communication interface 1202 enables device 1200 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1202 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1202 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1202 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1202 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces. The user interface 1204 includes receiving user input and providing output to the user. Therefore, user interface 1204 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1204 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1204 may include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, device 1200 may support remote access from other devices via communication interface 1202 or another physical interface (not shown). User interface 1204 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1204 may also be configured as a display device for rendering or displaying text fragments.

[0141] Processor 1206 may include one or more general-purpose processors and / or dedicated processors. Data storage 1208 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1206. Data storage 1208 may include removable and non-removable components.

[0142] Processor 1206 is capable of executing program instructions 1218 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 1208 to perform the various functions described herein. Data storage 1208 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1200, enable device 1200 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1218 by processor 1206 may result in processor 1206 using data 1212. For example, program instructions 1218 may include an operating system 1222 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1200 and one or more application programs 1220 (e.g., a browser, social application, or game application). Similarly, data 1212 may include operating system data 1216 and application data 1214. Operating system data 1216 is primarily accessible to operating system 1222, while application data 1214 is primarily accessible to one or more application programs 1220. Application data 1214 may reside in a file system visible or hidden from the user of device 1200. Application 1220 may communicate with operating system 1222 via one or more application programming interfaces (APIs). These APIs facilitate application 1220 reading and / or writing application data 1214, transmitting or receiving information via communication interface 1202, receiving or displaying information on user interface 1204, etc. In some terms, application 1220 may be simply referred to as an "app". Furthermore, application 1220 may be downloaded to device 1200 through one or more online app stores or app markets. However, applications may also be installed on device 1200 in other ways, such as through a web browser or a physical interface on device 1200 (e.g., a USB port).

[0143] In one specific embodiment, the detection and processing device for pharmaceuticals includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the detection and processing device for pharmaceuticals, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Acquire images of the medication obtained through image acquisition before administration; Drug detection is performed based on the drug image and at least one pre-configured drug administration task; If the medication test is passed, the user's medication status information for the drug uploaded by the user terminal is obtained; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after administration. Based on the medication status information and the corresponding medication feedback level, medication feedback is processed to the associated users.

[0144] Another example of a detection and processing device for pharmaceuticals provided in this instruction manual is as follows: Corresponding to the other drug detection and processing method described above, based on the same technical concept, one or more embodiments of this specification also provide another drug detection and processing device for performing the other drug detection and processing method provided above. Figure 13 This is a schematic diagram of the structure of a detection and processing device for pharmaceutical use provided in one or more embodiments of this specification.

[0145] This embodiment provides a detection and processing device for pharmaceutical use, comprising: like Figure 13As shown, device 1300 mainly consists of a communication interface 1302, a user interface 1304, a processor 1306, and a data storage 1308. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1310. The communication interface 1302 enables device 1300 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1302 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1302 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1302 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1302 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces. The user interface 1304 includes receiving user input and providing output to the user. Therefore, user interface 1304 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1304 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1304 may include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, device 1300 may support remote access from other devices via communication interface 1302 or another physical interface (not shown). User interface 1304 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1304 may also be configured as a display device for rendering or displaying text fragments.

[0146] Processor 1306 may include one or more general-purpose processors and / or special-purpose processors. Data storage 1308 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1306. Data storage 1308 may include removable and non-removable components.

[0147] Processor 1306 is capable of executing program instructions 1318 (e.g., compiled or uncompiled program logic and / or machine code) stored in data store 1308 to perform the various functions described herein. Data store 1308 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1300, enable device 1300 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1318 by processor 1306 may result in processor 1306 using data 1312. For example, program instructions 1318 may include an operating system 1322 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1300 and one or more application programs 1320 (e.g., a browser, social application, or game application). Similarly, data 1312 may include operating system data 1316 and application data 1314. Operating system data 1316 is primarily accessible to operating system 1322, while application data 1314 is primarily accessible to one or more application programs 1320. Application data 1314 may reside in a file system visible or hidden to the user of device 1300. Application 1320 may communicate with operating system 1322 via one or more application programming interfaces (APIs). These APIs facilitate application 1320 reading and / or writing application data 1314, transmitting or receiving information via communication interface 1302, receiving or displaying information on user interface 1204, etc. In some terms, application 1320 may be simply referred to as an "app". Furthermore, application 1320 may be downloaded to device 1300 through one or more online app stores or app markets. However, applications may also be installed on device 1300 in other ways, such as through a web browser or a physical interface on device 1300 (e.g., a USB port).

[0148] In one specific embodiment, the detection and processing device for pharmaceuticals includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the detection and processing device for pharmaceuticals, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Before medication, images of the medication are acquired and uploaded to a server to perform medication detection based on the medication images and at least one pre-configured medication task. If the medication test is passed, a rotational image sequence of the medication is obtained by acquiring rotational images of the medication after administration. Medication status information of the user for the drug is obtained by recognizing the medication status based on the rotated image sequence; The medication status information is uploaded to the server so that medication feedback can be processed for associated users based on the medication status information and the corresponding medication feedback level.

[0149] This specification provides an embodiment of a computer-readable storage medium as follows: Corresponding to the detection and processing method for a drug described above, based on the same technical concept, one or more embodiments of this specification also provide a computer-readable storage medium.

[0150] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed, perform the following steps: Acquire images of the medication obtained through image acquisition before administration; Drug detection is performed based on the drug image and at least one pre-configured drug administration task; If the medication test is passed, the user's medication status information for the drug uploaded by the user terminal is obtained; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after administration. Based on the medication status information and the corresponding medication feedback level, medication feedback is processed to the associated users.

[0151] It should be noted that the embodiments of a computer-readable storage medium described in this specification and the embodiments of a detection and processing method for the use of a drug described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0152] Another embodiment of a computer-readable storage medium provided in this specification is as follows: In response to another detection and processing method for a drug described above, based on the same technical concept, one or more embodiments of this specification also provide another computer-readable storage medium.

[0153] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed, perform the following steps: Before medication, images of the medication are acquired and uploaded to a server to perform medication detection based on the medication images and at least one pre-configured medication task. If the medication test is passed, a rotational image sequence of the medication is obtained by acquiring rotational images of the medication after administration. Medication status information of the user for the drug is obtained by recognizing the medication status based on the rotated image sequence; The medication status information is uploaded to the server so that medication feedback can be processed for associated users based on the medication status information and the corresponding medication feedback level.

[0154] It should be noted that the embodiments of another computer-readable storage medium described in this specification and the embodiments of another drug detection and processing method described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0155] This specification provides an example of a computer program product as follows: Corresponding to the detection and processing method for a drug described above, based on the same technical concept, one or more embodiments of this specification also provide a computer program product.

[0156] A computer program product includes a computer program / instructions that, when executed by a processor, perform the following steps: Acquire images of the medication obtained through image acquisition before administration; Drug detection is performed based on the drug image and at least one pre-configured drug administration task; If the medication test is passed, the user's medication status information for the drug uploaded by the user terminal is obtained; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after administration. Based on the medication status information and the corresponding medication feedback level, medication feedback is processed to the associated users.

[0157] It should be noted that the embodiments of a computer program product described in this specification and the embodiments of a detection and processing method for a drug described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0158] Another example of a computer program product provided in this specification is as follows: In response to another detection and processing method for a drug described above, based on the same technical concept, one or more embodiments of this specification also provide another computer program product.

[0159] A computer program product includes a computer program / instructions that, when executed by a processor, perform the following steps: Before medication, images of the medication are acquired and uploaded to a server to perform medication detection based on the medication images and at least one pre-configured medication task. If the medication test is passed, a rotational image sequence of the medication is obtained by acquiring rotational images of the medication after administration. Medication status information of the user for the drug is obtained by recognizing the medication status based on the rotated image sequence; The medication status information is uploaded to the server so that medication feedback can be processed for associated users based on the medication status information and the corresponding medication feedback level.

[0160] It should be noted that the embodiments of another computer program product described in this specification and the embodiments of another detection and processing method for a drug described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0161] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments. For example, the device embodiment, equipment embodiment and computer-readable storage medium embodiment are all similar to the method embodiment, so the description is relatively simple. When reading the relevant content of the device embodiment, equipment embodiment and computer-readable storage medium embodiment, please refer to the description of the method embodiment.

[0162] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

[0163] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0164] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0165] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0166] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0167] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0168] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0169] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable test processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable test processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable test processing equipment to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable test processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0175] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of features includes not only those features but also other features not expressly listed, or features inherent to such process, method, article, or apparatus. Without further limitations, a feature defined by the phrase "comprising one..." does not exclude the presence of other identical features in the process, method, article, or apparatus that includes said feature.

[0177] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0178] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0179] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A method for detecting and processing a drug, comprising: Acquire images of the medication obtained through image acquisition before administration; Drug detection is performed based on the drug image and at least one pre-configured drug administration task; If the medication test is passed, the user's medication status information for the drug uploaded by the user terminal is obtained; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after administration. Based on the medication status information and the corresponding medication feedback level, medication feedback is processed to the associated users.

2. The detection and processing method for drug use according to claim 1, wherein the drug use status information includes non-drug use status information carrying the reason for non-drug use; The step of providing medication feedback to associated users based on the medication status information and the corresponding medication feedback level includes: The user's number of times the drug has not been used is updated based on the information regarding the non-use status; Feedback on non-medication is sent to the associated user based on the non-medication level corresponding to the interval in which the number of non-medication instances fall.

3. The drug use detection and processing method according to claim 2, wherein the step of providing feedback on non-drug use to the associated user according to the non-drug use level corresponding to the interval in which the number of non-drug uses fall, includes: The system pushes an access link and the level of non-medication status to the associated user terminal through the system push channel, so as to start the service program after the non-medication reminder is issued to the associated user terminal; Based on the medication access request sent by the associated user terminal after starting the service program, a non-medication record containing the non-medication status information and the drug identifier is generated and sent to the associated user terminal.

4. The method for detecting and processing drug use according to claim 3, wherein activating the service procedure after issuing the unused medication reminder includes: Based on the non-medication level, a medication notification is generated and displayed using the access link. After the medication notification is triggered, the service program is started based on the access link. or, The system renders and displays a medication pop-up containing the access link according to the non-medication level, and calls a vibration sensor to vibrate. After detecting that the medication pop-up has been triggered, the service program is started based on the access link.

5. The method for detecting and processing drug use according to claim 2, wherein providing feedback on drug use to the associated user based on the drug use level corresponding to the interval in which the number of times the drug use occurred includes: The level of non-medication status is synchronized with the associated user terminal, so that a call request is sent to the user terminal through the signaling server based on the call instruction submitted by the associated user, and a voice call is established between the associated user terminal and the user terminal based on the call request.

6. The drug use detection and processing method according to claim 1, wherein the drug use detection based on the drug image and at least one pre-configured drug use task includes: Visual features of the drug are extracted from the drug image. Based on the visual features of the drug, the drug is matched with the task drugs included in each medication task. If the match is successful, the medication test is considered passed.

7. The drug use detection and processing method according to claim 6, wherein the step of matching the drug with the task drugs included in each drug use task based on the drug's visual features includes: The similarity is obtained by calculating the similarity between the drug color features and the benchmark drug color features in the user's drug color feature database; The drug color feature library is constructed based on the drug color features of the drugs included in each of the user's medication tasks. The matching degree is calculated based on the drug's outline features and the drug's baseline drug outline features. A matching index is calculated based on the similarity and the matching degree. If the matching index is greater than the index threshold, the matching is determined to be successful.

8. The drug use detection and processing method according to claim 6, after the step of matching the drug with the task drugs included in each medication task based on the drug's visual features is performed, it further includes: If the matching fails, standard drug identification is obtained by standardizing the drug's textual features. The text features of the drug are obtained by extracting text features from the drug packaging image; Based on the standard drug identifier, the system queries the drug database for structured information containing drug dosage, medication instructions, and / or standard packaging diagrams, and returns a matching failure result containing the structured information to the user terminal.

9. The detection and processing method for pharmaceutical use according to claim 8, further comprising: The drug image is compressed and then synchronized to the associated user. After the associated user reviews the medication based on the compressed drug image, the associated user terminal generates a medication error reminder based on the associated user's medication error instruction and sends it to the server. Send a medication error voice message generated based on the voiceprint characteristics of the associated user to the user terminal; generate a medication error warning based on the medication misuse prompt and the contact information of the associated user and send it to the user terminal.

10. The detection and processing method for drug use according to claim 1, wherein the rotating image sequence is acquired when the contact area of ​​the user's hand on the user terminal is greater than a preset contact area; The hand contact area is predicted based on the capacitance change data of the capacitance sensor of the user terminal.

11. The detection and processing method for drug use according to claim 10, wherein the step of identifying the drug use status based on the rotated image sequence of the drug after administration includes: The rotation parameters of the drug are obtained by calculating the rotation parameters of the drug based on the rotated image sequence; Based on the drug rotation parameters, medication status identification is performed to obtain the user's medication status information for the drug.

12. The detection and processing method for drug use according to claim 11, wherein the operation of calculating the rotation parameters of the drug based on the rotation image sequence is performed based on a lightweight rotation recognition model deployed on the user terminal; The input to the lightweight rotation recognition model includes the rotated image sequence, and the output of the rotation recognition model includes the drug rotation parameters.

13. The detection and processing method for pharmaceuticals according to claim 11, wherein the step of calculating the rotation parameters of the pharmaceuticals based on the rotated image sequence to obtain the rotation parameters of the pharmaceuticals includes: Based on the drug feature points in each rotated image in the rotated image sequence, the rotation direction and rotation angle of the drug are identified to obtain the rotation direction and rotation angle of the drug. The step of obtaining the user's medication status information for the drug based on the drug rotation parameters includes: If the drug rotates clockwise and the rotation angle is greater than a preset angle, the medication status information is determined to be "medication already used".

14. The method for detecting and processing drug use according to claim 13, wherein the step of providing drug use feedback to the associated user based on the drug use status information and the corresponding drug use feedback level includes: The system pushes an access link to the associated user terminal to start the service program; Based on the medication access request submitted after the associated user terminal starts the service program, the medication record generated based on the medication status information is synchronized to the associated user terminal.

15. The drug use detection and processing method according to claim 13, wherein the step of obtaining the user's drug use status information based on the drug rotation parameters for drug use status identification further includes: If the rotation direction of the drug is counterclockwise and the rotation angle of the drug is greater than the angle threshold, or the rotation duration of the drug is less than the preset duration, render and display the unused drug interface containing a list of reasons for not using the drug; Medication non-use status information is generated based on the reason for non-use selected by the user in the list of reasons for non-use.

16. The detection and processing method for drug use according to claim 13, further comprising, before executing the operation of determining the drug use status information as drug use status information: The quantity of the drug before administration is identified based on the drug image, and the quantity of the drug after administration is identified based on the rotated image sequence. The dosage is calculated based on the quantity before and after medication. If the dosage matches the medication method in the medication task, the operation of determining the medication status information as the medication status information is executed.

17. A method for detecting and processing a drug, comprising: Before medication, images of the medication are acquired and uploaded to a server to perform medication detection based on the medication images and at least one pre-configured medication task. If the drug test is passed, a rotational image sequence of the drug is obtained by acquiring rotational images of the drug after administration. Medication status information of the user for the drug is obtained by recognizing the medication status based on the rotated image sequence; The medication status information is uploaded to the server so that medication feedback can be processed for associated users based on the medication status information and the corresponding medication feedback level.

18. A detection and processing device for pharmaceuticals, comprising: The image acquisition module is configured to acquire images of the drug obtained by image acquisition of the drug before administration. The medication detection module is configured to perform medication detection based on the drug image and at least one pre-configured medication task; The information acquisition module is configured to acquire the user's medication status information for the drug uploaded by the user terminal if the medication test is passed; the medication status information is obtained by identifying the medication status based on the rotation image sequence of the drug after medication. The medication feedback module is configured to provide medication feedback to associated users based on the medication status information and the corresponding medication feedback level.

19. A detection and processing device for pharmaceuticals, characterized in that, The device includes: A processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method as claimed in claim 1 or 17.

20. A computer-readable storage medium for storing computer-executable instructions that, when executed, implement the steps of the method of claim 1 or 17.

Images