A method for hierarchical recognition of a combination pill

Abstract

The embodiment of the present application discloses a kind of combination pill hierarchical identification method, comprising: obtaining the combination pill image to be identified;Identify the position information of each pill in the combination pill image;According to the position information of each pill, each pill in the combination pill image is separated and handled, and single pill image is obtained;The combination pill image is classified and identified, and the class attribution of the combination pill image in the combination pill image is obtained;The single pill image is extracted and handled, and the pill feature vector in the single pill image is obtained;According to the class attribution of combination pill and the pill feature vector in the single pill image, the feature representation of each pill is calculated;According to the feature representation of each pill, the classification result of each pill is confirmed.The accurate pill identification reduces the medication error caused by human error, so as to improve the safety of the whole drug delivery system and the safety of patient medication.

Description

Technical Field

[0001] This invention relates to the field of drug identification technology, and specifically to a method for classifying and identifying combination pills. Background Technology

[0002] The booming economy and increasing life expectancy have made the role of medicines in safeguarding human health increasingly prominent. In my country, the aging population and the continuous expansion of the pharmaceutical market have jointly driven the rapid development of the pharmaceutical industry. However, the accompanying issues of drug safety are also becoming increasingly prominent. These issues not only directly relate to patients' lives and health but also place more stringent demands on the stability of the drug dispensing system. In particular, with the modernization of the medical service environment and the continuous advancement of automation technology, traditional manual drug verification methods can no longer meet the dual demands of modern pharmacies for high efficiency and high accuracy.

[0003] The inpatient pharmacy is responsible for the dispensing, verification, and distribution of medications, including oral pills, for all inpatients in the hospital. Pharmacists must accurately dispense the prescribed dosages according to doctors' orders. Currently, most hospital inpatient pharmacies still use the traditional dispensing model: doctors write prescriptions, pharmacists print out the prescription slips and dispense the medications, and then another pharmacist or nurse verifies and dispenses the medications. However, this process has many problems. For example, pharmacists are prone to fatigue from repetitive visual verification work for extended periods, increasing the risk of dispensing incorrect medications. Furthermore, the pill verification process is inefficient; pharmacists must check each prescription and pill individually, a process that is not only time-consuming and labor-intensive but often only ensures the accuracy of the number of pills, not the correct type.

[0004] In the current context of relatively strained medical resources, this manual drug verification method not only increases the hospital's operating costs but also reduces the efficiency and safety of drug management. Summary of the Invention

[0005] To address the shortcomings of existing methods, this invention aims to develop an automated technology for the rapid and accurate identification of combination drugs, thereby improving the efficiency and accuracy of prescription drug verification in hospitals. This technology aims to accurately identify combination pills already dispensed to patients. Facing the challenges of pills exhibiting minimal differences in size, color, and shape, making subtle features difficult to discern, and the diverse types of pills and prescription combinations in pharmacies, which limits the accuracy of directly applying deep learning models for pill classification, this invention proposes a method for hierarchical identification of combination pills.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for classifying and identifying combination pills, comprising:

[0007] Acquire images of the combination pills to be identified;

[0008] The location information of each pill in the combined pill image is identified and obtained;

[0009] Based on the position information of each pill, each pill in the combined pill image is separated to obtain a single pill image;

[0010] The single pill image is subjected to feature extraction processing to obtain the pill feature vector in the single pill image;

[0011] The combined pill image is subjected to category recognition processing to obtain the category to which the combined pill in the combined pill image belongs;

[0012] Based on the category to which the combination pills belong and the feature vector of the pills in the single pill image, the feature representation of each pill is calculated.

[0013] The classification result of each pill is confirmed based on the characteristics of each pill.

[0014] Furthermore, the step of identifying and obtaining the position information of each pill in the combined pill image includes:

[0015] The combined pill image is input into a preset position detection model for target detection processing to obtain the position information of each pill in the combined pill image.

[0016] Furthermore, the training method for the location detection model includes:

[0017] Pre-built YOLOv5s object detection model;

[0018] The YOLOv5s object detection model is trained using a training dataset consisting of multiple training images containing combined pill images and their corresponding pill annotation location information to obtain the location detection model.

[0019] Furthermore, the training images are enhanced using Mosaic image enhancement technology to obtain more training images.

[0020] Furthermore, the combined pill image is subjected to category recognition processing to obtain the category to which the combined pill in the combined pill image belongs, including:

[0021] The combined pill image is input into a preset combined category recognition model for category recognition processing to obtain the category to which the combined pill in the combined pill image belongs.

[0022] Furthermore, the combined category recognition model is obtained by training a pre-built first ShuffleNet v2 model.

[0023] Further, feature extraction processing is performed on the single pill image to obtain the pill feature vector in the single pill image, including:

[0024] The single pill image is input into a preset feature extraction model for feature extraction processing to obtain the pill feature vector in the single pill image.

[0025] Furthermore, the feature extraction model is obtained by training a pre-built second ShuffleNet v2 model.

[0026] Furthermore, the step of calculating the feature representation of each pill based on the category to which the combined pills belong and the pill feature vector in the single pill image includes:

[0027] Generate a mask matrix based on the category to which the combination pills belong;

[0028] The mask matrix is ​​multiplied with the feature vector of the pill to obtain the feature representation of each pill.

[0029] Furthermore, the step of confirming the classification result of each pill based on the characteristic representation of each pill includes:

[0030] The softmax function is used to process the feature representation, calculate the probability distribution of each pill belonging to each category, and take the category with the highest probability as the final classification result.

[0031] The beneficial effects of this invention are as follows:

[0032] (1) Improved recognition accuracy: The adoption of a hierarchical label classification strategy significantly improved the accuracy of pill recognition. This strategy refines the recognition process in stages, enabling the model to more accurately identify the combination categories and specific types of pills.

[0033] (2) Enhanced target recognition focus: By introducing a mask matrix, the model can focus more on the recognition of target categories in the secondary classification stage, thereby further improving the focus of recognition.

[0034] (3) High adaptability: This method is particularly suitable for resource-constrained devices, such as small and medium-sized medical auxiliary devices, and has a low computational cost. This enables the invention to operate with limited hardware resources while maintaining high performance.

[0035] (4) Flexibility and scalability: The designed method allows the target detection network to be updated only when the types of pill combinations increase, without the need to retrain the target detection network. This provides convenience for future upgrades and expansions of the system.

[0036] (5) Improve medical safety: Accurate pill identification reduces medication errors caused by human mistakes, thereby improving the safety of the entire drug delivery system and the safety of patients taking medication.

[0037] (6) Optimize medical resources: The automated pill identification process reduces the burden on pharmacists in drug verification work, allowing them to focus on tasks that require more professional judgment. This invention helps optimize the allocation of medical resources, reduce hospital operating costs, and improve the efficiency and quality of pharmacy work. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0039] Figure 1 This is a flowchart illustrating the combination pill grading and identification method provided in this embodiment of the invention. Figure 1 ;

[0040] Figure 2 This is the framework structure of the combined pill classification and identification method provided in the embodiments of the present invention;

[0041] Figure 3 This is a flowchart illustrating the combination pill grading and identification method provided in this embodiment of the invention. Figure 2 ;

[0042] Figure 4 This is a schematic diagram of the structure of the YOLOv5s target detection model network provided in an embodiment of the present invention;

[0043] Figure 5 This is the training and construction process of the YOLOv5s object detection model provided in this embodiment of the invention;

[0044] Figure 6 This is a schematic diagram of the combined pill dataset provided in an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the operation flow of the Mosaic image enhancement technology provided in the embodiments of the present invention;

[0046] Figure 8 This is a graph showing the relationship between the training effect and the number of iterations of YOLOv5s in this embodiment of the invention.

[0047] Figure 9 This describes the test accuracy and loss of the Shufflenet v2 first-level classification model provided in this embodiment of the invention as a function of the number of iterations.

[0048] Figure 10This is a schematic diagram of the structure for building a second ShuffleNet v2 model provided in an embodiment of the present invention;

[0049] Figure 11 This is a schematic diagram of the single pill dataset provided in an embodiment of the present invention;

[0050] Figure 12 This is a schematic diagram showing the accuracy changes of the Shufflenet v2 secondary classification model provided in this embodiment of the invention during the training process;

[0051] Figure 13 This is a schematic diagram of the structure of a combined pill recognition system provided in an embodiment of the present invention. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0054] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, may be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used in this application, may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0055] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0056] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0057] It should be noted that step designations such as S1 and S2 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the protection scope of this application.

[0058] The technical solution of this application aims to accurately identify the name of each pill in a combination of pills dispensed to a patient. Facing the challenge of pills exhibiting minimal differences in size, color, and shape, making subtle features difficult to discern, and the diverse types of pills and prescription combinations in pharmacies, which limits the accuracy of directly applying object detection networks for classification, this embodiment proposes a method for hierarchical identification of combination pills. The specific experimental steps of this method are detailed in the appendix. Figure 1 , Figure 2 As shown.

[0059] Specifically, such as Figure 3 The flowchart shown is a method for classifying and identifying combination pills according to an embodiment of the present invention. The method includes the following steps:

[0060] Step S110: Obtain the image of the combination pill to be identified.

[0061] In this embodiment, an image of the combination pills to be identified is captured by a camera or other imaging device. The acquired image of the combination pills contains the combination pills to be identified. The number of combination pills can be one, two, or more, such as three, four, five, etc.; the types of combination pills can be one, two, or more, such as three, four, five, etc.; the number of each type of pill can also be one, two, or more, such as three, four, five, etc.

[0062] In this embodiment, all pills are oral solid dosage forms with the outer packaging removed.

[0063] Step S120: Identify and obtain the position information of each pill in the combined pill image.

[0064] Specifically, this includes: inputting the combined pill image into a preset position detection model for target detection processing to obtain the position information of each pill in the combined pill image.

[0065] The training method for the location detection model includes:

[0066] Pre-built YOLOv5s object detection model;

[0067] The YOLOv5s object detection model is trained using a training dataset consisting of multiple training images containing combined pill images and their corresponding pill annotation location information to obtain the location detection model.

[0068] Figure 4 This is a schematic diagram of the YOLOv5s object detection model network provided in this embodiment. YOLOv5s consists of four key parts: Input, Backbone, Neck, and Head.

[0069] Figure 5 This paper demonstrates the training and construction process of the YOLOv5s object detection model in this invention. In the pre-training phase, the model is trained using images of combined pills as the training dataset and their corresponding labeled location information (in txt files). In the testing phase, YOLOv5s performs accurate detection and localization of each pill in the combined pill images.

[0070] The dataset used in this embodiment includes:

[0071] Single pill image dataset: Contains 300 classes of single pill images, with at least 40 images per class, and the image resolution is uniformly 224×224 pixels.

[0072] Combination Pills Dataset: Covers 15 different combinations of 17 single pills, totaling 782 images.

[0073] Figure 6 This is a schematic diagram of the combined pill dataset provided in this embodiment. All pills used for combination are from 300 types of pills collected in the single pill image dataset, while ensuring consistency of background and shooting conditions with the combined pill dataset.

[0074] To ensure the accuracy and reliability of the training dataset, this embodiment has specifically established clear inclusion and exclusion criteria. The inclusion criteria include: solid dosage forms with their hard packaging removed (excluding granules and powders), covering various tablets (such as regular tablets, enteric-coated tablets, coated tablets, etc.), hard capsules, and soft capsules (or capsules). Simultaneously, it is ensured that each capsule has at least one distinguishing feature (color, shape, markings, etc.) so that the model can effectively differentiate them. The exclusion criteria exclude capsules with obvious defects due to damage, contamination, or other reasons, ensuring the image quality in the dataset and thereby improving the accuracy of model training and recognition.

[0075] Furthermore, when constructing the training dataset, Mosaic image enhancement techniques can be used to enhance the training images to obtain more training images. For example... Figure 7As shown, the specific implementation steps are described as follows: First, four images containing the combined pills are randomly selected, and a series of data augmentation operations are performed on these images, including adjusting brightness and color temperature, flipping, or scaling. Then, these four processed images are stitched together in the order from top left to bottom left and top right to bottom right to form a new image. Assuming the model's input size is s, a 2s×2s gray background image will be generated. On this background image, within the rectangular area defined by point M (located at s / 2, s / 2) and point N (located at 3s / 2, 3s / 2), an anchor point is randomly selected for the stitching operation, merging the selected portion of the four images with the gray background image. This constructs an augmented image for YOLOv5s model training, improving the model's generalization ability to different pill combinations.

[0076] Figure 8 This is a graph showing the relationship between the training performance of YOLOv5s and the number of iterations in this embodiment. YOLOv5s was used as the object detection network in the combined pill recognition model. Training parameters included: a total of 250 iterations, a batch size of 128 samples, weight updates using the Adam optimizer, an initial learning rate of 0.01, and a reduction of the learning rate to 20% of the initial value during training. Figure 8 It can be observed that as the number of iterations increases, the model's mAP (mean accuracy) on the test set steadily improves and eventually stabilizes at a relatively high level.

[0077] Step S130: Based on the position information of each pill, separate each pill in the combined pill image to obtain a single pill image.

[0078] In this embodiment, a single pill image can be obtained by separating each pill in the combined pill image based on the position information of each pill using a preset cropping module.

[0079] Step S140: Perform feature extraction processing on the single pill image to obtain the pill feature vector in the single pill image.

[0080] Specifically, this includes: inputting the single pill image into a preset feature extraction model for feature extraction processing to obtain the pill feature vector in the single pill image. The feature extraction model is a binary classification model.

[0081] In this embodiment, the feature extraction model is obtained by training a pre-built second ShuffleNet v2 model. Figure 10 This is a schematic diagram of the structure of the second ShuffleNet v2 model built in this embodiment.

[0082] Figure 11 This illustration shows a photographic representation of the single pill dataset provided in this embodiment. All collected pill data consists of oral solid dosage forms with the outer packaging removed.

[0083] In this embodiment, the feature extraction model is based on the lightweight ShuffleNet v2. ShuffleNet v2 was chosen as the basic architecture for the secondary classification model. During the training phase, the model is trained using a dataset containing images of 300 different types of pills, learning and capturing key features that can distinguish different types of pills, thereby obtaining an efficient pre-trained model.

[0084] In the testing phase, the model receives a single pill image cropped from a combined drug image and performs deep feature extraction through convolutional layers of ShuffleNet v2. Subsequently, fully connected layers of ShuffleNet v2 transform and map the extracted high-level features into a 300-dimensional feature vector space to obtain the pill feature vector in the single pill image.

[0085] Figure 12 This embodiment of the invention illustrates the accuracy changes of the ShuffleNet v2 secondary classification model during training, including the growth trends of training accuracy and test accuracy with the number of iterations. The training parameters are the same as those of the primary classification model. The accuracy on the training set steadily increases with the number of training epochs, saturating around 50 epochs, with an accuracy close to 1. The accuracy on the test set fluctuates initially, but stabilizes after 50 epochs of training, reaching above 0.9 after 75 epochs, indicating that the secondary classification model has good recognition capabilities for new data.

[0086] Step S150: Perform category recognition processing on the combined pill image to obtain the category to which the combined pill in the combined pill image belongs.

[0087] Specifically, this includes: inputting the combined pill image into a preset combined category recognition model for category recognition processing to obtain the category to which the combined pill in the combined pill image belongs.

[0088] In this embodiment, the combined category recognition model is obtained by training a pre-built first ShuffleNet v2 model, and the combined category recognition model serves as a first-level classification model.

[0089] The combination category recognition model performs primary classification of combination pills, assigning primary labels to them, such as distinguishing between different combination forms like "combination drug 1" and "combination drug 2". Pill images are then categorized into a smaller, defined category.

[0090] Figure 9 This embodiment of the invention illustrates the test accuracy and loss of the ShuffleNet v2 first-level classification model as a function of the number of iterations. Training parameters included: 200 iterations (Epochs), 128 samples per batch (batch size), the Adam optimizer, and an initial learning rate of 0.1. As shown in the figure, the model's accuracy gradually improves with training. After 25 epochs, the accuracy begins to stabilize, reaching a good level of over 85%. Continuing training up to 200 epochs, the model's accuracy further stabilizes at around 97%, while the loss value drops below 0.2, demonstrating the model's high learning efficiency and generalization ability on the training data.

[0091] Step S160: Calculate the feature representation of each pill based on the category to which the combined pills belong and the feature vector of the pills in the single pill image.

[0092] Specifically, this includes: generating a mask matrix based on the category to which the combined pills belong; and multiplying the mask matrix with the pill feature vector to obtain the feature representation of each pill.

[0093] The mask matrix has the same dimension as the output vector of the fully connected layer in the feature extraction model. The category to which the combination pill belongs is determined by the first-level classification of the combination pills using a combined category recognition model; correspondingly, the position of the target category is set to 1 in the mask matrix, and other positions are set to 0. For example... Figure 10 As shown, by multiplying the mask matrix with the pill feature vector, a feature representation focused on the target category can be obtained.

[0094] Step S170: Confirm the classification result of each pill based on the characteristic representation of each pill.

[0095] Specifically, the classifier uses the softmax function to process the feature representation, calculates the probability distribution of each pill belonging to each category, and takes the category with the highest probability as the final classification result to obtain the name of each pill.

[0096] Based on the same inventive concept, embodiments of the present invention provide a combined pill recognition system. For example... Figure 13As shown, the system may include one or more processors 101, one or more input devices 102, one or more output devices 103, and a memory 104. The processors 101, input devices 102, output devices 103, and memory 104 are interconnected via a bus 105. The memory 104 stores a computer program, which includes program instructions. The processor 101 is configured to invoke the program instructions to execute the method described in the embodiment of the combined pill grading identification method.

[0097] It should be understood that, in this embodiment of the invention, the processor 101 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0098] Input device 102 may include a keyboard, etc., and output device 103 may include a display (LCD, etc.), a speaker, etc.

[0099] The memory 104 may include read-only memory and random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store device type information.

[0100] In specific implementations, the processor 101, input device 102, and output device 103 described in the embodiments of the present invention can execute the implementation methods described in the embodiments of the combined pill classification and identification method provided in the present invention, which will not be repeated here.

[0101] It should be noted that the specific workflow of the combination pill identification system can be found in the aforementioned method implementation section, and will not be repeated here.

[0102] Furthermore, embodiments of the present invention also provide a readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, implement the above-described method for graded identification of combined pills.

[0103] The computer-readable storage medium can be an internal storage unit of the backend server described in the foregoing embodiments, such as the system's hard drive or memory. The computer-readable storage medium can also be an external storage device of the system, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the system. Further, the computer-readable storage medium may include both internal storage units and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the system. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0104] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0105] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for classifying and identifying combination pills, characterized in that, include: Acquire images of the combination pills to be identified; The location information of each pill in the combined pill image is identified and obtained; Based on the position information of each pill, each pill in the combined pill image is separated to obtain a single pill image; The single pill image is subjected to feature extraction processing to obtain the pill feature vector in the single pill image; The process of performing category recognition processing on the combined pill image to obtain the category to which the combined pill in the combined pill image belongs specifically includes: inputting the combined pill image into a preset combined category recognition model for category recognition processing to obtain the category to which the combined pill in the combined pill image belongs; Based on the category to which the combination pills belong and the feature vector of the pills in the image of a single pill, the feature representation of each pill is calculated; this includes: generating a mask matrix based on the category to which the combination pills belong; and multiplying the mask matrix with the feature vector of the pills to obtain the feature representation of each pill. The classification result of each pill is confirmed based on the characteristics of each pill.

2. The method for classifying and identifying combination pills as described in claim 1, characterized in that, The step of identifying and obtaining the position information of each pill in the combined pill image includes: The combined pill image is input into a preset position detection model for target detection processing to obtain the position information of each pill in the combined pill image.

3. The method for classifying and identifying combination pills as described in claim 2, characterized in that, The training method for the location detection model includes: Pre-built YOLOv5s object detection model; The YOLOv5s object detection model is trained using a training dataset consisting of multiple training images containing combined pill images and their corresponding pill annotation location information to obtain the location detection model.

4. The method for classifying and identifying combination pills as described in claim 3, characterized in that, The training images are enhanced using Mosaic image enhancement technology to obtain more training images.

5. The method for classifying and identifying combination pills as described in claim 1, characterized in that, The combined category recognition model is obtained by training a pre-built first ShuffleNet v2 model.

6. The method for classifying and identifying combination pills as described in claim 1, characterized in that, The single pill image is subjected to feature extraction processing to obtain the pill feature vector in the single pill image, including: The single pill image is input into a preset feature extraction model for feature extraction processing to obtain the pill feature vector in the single pill image.

7. The method for classifying and identifying combination pills as described in claim 6, characterized in that, The feature extraction model is obtained by training a pre-built second ShuffleNet v2 model.

8. The method for classifying and identifying combination pills as described in claim 1, characterized in that, The process of determining the classification result of each pill based on its characteristic representation includes: The softmax function is used to process the feature representation, calculate the probability distribution of each pill belonging to each category, and take the category with the highest probability as the final classification result.

Images