Combined pill grading identification method

Abstract

The embodiment of the invention discloses a combined pill grading identification method. The method comprises the following steps: acquiring a to-be-identified combined pill image; identifying and acquiring position information of each pill in the combined pill image; according to the position information of each pill, separating each pill in the combined pill image to obtain a single pill image; performing category identification processing on the combined pill image to obtain the category attribution of the combined pill in the combined pill image; performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image; according to the category affiliation of the combined pill and the pill feature vector in the single pill image, the feature representation of each pill is calculated; and determining a classification result of each pill according to the feature representation of each pill. The accurate pill identification reduces the medication errors caused by human errors, thereby improving the safety of the whole medicine dispensing system and the medication safety of the patient.

Description

Technical Field

[0001] The invention relates to the technical field of drug identification, and in particular to a combined pill classification identification method. Background Art

[0002] The booming economy and the extension of human life span have made the role of drugs in protecting human health more prominent. In my country, the aging of the population and the continuous expansion of the pharmaceutical market have jointly promoted the rapid development of the pharmaceutical industry. However, the safety issues of drug use have also become increasingly prominent. These issues are not only directly related to the life and health of patients, but also put forward more stringent requirements on the stability of the drug distribution system. In particular, with the modernization of the medical service environment and the continuous advancement of automation technology, the traditional manual drug verification method can no longer meet the dual needs of modern pharmacies for high efficiency and high precision.

[0003] The inpatient pharmacy is responsible for the preparation, verification and distribution of medicines, including oral pills, for all inpatients in the hospital. Pharmacists need to accurately and correctly distribute the prescribed drug doses to patients according to the doctor's orders. At present, most hospital inpatient pharmacies still use the traditional drug placement model, that is, the doctor prescribes the doctor's order, the pharmacist prints the doctor's order sheet and dispenses the medicine, and then another pharmacist or nurse verifies and dispenses the medicine. However, there are many problems with this process. For example, pharmacists are prone to fatigue due to long-term repetitive visual verification work, thereby increasing the risk of mis-prescribing medicines. In addition, the efficiency of the pill verification process is not high. Pharmacists need to check prescriptions and pills one by one. This process is not only time-consuming and labor-intensive, but often can only ensure the accuracy of the number of pills, and it is difficult to completely ensure the accuracy of the type of pills.

[0004] At a time when medical resources are relatively tight, this method of drug verification that relies on manual operations not only increases the hospital's operating costs, but also reduces the efficiency and safety of drug management. Summary of the invention

[0005] In view of the defects in the existing methods, the present invention aims to develop an automated technology to quickly and accurately identify combination drugs, so that hospitals can improve efficiency and accuracy in the process of checking prescription drugs. This technology aims to accurately identify the combination pills that have been assigned to patients. Faced with the challenges of small differences in size, color, and shape of pills, and the difficulty in identifying subtle features, as well as the wide variety of pills and various prescription combinations in pharmacies, the direct application of deep learning models to classify pills is limited in accuracy. The present invention proposes a method for grading and identifying combination pills.

[0006] To achieve the above object, an embodiment of the present invention provides a combined pill classification identification method, comprising:

[0007] Acquire a combination pill image to be identified;

[0008] Identify and obtain the position information of each pill in the combined pill image;

[0009] Separating each pill in the combined pill image according to the position information of each pill to obtain a single pill image;

[0010] Performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image;

[0011] Performing category recognition processing on the combined pill image to obtain the category to which the combined pills in the combined pill image belong;

[0012] According to the category to which the combined pills belong and the pill feature vector in the single pill image, the feature representation of each pill is calculated;

[0013] The classification result of each pill is confirmed based on the feature representation of each pill.

[0014] Further, the identifying and obtaining the position information of each pill in the combined pill image includes:

[0015] The combined pill image is input into a preset position detection model for target detection processing to obtain the position information of each pill in the combined pill image.

[0016] Furthermore, the training method of the position detection model includes:

[0017] Pre-built YOLOv5s object detection model;

[0018] The YOLOv5s target detection model is trained using a training data set consisting of a plurality of training pictures containing combined pill images and their corresponding pill annotation position information to obtain the position detection model.

[0019] Furthermore, the training images are enhanced using Mosaic image enhancement technology to obtain more training images.

[0020] Further, performing category recognition processing on the combined pill image to obtain the category to which the combined pills in the combined pill image belong includes:

[0021] The combined pill image is input into a preset combined category recognition model for category recognition processing to obtain the category to which the combined pills in the combined pill image belong.

[0022] Furthermore, the combined category recognition model is obtained by training a pre-built first ShuffleNet v2 model.

[0023] Further, performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image includes:

[0024] The single pill image is input into a preset feature extraction model for feature extraction processing to obtain a pill feature vector in the single pill image.

[0025] Furthermore, the feature extraction model is obtained by training a pre-built second ShuffleNet v2 model.

[0026] Furthermore, the feature representation of each pill is calculated based on the category to which the combined pills belong and the pill feature vector in the single pill image, including:

[0027] Generate a mask matrix according to the category to which the combined pills belong;

[0028] The mask matrix is ​​multiplied by the pill feature vector to obtain a feature representation of each pill.

[0029] Further, confirming the classification result of each pill according to the characteristic representation of each pill includes:

[0030] The softmax function is used to process the feature representation, calculate the probability distribution of each pill belonging to each category, and take the category with the highest probability as the final classification result.

[0031] The beneficial effects of the present invention are:

[0032] (1) Improved recognition accuracy: The hierarchical label classification strategy significantly improved the accuracy of pill recognition. This strategy refines the recognition process in stages, allowing the model to more accurately identify the combination and specific types of pills.

[0033] (2) Enhanced target recognition focus: By introducing the mask matrix, the model can focus more on the recognition of target categories in the secondary classification stage, thereby further improving the recognition focus.

[0034] (3) Strong adaptability: This method is particularly suitable for resource-constrained devices, such as small and medium-sized medical auxiliary equipment, and has a low computing cost. This enables the present invention to run under limited hardware resources while maintaining efficient performance.

[0035] (4) Flexibility and scalability: The designed method allows that when the number of pill combinations increases, there is no need to retrain the target detection network. Only the classification network needs to be updated, which facilitates future upgrades and expansions of the system.

[0036] (5) Improve medical safety: Accurate pill identification reduces medication errors caused by human error, thereby improving the safety of the entire drug delivery system and the safety of patient medication.

[0037] (6) Optimizing medical resources: The automated pill identification process reduces the burden on pharmacists in drug verification work, allowing them to focus on tasks that require more professional judgment. The present invention helps to optimize the allocation of medical resources, reduce hospital operating costs, and improve the efficiency and quality of pharmacy work. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0039] Figure 1 The process diagram of the combined pill classification identification method provided by the embodiment of the present invention is as follows Figure 1 ;

[0040] Figure 2 It is the framework structure of the combined pill classification identification method provided by the embodiment of the present invention;

[0041] Figure 3 The process diagram of the combined pill classification identification method provided by the embodiment of the present invention is as follows Figure 2 ;

[0042] Figure 4 It is a schematic diagram of the structure of the YOLOv5s target detection model network provided by an embodiment of the present invention;

[0043] Figure 5 This is the training and construction process of the YOLOv5s target detection model provided by an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of shooting a combined pill data set provided by an embodiment of the present invention;

[0045] Figure 7 is a schematic diagram of the operation flow of the Mosaic image enhancement technology provided by an embodiment of the present invention;

[0046] Figure 8 : is a relationship diagram between the training effect and the number of iterations of YOLOv5s provided in the embodiment of the present invention;

[0047] Fig. 9 The test accuracy and loss of the first-level classification model Shufflenet v2 training effect provided by the embodiment of the present invention vary with the number of iterations;

[0048] Fig.10It is a structural diagram of building a second ShuffleNet v2 model provided by an embodiment of the present invention;

[0049] Fig.11 is a schematic diagram of shooting a single pill data set provided by an embodiment of the present invention;

[0050] Fig.12 It is a schematic diagram of the change in accuracy of the two-level classification model Shufflenet v2 during the training process provided by an embodiment of the present invention;

[0051] Fig.13 It is a structural schematic diagram of a combined pill identification system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0053] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0054] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0055] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0056] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0057] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0058] The technical solution of this application aims to accurately identify the name of each pill in the combination pills that have been distributed to the patient. Faced with the challenges of small differences in size, color, and shape of pills, and the difficulty in identifying subtle features, as well as the wide variety of pills and various prescription combinations in pharmacies, the accuracy of directly applying the target detection network for classification is limited. This embodiment proposes a combination pill classification recognition method. The specific experimental steps of this method are shown in the attached Figure 1 , Figure 2 shown.

[0059] Specifically, Figure 3 As shown, a flow chart of a combined pill classification identification method provided by an embodiment of the present invention. The method comprises the following steps:

[0060] Step S110: Acquire the combined pill image to be identified.

[0061] In this embodiment, a combination pill image to be identified is captured by a camera or other shooting device. The acquired combination pill image contains combination pills to be identified, and the number of combination pills can be one, two, or more, such as three, four, five, etc.; the types of combination pills can be one, two, or more, such as three, four, five, etc.; the number of pills in each type can also be one, two, or more, such as three, four, five, etc.

[0062] In this embodiment, all the pills are oral solid dosage pills without outer packaging.

[0063] Step S120: Identify and obtain the position information of each pill in the combined pill image.

[0064] Specifically, the method includes: inputting the combined pill image into a preset position detection model to perform target detection processing, and obtaining the position information of each pill in the combined pill image.

[0065] Wherein, the training method of the position detection model includes:

[0066] Pre-built YOLOv5s object detection model;

[0067] The YOLOv5s target detection model is trained using a training data set consisting of a plurality of training pictures containing combined pill images and their corresponding pill annotation position information to obtain the position detection model.

[0068] Figure 4 : is a schematic diagram of the structure of the YOLOv5s target detection model network provided in this embodiment. YOLOv5s includes four key parts: Input, Backbone, Neck and Head.

[0069] Figure 5 The training and construction process of the YOLOv5s object detection model in the present invention is shown. In the pre-training stage, the model is trained using a training dataset containing images of combined pills and their corresponding annotated position information (in the form of a txt file). In the testing stage, YOLOv5s will perform accurate detection and positioning of each pill in the combined medicine image.

[0070] The data sets used in this example include:

[0071] Single pill image dataset: contains 300 categories of single pill images, with at least 40 images in each category, and the image resolution is uniformly 224×224 pixels.

[0072] Combination Pills Dataset: Covers 15 different combinations of 17 single pills, totaling 782 images.

[0073] Figure 6 Schematic diagram of shooting the combined pill dataset provided in this embodiment. All pills used for combination are from 300 types of pills collected in the single pill image dataset, while ensuring the consistency of the background and shooting conditions with the combined pill dataset.

[0074] In order to ensure the accuracy and credibility of the training data set, this embodiment has specially formulated clear inclusion and exclusion criteria. The inclusion criteria include: solid dosage forms that have been stripped of hard packaging (excluding granules and powder forms), covering a variety of tablets (such as ordinary tablets, enteric-coated tablets, coated tablets, etc.), hard capsules and soft capsules (or capsules). At the same time, ensure that each pill has at least one difference in appearance (color, shape, imprint, etc.) so that the model can effectively distinguish. The exclusion criteria exclude pills that have obvious defects due to damage, contamination or other reasons, ensuring the image quality in the data set, thereby improving the accuracy of model training and recognition.

[0075] Furthermore, when constructing a training data set, the Mosaic image enhancement technology can be used to enhance the training images to obtain more training images. Figure 7As shown in the figure, the specific implementation steps are described as follows: First, four pictures containing combined pills are randomly selected, and a series of data augmentation operations are performed on these pictures, including adjusting brightness, color temperature, flipping or scaling, etc. Subsequently, the four processed pictures are spliced ​​together in the order from upper left to lower left and upper right to lower right to form a new picture. Assuming that the input size of the model is s, a gray background image of size 2s×2s will be generated in the end. On this background image, within the rectangular area defined by point M (located at s / 2, s / 2) and point N (located at 3s / 2, 3s / 2), an anchor point is randomly selected for the splicing operation, and the selected parts of the four pictures are merged with the gray background image to construct an enhanced image for YOLOv5s model training to improve the model's generalization ability for different pill combinations.

[0076] Figure 8 The figure is a graph showing the relationship between the training effect and the number of iterations of YOLOv5s in this embodiment. YOLOv5s is used as the target detection network of the combined pill recognition model. The training parameter settings include: the total number of iterations (epoch) is 250, the number of samples processed in each batch (batch_size) is 128, the Adam optimizer is used for weight update, the initial learning rate is set to 0.01, and the learning rate is reduced to 20% of the initial value during the training process. Figure 8 It can be observed that with the increase in the number of iterations, the mAP (average precision) index of the model on the test set steadily improves and eventually stabilizes at a higher level.

[0077] Step S130: Separate each pill in the combined pill image according to the position information of each pill to obtain a single pill image.

[0078] In this embodiment, a single pill image can be obtained by separating each pill in the combined pill image according to the position information of each pill through a preset cropping module.

[0079] Step S140: performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image.

[0080] Specifically, the method includes: inputting the single pill image into a preset feature extraction model for feature extraction processing to obtain a pill feature vector in the single pill image. The feature extraction model is used as a two-pole classification model.

[0081] In this embodiment, the feature extraction model is obtained by training a pre-built second ShuffleNet v2 model. Fig.10 Schematic diagram of the structure of the second ShuffleNet v2 model built in this embodiment.

[0082] Fig.11 The schematic diagram of shooting a single pill data set provided in this embodiment is shown. The collected pill data are all oral solid dosage pills without outer packaging.

[0083] In this embodiment, the feature extraction model is based on the lightweight ShuffleNet v2. In this embodiment, ShuffleNet v2 is selected as the basic architecture of the two-level classification model. In the training stage, the model is trained using a data set containing 300 types of single pill images to learn and capture key features that can distinguish different types of pills, thereby obtaining an efficient pre-training model.

[0084] In the test phase, the model receives a single pill image cropped from the combined drug image and performs deep feature extraction through the convolutional layer of ShuffleNet v2. Subsequently, the fully connected layer of ShuffleNet v2 maps the extracted high-level features into a 300-dimensional feature vector space to obtain the pill feature vector in the single pill image.

[0085] Fig.12 In the embodiment of the present invention, the accuracy change of the secondary classification model Shufflenet v2 during the training process, including the growth trend of the training accuracy and the test accuracy with the number of iterations. The training parameters are the same as those of the primary classification model. The accuracy of the training set steadily increases with the increase in the number of training cycles, and tends to saturation at about 50 epochs, with the accuracy close to 1. The accuracy on the test set fluctuates in the early stage, but after the model is trained for 50 epochs, the accuracy begins to stabilize, and reaches more than 0.9 after 75 epochs, indicating that the secondary classification model has good recognition ability for new data.

[0086] Step S150: performing category recognition processing on the combined pill image to obtain the category to which the combined pills in the combined pill image belong.

[0087] Specifically, it includes: inputting the combined pill image into a preset combined category recognition model to perform category recognition processing, and obtaining the category to which the combined pills in the combined pill image belong.

[0088] In this embodiment, the combined category recognition model is obtained by training a pre-built first ShuffleNet v2 model, and the combined category recognition model serves as a primary classification model.

[0089] The combination category recognition model performs a first-level classification on the combination pills and assigns a first-level label to the combination pills, such as distinguishing different combination forms such as "combination pill 1" and "combination pill 2". The pill images are classified into a smaller category.

[0090] Fig. 9 This is an example of the test accuracy and loss of the first-level classification model Shufflenet v2 training effect changing with the number of iterations in an embodiment of the present invention. The training parameters include: the number of iterations (Epoch) is set to 200, the number of samples per batch (batch_size) is 128, the Adam optimizer is also used, and the initial learning rate is set to 0.1. As can be seen from the figure, with the deepening of training, the accuracy of the model gradually improves. After 25 epochs of training, the accuracy began to stabilize and reached a good level of more than 85%. Continue training until 200 epochs, the accuracy of the model further stabilizes at about 97%, and the loss value also drops below 0.2, showing that the model has a high learning efficiency and generalization ability for the training data.

[0091] Step S160: Calculate the feature representation of each pill according to the category to which the combined pill belongs and the pill feature vector in the single pill image.

[0092] Specifically, the method includes: generating a mask matrix according to the category to which the combined pills belong; and multiplying the mask matrix with the pill feature vector to obtain a feature representation of each pill.

[0093] The mask matrix has the same dimension as the output vector of the fully connected layer of the feature extraction model. The combination pills are classified into the same category using the combination category recognition model. The position of the target category is set to 1 in the mask matrix, and the other positions are set to 0. Fig.10 As shown in , by multiplying the mask matrix with the pill feature vector, a feature representation that focuses on the target category can be obtained.

[0094] Step S170: confirming the classification result of each pill according to the characteristic representation of each pill.

[0095] Specifically, the classifier uses the softmax function to process the feature representation, calculates the probability distribution of each pill belonging to each category, takes the category with the highest probability as the final classification result, and obtains the name of each pill.

[0096] Based on the same inventive concept, an embodiment of the present invention provides a combined pill identification system. Fig.13As shown, the system may include: one or more processors 101, one or more input devices 102, one or more output devices 103 and a memory 104, wherein the processor 101, the input device 102, the output device 103 and the memory 104 are interconnected via a bus 105. The memory 104 is used to store a computer program, wherein the computer program includes program instructions, and the processor 101 is configured to call the program instructions to execute the method of the embodiment of the above-mentioned method for grading and identifying combined pills.

[0097] It should be understood that in the embodiment of the present invention, the processor 101 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0098] The input device 102 may include a keyboard, etc., and the output device 103 may include a display (LCD, etc.), a speaker, etc.

[0099] The memory 104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include a non-volatile random access memory. For example, the memory 104 may also store information on the device type.

[0100] In a specific implementation, the processor 101, input device 102, and output device 103 described in the embodiment of the present invention can execute the implementation method described in the embodiment of the combined pill grading identification method provided by the embodiment of the present invention, which will not be described in detail here.

[0101] It should be noted that, for the specific working process of the combined pill identification system, reference may be made to the aforementioned method embodiment section, which will not be described in detail here.

[0102] Furthermore, an embodiment of the present invention also provides a readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the above-mentioned combined pill grading identification method is implemented.

[0103] The computer-readable storage medium may be an internal storage unit of the background server described in the aforementioned embodiment, such as a hard disk or memory of the system. The computer-readable storage medium may also be an external storage device of the system, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the system. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the system and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the system. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0104] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0107] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for grading and identifying combined pills, characterized in that: include: Acquire a combination pill image to be identified; Identify and obtain the position information of each pill in the combined pill image; Separating each pill in the combined pill image according to the position information of each pill to obtain a single pill image; Performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image; Performing category recognition processing on the combined pill image to obtain the category to which the combined pills in the combined pill image belong; According to the category to which the combined pills belong and the pill feature vector in the single pill image, the feature representation of each pill is calculated; The classification result of each pill is confirmed based on its feature representation.

2. A combined pill classification and identification method as claimed in claim 1, characterized in that: The identifying and obtaining the position information of each pill in the combined pill image includes: The combined pill image is input into a preset position detection model for target detection processing to obtain the position information of each pill in the combined pill image.

3. A combined pill classification and identification method as claimed in claim 2, characterized in that: The training method of the position detection model includes: Pre-built YOLOv5s object detection model; The YOLOv5s target detection model is trained using a training data set consisting of a plurality of training pictures containing combined pill images and their corresponding pill annotation position information to obtain the position detection model.

4. A combined pill classification and identification method as claimed in claim 3, characterized in that: The training images are enhanced using Mosaic image enhancement technology to obtain more training images.

5. A combined pill classification and identification method as claimed in claim 1, characterized in that: Performing category recognition processing on the combined pill image to obtain the category to which the combined pills in the combined pill image belong, includes: The combined pill image is input into a preset combined category recognition model for category recognition processing to obtain the category to which the combined pills in the combined pill image belong.

6. A combined pill classification and identification method as claimed in claim 5, characterized in that: The combined category recognition model is obtained by training the pre-built first ShuffleNet v2 model.

7. A combined pill classification and identification method as claimed in claim 1, characterized in that: Performing feature extraction processing on the single pill image to obtain a pill feature vector in the single pill image includes: The single pill image is input into a preset feature extraction model for feature extraction processing to obtain a pill feature vector in the single pill image.

8. A combined pill classification and identification method as claimed in claim 7, characterized in that: The feature extraction model is obtained by training a pre-built second ShuffleNet v2 model.

9. A method for grading and identifying combined pills as claimed in claim 1, characterized in that: The method of calculating the feature representation of each pill according to the category to which the combined pills belong and the feature vector of the pill in the single pill image includes: Generate a mask matrix according to the category to which the combined pills belong; The mask matrix is ​​multiplied by the pill feature vector to obtain a feature representation of each pill.

10. A combined pill classification and identification method as claimed in claim 1, characterized in that: The step of confirming the classification result of each pill according to the characteristic representation of each pill includes: The softmax function is used to process the feature representation, calculate the probability distribution of each pill belonging to each category, and take the category with the highest probability as the final classification result.

Images