A computer vision-based automatic drug identification system
By using a computer vision-based automatic drug identification system, the system separates the identity and degradation features from drug images, solving the accuracy problem of drug identification and quality assessment, and enabling safe supervision support throughout the entire drug lifecycle.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU MUNICIPAL HOSPITAL
- Filing Date
- 2025-07-29
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies struggle to accurately identify drug identities and assess their quality when drug appearance changes, and are unable to effectively distinguish between counterfeit and substandard drugs.
An automatic drug identification system based on computer vision is adopted. The intrinsic identity vector and degradation state vector are separated from the drug image through the feature extraction module. Combined with the identity recognition, quality assessment and risk identification modules, the system can realize drug identity recognition, quality assessment and identification of counterfeit and substandard drugs.
It achieves accuracy and robustness in drug identification, provides objective quantitative assessment of drug quality and efficient identification of counterfeit and substandard drugs, and enhances the full life-cycle support for drug safety supervision.
Smart Images
Figure CN120689631B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer technology and artificial intelligence, specifically to an automatic drug identification system based on computer vision. Background Technology
[0002] In the field of drug safety supervision, rapid and accurate drug identification is fundamental to ensuring public medication safety. Traditional drug identification technologies, whether manual verification or early computer vision systems, primarily rely on matching static features of the drug's appearance. However, these methods have the following shortcomings:
[0003] The physical state of medicines is not static. During storage, transportation and use, medicines will degrade to varying degrees due to environmental factors such as temperature and humidity changes and light exposure, resulting in changes in appearance. Traditional identification systems often confuse these changes in appearance caused by degradation with the identity information of the medicine itself, which may lead to misidentification of a medicine that has deteriorated but is still genuine, or make it impossible to determine its usability.
[0004] Existing technologies generally lack the ability to quantitatively assess drug quality. When drugs show changes in appearance, regulators or users find it difficult to judge by the naked eye whether the degree of degradation has exceeded the limits of safe drug use.
[0005] For high-quality counterfeit drugs with sophisticated manufacturing processes, their appearance is almost identical to that of genuine products when they leave the factory. Traditional static identification methods are difficult to effectively distinguish them. Although these counterfeit drugs look similar, the differences in their internal excipients and production processes will lead to a fundamental difference in their aging and degradation process under the influence of environmental factors. This difference in dynamic process is a key clue that traditional methods cannot capture.
[0006] Therefore, there is an urgent need for an intelligent solution that can transcend the limitations of static identification, accurately determine the identity of drugs, assess their real-time appearance quality, and understand their internal physical processes to identify the risks of counterfeit and substandard products. Summary of the Invention
[0007] The purpose of this invention is to provide an automatic drug identification system based on computer vision to solve the problems mentioned in the background art.
[0008] The technical solution of this invention is a computer vision-based automatic drug identification system, comprising:
[0009] The feature extraction module is used to extract intrinsic identity vector and degradation state vector from the acquired digital image of the drug to be identified; wherein, the intrinsic identity vector represents the intrinsic identity information of the drug, and the degradation state vector represents the appearance degradation information of the drug;
[0010] An identity recognition module is used to generate the identity recognition result of the drug based on the intrinsic identity vector and a pre-set classifier network;
[0011] The quality assessment module is used to generate a quality quantification score of the drug based on the degradation state vector and a preset degradation tolerance threshold.
[0012] The risk identification module is used to generate clues for identifying counterfeit and substandard drugs based on the degradation state vector, a preset library of genuine degradation patterns, and a preset pattern difference threshold.
[0013] Preferably, the feature extraction module, through an adversarial training mechanism, constrains the intrinsic identity vector to not contain information that can be used to determine the drug degradation state, and constrains the degradation state vector to not contain information that can be used to determine the drug identity, thereby achieving feature decoupling between the intrinsic identity vector and the degradation state vector.
[0014] Preferably, the quality assessment module is specifically used for:
[0015] Calculate the L2 norm of the degradation state vector;
[0016] The calculated L2 norm is divided by the preset degradation tolerance threshold to obtain the relative degree of degradation.
[0017] Based on the relative degree of degradation, a mass quantification score of the drug is generated.
[0018] Preferably, the degradation tolerance threshold is derived from the degradation vector norm corresponding to the critical point of non-compliance of physicochemical indicators as specified in pharmacopoeia standards or drug stability test data.
[0019] Preferably, the risk identification module is specifically used for:
[0020] The degradation state vector is normalized to obtain a normalized degradation state vector.
[0021] Calculate the cosine distance between the normalized degradation state vector and each normalized genuine degradation pattern vector in the genuine degradation pattern library;
[0022] Determine the minimum value among the cosine distances to obtain the minimum pattern distance;
[0023] The minimum pattern distance is compared with the preset pattern difference threshold to generate the clue for identifying the risk of counterfeit and substandard drugs.
[0024] Preferably, the method for generating the risk identification clues for counterfeit and substandard drugs includes:
[0025] When the minimum pattern distance is greater than the preset pattern difference threshold, a high-risk level clue is generated;
[0026] When the minimum pattern distance is not greater than the preset pattern difference threshold, a low-risk level clue is generated.
[0027] Preferably, the genuine degradation pattern library is derived from a set of degradation state vectors extracted from real drugs during their life cycle by conducting accelerated aging experiments under preset conditions.
[0028] Preferably, the feature extraction module includes: the extracted intrinsic identity vector and the degradation state vector, which are jointly complete in terms of information and sufficient to characterize all the visual information of the digital image of the drug to be identified.
[0029] This invention provides an improved computer vision-based automatic drug identification system, which has the following improvements and advantages compared with the prior art:
[0030] 1. This invention, through a unique adversarial training mechanism, enables the system to extract features representing the inherent identity of a drug and features representing its appearance degradation state from a single image. This makes the identification no longer affected by changes in the appearance of the drug caused by factors such as aging and moisture. Even if the drug changes color or cracks, the system can still accurately determine its true identity, greatly improving the accuracy and robustness of the identification.
[0031] 2. This invention can not only identify the degradation state of drugs, but also correlate it with the non-compliance critical point of physicochemical indicators determined by pharmacopoeia or drug stability test, generating a standardized and intuitive quality quantification score. This transforms the evaluation of drug quality from subjective judgment to objective and repeatable digital measurement, providing an automated decision-making basis for drug quality monitoring.
[0032] 3. This invention targets counterfeit and substandard drugs that are highly similar in appearance to genuine products. It determines their authenticity by analyzing the directional patterns of their degradation characteristics in a high-dimensional space, rather than simply the degree of degradation. Since the differences in materials and processes of counterfeit and substandard drugs can cause their aging paths to deviate from those of genuine products, this invention can effectively identify such anomalies based on physical processes by comparing the degradation patterns of the drug under test with a pre-set library of degradation patterns of genuine products, thereby accurately identifying high-risk counterfeit and substandard drugs.
[0033] 4. This invention integrates three major functional modules: identity recognition, quality assessment, and risk identification. It can output the identity result, quality score, and counterfeit risk level of a drug from a single image. This multi-dimensional comprehensive analysis capability provides solid technical support and comprehensive decision-making basis for the safety supervision of drugs throughout their entire life cycle, significantly surpassing the limitations of traditional single identification functions. Attached Figure Description
[0034] Figure 1 This is a flowchart of the system of the present invention;
[0035] Figure 2 This is a flowchart of the method for generating clues for identifying counterfeit and substandard drugs according to the present invention. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.
[0037] Example 1
[0038] Please see Figure 1 A computer vision-based automatic drug identification system includes:
[0039] The feature extraction module is used to extract intrinsic identity vector and degradation state vector from the acquired digital image of the drug to be identified; wherein, the intrinsic identity vector represents the intrinsic identity information of the drug, and the degradation state vector represents the appearance degradation information of the drug.
[0040] The identity recognition module is used to generate drug identity recognition results based on intrinsic identity vectors and a pre-set classifier network;
[0041] The quality assessment module is used to generate a quantitative score of drug quality based on the degradation state vector and a preset degradation tolerance threshold.
[0042] The risk identification module is used to generate clues for identifying counterfeit and substandard drugs based on degradation state vectors, a preset library of genuine degradation patterns, and preset pattern difference thresholds.
[0043] This invention provides a computer vision-based automatic drug identification system. This system integrates multiple functional modules, aiming to overcome the limitations of traditional static identification and achieve in-depth analysis of drug identity, appearance, and even potential risks. The core technology of this system lies in its feature extraction module, which can simultaneously parse two fundamentally different but interrelated pieces of information from a single digital image of a drug: an intrinsic identity vector representing its unchanging core identity and appearance degradation information describing its current physical state. Based on the output of this feature extraction module, the identity recognition module, quality assessment module, and risk identification module work collaboratively using this decoupled information, not only outputting accurate identity judgments but also providing quantitative quality assessment and the ability to identify counterfeit and substandard drugs. This provides solid technical support and multi-dimensional decision-making basis for the safe supervision of drugs throughout their entire lifecycle.
[0044] Example 2
[0045] The feature extraction module, through an adversarial training mechanism, constrains the intrinsic identity vector to not contain information that can be used to determine the drug degradation state, and constrains the degradation state vector to not contain information that can be used to determine the drug identity, thereby achieving feature decoupling between the intrinsic identity vector and the degradation state vector;
[0046] The feature extraction module includes: the extracted intrinsic identity vector and degradation state vector, which are jointly complete in terms of information and sufficient to characterize all the visual information of the digital image of the drug to be identified;
[0047] To achieve the above system functions, the key technology lies in the training process of the feature extraction module, which is implemented through a composite decoupling loss function. End-to-end optimization is performed using the following formula:
[0048] ;
[0049] This function's design integrates the ideas of multi-task learning and generative adversarial networks, aiming to ensure the effectiveness of feature extraction while actively penalizing the leakage of different types of information in the feature vector, thereby forcing the pure separation of features. This design enables the model to learn the essential laws governing the changes and invariants between identity and degradation, rather than merely memorizing superficial phenomena. It is the reconstruction and classification loss. It is orthogonal decoupling loss. It is a dimensionless hyperparameter used to balance the performance of the main task with the strength of decoupling;
[0050] To further clarify, The specific form is:
[0051] ;
[0052] This formula is derived from image reconstruction loss. and classification loss Weighted composition ensures the joint integrity of information; among which, Using the L1 norm, i.e. This constraint, used to penalize pixel-level differences between the generated and original images, ensures the integrity of the intrinsic identity vector. and degradation state vector The union contains all the visual information needed to reconstruct the original image; Cross-entropy loss is used to penalize the identity classifier. Predictions and Real Labels The difference between them, this constraint ensures that the vector It contains sufficient and efficient identity discrimination information; G represents the generator network. This represents the input digital image of the drug to be identified. It is an identity classifier network. It is the true identity label of the drug. and It is a dimensionless hyperparameter used to balance the importance of two tasks;
[0053] Orthogonal decoupling loss This is the key mechanism for achieving the core innovation of this invention, and its formula is defined as follows: ;
[0054] The underlying logic of this loss term lies in achieving feature decoupling through two parallel adversarial game processes; to this end, an identity discriminator network is introduced. and degradation information discriminator network ;
[0055] Identity detector Attempt to degrade vector In the process of predicting drug identity, the encoder branch in the feature extraction module responsible for generating degradation vectors is optimized to generate more accurate and accurate data. Vectors that cannot identify identities This allows identity information to be transferred from vectors. Eradication; Degradation information discriminator Attempt from intrinsic identity vector The system predicts whether a drug is in a degraded state. Its classification labels can be pre-set based on whether the training images come from degraded samples. Meanwhile, the encoder branch in the feature extraction module, responsible for generating identity vectors, is optimized to generate vectors without any cues related to the drug's degradation state. Thus, the degradation information is transferred from Remove from the middle;
[0056] Among them, the first expected term It is for the degradation information encoder branch From the image Degradation vector extracted from The calculation is performed similarly for the second expected term. It is for the branch of identity information encoder From the image identity vector extracted from To calculate;
[0057] After this adversarial training, the system finally reached an equilibrium, and the intrinsic identity vector... It is not sensitive to changes in appearance, but the degradation state vector This is independent of the specific drug identity, thus achieving feature decoupling.
[0058] Example 3
[0059] The quality assessment module is specifically used for: calculating the L2 norm of the degradation state vector; dividing the calculated L2 norm by a preset degradation tolerance threshold to obtain the relative degradation degree; and generating a quality quantification score for the drug based on the relative degradation degree.
[0060] The degradation tolerance threshold is derived from the degradation vector norm corresponding to the critical point of non-compliance of physicochemical indicators as specified in pharmacopoeia standards or drug stability test data.
[0061] The quality assessment module of this invention combines abstract deep learning features with pharmaceutical standards that have clear physical meaning. Its specific operating mechanism lies in the quantification score of drug quality. The calculation formula is as follows:
[0062] ;
[0063] This formula aims to convert the continuously degrading state vector inferred by the model into... This is converted into an intuitive, standardized quality score; by introducing a physical threshold derived from pharmaceutical practice. This formula establishes a quantitative correlation between computer vision feature space and drug physicochemical stability standards;
[0064] in, It is the final output quality quantification score; The feature extraction module of this invention, which has already been trained, receives a new image to be tested. The degradation state vector output when used as input; It is the L2 norm of the vector, and its value is positively correlated with the overall severity of drug degradation; This is the degradation tolerance threshold, which is not obtained through model training but rather through experimental calibration. Its value originates from pharmacopoeia standards or drug stability test data, and represents the norm of the degradation vector corresponding to when a specific drug's key physicochemical indicators reach the non-compliance critical point. Because... and Having the same physical dimensions, their ratio is dimensionless, ensuring the rigor of the formula; The function ensures that the fractions are non-negative;
[0065] In practical applications, the calculation process of this module begins with calculating the degradation vector of the drug to be tested. The L2 norm, which objectively reflects the degree to which it deviates from the ideal state; based on this norm, and by comparing it with the degradation tolerance threshold... The original degradation measure is normalized to a relative degree of degradation by performing a division operation; the system then subtracts this ratio from 1 to generate a mass fraction between 0 and 1. ;when This indicates that the medicine is in good condition; the lower the score, the more severe the degradation. This means that the degradation of the drug has reached or exceeded the defined unacceptable threshold.
[0066] This function provides automated and digital decision-making support for quality monitoring in the drug distribution and use process, significantly improving the level of medication safety assurance.
[0067] Example 4
[0068] Please see Figure 2 The risk identification module is specifically used for: normalizing the degradation state vector to obtain a normalized degradation state vector; calculating the cosine distance between the normalized degradation state vector and each normalized genuine degradation pattern vector in the genuine degradation pattern library; determining the minimum value of the cosine distance to obtain the minimum pattern distance; and comparing the minimum pattern distance with a preset pattern difference threshold to generate clues for identifying counterfeit and substandard drugs.
[0069] The methods for generating clues for identifying counterfeit and substandard drugs include: generating a high-risk level when the minimum pattern distance is greater than a preset pattern difference threshold; and generating a low-risk level clue when the minimum pattern distance is not greater than a preset pattern difference threshold.
[0070] The genuine product degradation pattern library is derived from the set of degradation state vectors extracted from real drugs during their life cycle by conducting accelerated aging experiments under preset conditions.
[0071] The risk identification module of this invention provides a technical solution for identifying counterfeit and substandard drugs. Its identification logic lies in analyzing the directional pattern of the degradation vector, rather than its intensity norm. The operating mechanism of this module is to compare the degradation pattern of the drug to be tested with a pre-established library of genuine degradation patterns to generate a risk level.
[0072] Many counterfeit and substandard drugs differ from genuine products in their excipients or manufacturing processes, resulting in physicochemical degradation pathways that deviate from those of the genuine products; this difference manifests as degradation vectors. This module aims to identify counterfeit products that are difficult to distinguish by static appearance by capturing such physical process-based feature differences that deviate from the genuine product cluster in a high-dimensional feature space. The core of risk identification lies in the degradation vector of the drug under test. Degradation pattern library of genuine products Comparison; among which, It is a set of data extracted by conducting accelerated aging experiments on a large number of real drugs under preset temperature, humidity, and light conditions, collecting images throughout their entire life cycle, and then extracting corresponding degradation vectors. Each genuine product degradation pattern vector in the set is denoted as... ; Calculation of the discrimination process With all genuine pattern vectors in the library The minimum cosine distance between them is expressed as:
[0073] ;
[0074] in, and express and The L2 norm;
[0075] The minimum distance is compared with a preset pattern difference threshold. The threshold is set based on a statistical analysis of the intra-class distance within the genuine degradation pattern library and the inter-class distance between known counterfeit and substandard drug degradation patterns, thereby determining a boundary that can effectively distinguish between normal and abnormal degradation paths.
[0076] When performing risk assessment, this module's operation process is as follows: It calculates the degradation vector of the drug to be tested. Degradation pattern library of genuine products All vectors in The minimum pattern distance is obtained by finding the minimum cosine distance between the vectors. This calculation process aims to eliminate the influence of degradation severity (i.e., the magnitude of the vector) and only compares directional information.
[0077] Compare this minimum pattern distance with the pattern difference threshold. Comparison: When the minimum pattern distance is greater than the threshold, the system determines that the degradation pattern of the drug is abnormal and generates a high-risk level clue; otherwise, if the minimum pattern distance is not greater than the threshold, a low-risk level clue is generated.
[0078] This mechanism provides a high-precision identification method based on differences in physical processes, offering regulatory agencies a powerful technical means to greatly improve the accuracy and efficiency of combating counterfeit and substandard drugs.
[0079] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A computer vision-based automatic drug identification system, characterized in that, include: The feature extraction module is used to extract intrinsic identity vector and degradation state vector from the acquired digital image of the drug to be identified; wherein, the intrinsic identity vector represents the intrinsic identity information of the drug, and the degradation state vector represents the appearance degradation information of the drug; An identity recognition module is used to generate the identity recognition result of the drug based on the intrinsic identity vector and a pre-set classifier network; The quality assessment module is used to generate a quality quantification score of the drug based on the degradation state vector and a preset degradation tolerance threshold. The risk identification module is used to generate clues for identifying counterfeit and substandard drugs based on the degradation state vector, a preset library of genuine degradation patterns, and a preset pattern difference threshold. The feature extraction module, through an adversarial training mechanism, constrains the intrinsic identity vector from containing information that can be used to determine the drug degradation state, and also constrains the degradation state vector from containing information that can be used to determine the drug identity, thereby achieving feature decoupling between the intrinsic identity vector and the degradation state vector; The quality assessment module is specifically used for: Calculate the L2 norm of the degradation state vector; The calculated L2 norm is divided by the preset degradation tolerance threshold to obtain the relative degree of degradation. Based on the relative degree of degradation, a mass quantification score of the drug is generated; The risk identification module is specifically used for: The degradation state vector is normalized to obtain a normalized degradation state vector. Calculate the cosine distance between the normalized degradation state vector and each normalized genuine degradation pattern vector in the genuine degradation pattern library; Determine the minimum value among the cosine distances to obtain the minimum pattern distance; The minimum pattern distance is compared with the preset pattern difference threshold to generate the clue for identifying the risk of counterfeit and substandard drugs; The adversarial training mechanism is optimized end-to-end using a composite decoupling loss function, as shown in the formula: ,in, For composite decoupling loss function, For reconstruction and classification loss, A dimensionless hyperparameter used to balance the performance of the main task with the strength of decoupling. This refers to the orthogonal decoupling loss; the orthogonal decoupling loss formula is defined as follows: ,in, For the degradation information encoder branch, The input is a digital image of the drug to be identified. Let the degradation state vector be... For identity discriminator networks, For the identity information encoder branch, For the intrinsic identity vector, For degrading information discriminator networks; The formula for calculating the drug quality quantification score is as follows: ,in, To quantify the quality score, Let be the degradation state vector of the drug to be tested. This represents the degradation tolerance threshold. The minimum distance is expressed as: ； in, For genuine product degradation model library, This is the degradation mode vector for genuine products. and express and The L2 norm.
2. The automatic drug identification system based on computer vision according to claim 1, characterized in that, The degradation tolerance threshold is derived from the degradation vector norm corresponding to the critical point of non-compliance of physicochemical indicators as specified in pharmacopoeia standards or drug stability test data.
3. The automatic drug identification system based on computer vision according to claim 2, characterized in that, The methods for generating the risk identification clues for counterfeit and substandard drugs include: When the minimum pattern distance is greater than the preset pattern difference threshold, a high-risk level clue is generated; When the minimum pattern distance is not greater than the preset pattern difference threshold, a low-risk level clue is generated.
4. The automatic drug identification system based on computer vision according to claim 3, characterized in that, The genuine product degradation pattern library is derived from the set of degradation state vectors extracted from real drugs during their life cycle by conducting accelerated aging experiments under preset conditions.
5. The automatic drug identification system based on computer vision according to claim 1, characterized in that, The feature extraction module includes: the extracted intrinsic identity vector and the degradation state vector, which are jointly complete in terms of information and sufficient to characterize all the visual information of the digital image of the drug to be identified.