An intelligent management system applied to a special medicine cabinet

Abstract

This application relates to the technical field of drug management. Specifically, it discloses an intelligent management system applied to a special drug dedicated cabinet, which includes a login module, a drug addition module, a drug retrieval module, a drug return module, a recycling module, an inventory module, a query module, an account management module, and an exit module. Among them, when the recycling module conducts drug recycling, it uses a camera to collect images of damaged drugs, and through cross-modal semantic interaction analysis of the damaged drug images and the reported damage reasons input by the user, it intelligently identifies whether the drug status is consistent with the reported damage reasons and confirms whether the reported damage reasons of the drugs are input incorrectly. In this way, the accuracy and efficiency of damaged drug management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special drugs can be ensured.

Description

Technical Field

[0001] This application relates to the technical field of drug management processing, and more specifically, to an intelligent management system applied to a special drug dedicated cabinet. Background Art

[0002] The pharmaceutical industry is an important field related to public health, and the accuracy and safety of drug management are particularly important. In medical institutions such as hospitals, clinics, and pharmacies, special drugs, due to their particularity (such as high value, strict storage conditions, and usage regulations), pose higher requirements for management and tracking. Traditional drug management systems mostly rely on manual operations, including drug addition, dispensing, return, recycling, inventory taking, etc. This method is not only time-consuming and laborious but also prone to errors, and cannot ensure the accuracy and timeliness of information. Especially in the management of special drugs, due to the high value, special storage conditions, and strict safety supervision requirements of these drugs, incorrect handling may lead to serious consequences, including economic losses, medical accidents, and even legal risks.

[0003] In recent years, with the continuous progress of information technology, intelligent management systems have gradually been applied to the field of drug management, aiming to improve the automation and intelligence level of drug management. However, most of the current market solutions are limited to simple barcode or QR code scanning and still remain at the level of simple information entry and query, and can only basically track and record operations such as drug storage and retrieval, lacking intelligent recognition and verification mechanisms. For example, in the recycling management of damaged drugs, accurately recording the reasons for drug damage is crucial for ensuring drug safety, optimizing inventory management, and controlling costs. However, existing intelligent management systems often lack an intelligent verification mechanism for the reasons for drug damage and cannot automatically judge the rationality and accuracy of the input information, resulting in certain management loopholes and security risks.

[0004] Therefore, an optimized intelligent management system applied to a special drug dedicated cabinet is expected. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. An embodiment of this application provides an intelligent management system applied to a special drug dedicated cabinet, which includes a login module, a drug addition module, a drug dispensing module, a drug return module, a recycling module, an inventory taking module, a query module, an account management module, and an exit module. Among them, when the recycling module conducts drug recycling, it uses a camera to collect images of damaged drugs, and through cross-modal semantic interaction analysis of the images of damaged drugs and the reasons for drug damage input by the user, it intelligently identifies whether the drug status conforms to the reasons for drug damage and confirms whether the reasons for drug damage are incorrectly input. In this way, the accuracy and efficiency of damaged drug management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special drugs can be ensured.

[0006] Accordingly, in one aspect of the present application, an intelligent management system for a special medicine cabinet is provided, which includes:

[0007] A login module for performing identity authentication;

[0008] A medicine addition module for inputting medicine information, where the medicine information includes medicine name, packaging specification, medicine approval number, medicine expiration date, medicine traceability code, medicine addition layer number, and storage grid position code;

[0009] A medicine retrieval module for retrieving medicines from the storage grids of the medicine cabinet based on medicine retrieval information and updating the medicine inventory;

[0010] A medicine return module for performing a medicine return operation, where the medicine return operation includes selecting a storage grid where the medicine can be returned and completing the medicine return operation after determining to close the drawer of the storage grid;

[0011] A recycling module for medicine recycling and medicine write-off, where the medicine write-off includes inputting the write-off reason and confirming whether the input write-off reason is incorrect;

[0012] An inventory module for taking an inventory of all medicine information in the medicine cabinet;

[0013] A query module for querying medicine information and medicine storage information;

[0014] An account management module for performing account management;

[0015] An exit module for exiting the system.

[0016] In the above intelligent management system for a special medicine cabinet, the recycling module includes: a damaged medicine image acquisition unit for acquiring a damaged medicine image collected by a camera; a medicine state feature extraction unit for extracting the medicine state features of the damaged medicine image to obtain a damaged medicine state image semantic coding feature map; a damaged reason semantic coding unit for performing semantic coding on the input damaged reason to obtain a damaged reason semantic coding vector; a cross-modal interaction unit for performing damaged information-damaged state cross-modal interaction on the damaged reason semantic coding vector and the damaged medicine state image semantic coding feature map to obtain a damaged information-damaged state semantic alignment coding feature map; a damaged reason confirmation unit for determining whether the input damaged reason is incorrect based on the damaged information-damaged state semantic alignment coding feature map.

[0017] Compared with the prior art, the intelligent management system for special drug cabinets provided by the present application includes a login module, a drug addition module, a drug retrieval module, a drug return module, a recycling module, an inventory module, a query module, an account management module, and an exit module. Among them, when the recycling module conducts drug recycling, it uses a camera to collect images of damaged drugs, and through cross-modal semantic interaction analysis of the damaged drug images and the reported damage reasons input by the user, it intelligently identifies whether the drug status conforms to the reported damage reason and confirms whether the reported damage reason of the drug is input incorrectly. In this way, the accuracy and efficiency of damaged drug management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special drugs can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 It is a block diagram of an intelligent management system for special drug cabinets according to an embodiment of the present application.

[0020] Figure 2 It is a block diagram of the recycling module in the intelligent management system for special drug cabinets according to an embodiment of the present application.

[0021] Figure 3 It is a schematic diagram of data flow of the recycling module in the intelligent management system for special drug cabinets according to an embodiment of the present application.

[0022] Figure 4 It is a block diagram of the cross-modal interaction unit in the intelligent management system for special drug cabinets according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Next, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other objects, features, and advantages of the present application will become more apparent. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0024] In response to the technical problems described in the above background art, the present application proposes an intelligent management system for special drug cabinets. Figure 1 It is a block diagram of an intelligent management system for special drug cabinets according to an embodiment of the present application. As Figure 1As shown, the intelligent management system 100 applied to the special medicine cabinet includes: a login module 110 for performing identity authentication; a medicine addition module 120 for inputting medicine information, where the medicine information includes medicine name, packaging specification, medicine approval number, medicine expiration date, medicine traceability code, medicine addition layer number, and storage grid location code; a medicine retrieval module 130 for extracting medicines from the storage grids of the medicine cabinet based on the medicine retrieval information and updating the medicine inventory; a medicine return module 140 for performing the medicine return operation, where the medicine return operation includes selecting the storage grids where medicines can be returned and completing the medicine return operation after determining to close the drawers of the storage grids; a recycling module 150 for performing medicine recycling and medicine write-off, where the medicine write-off includes inputting the write-off reason and confirming whether the input write-off reason is incorrect; an inventory module 160 for taking inventory of all the medicine information in the medicine cabinet; a query module 170 for querying medicine information and medicine storage information; an account management module 180 for performing account management; and an exit module 190 for exiting the system.

[0025] Specifically, the operating environment of the intelligent management system applied to the special medicine cabinet in this application is Android 11. This system integrates multiple functional modules, and through the collaborative effect between each functional module, it realizes the safe management, efficient allocation, and precise tracking of special medicines, while providing users with a convenient operation experience.

[0026] Specifically, the login module 110 is used for performing identity authentication. It should be understood that users need to perform identity authentication before using the system. The login module 110 includes a first-level verification - account password or IC card verification, and a second-level verification - the application of fingerprint or face recognition technology. Through this dual-verification mechanism, it is ensured that only authorized personnel can access the system, greatly enhancing the security. After the dual verification is successful, the user can enter the main menu page. The main menu page contains function entry points for medicine addition, medicine retrieval, medicine return, recycling, inventory, query, account management, operation records, inventory records, and exit, meeting the requirements in different scenarios.

[0027] Specifically, the medicine adding module 120 is used to input medicine information, where the medicine information includes medicine name, packaging specification, medicine approval number, medicine expiration date, medicine traceability code, adding medicine layer number, and storage cell position code. On the main menu page, after the user clicks the "Medicine Adding" button, the user will enter the secondary verification login page of the medicine adding module 120, which requires the operator to perform double verification login of "swiping card or username password + fingerprint or face recognition" again to confirm their identity. After the verification is completed, the operator will enter the medicine adding interface. The operator can scan or manually input the medicine information to be added to the intelligent cabinet through the device, and then click the "Confirm" button. The system will display the layers where the medicine can be added and the remaining available positions for adding medicine on each layer, so that the operator can intuitively select a suitable storage cell. After the operator clicks on the layer with remaining positions, the system will enter the medicine information input interface and the medicine storage cell page of that layer. At the same time, the intelligent cabinet will automatically open the drawer of the corresponding layer. At this time, the operator can scan or manually input the medicine information through the device, and then click the "OK" button to put the medicine into the storage cell showing "Can be placed". The status of this storage cell will change to "Placed", indicating that the medicine adding operation for a single medicine has been completed. Repeating the above steps multiple times can complete the medicine adding for multiple medicines. After the medicine adding is completed, click the "Medicine Adding Completed" button. The system will close the drawer and end this medicine adding process, and at the same time update the relevant data to maintain the accuracy of the inventory information.

[0028] Specifically, the medicine dispensing module 130 is used to extract medicines from the storage cells of the dedicated medicine cabinet based on the medicine dispensing information and update the medicine inventory. On the main menu page, after the user clicks the "Medicine Dispensing" button, the user also needs to go through the above-mentioned secondary verification steps of "swiping card or username password + fingerprint or face recognition". After passing the verification, the user will enter the medicine dispensing type selection page, where the user can select two different medicine dispensing methods: "Prescription Medicine Dispensing" and "Transfer Medicine Dispensing". "Prescription Medicine Dispensing" is applicable for receiving specific medicines according to the prescription issued by the doctor. Clicking on "Prescription Medicine Dispensing" requires selecting a prescription, and then entering the medicine dispensing page. Clicking on "Transfer Medicine Dispensing" can directly enter the medicine dispensing page. After entering the medicine dispensing page, the operator can scan or manually input the medicine information through the device, and then click the "Confirm" button. The system will display the layers where the medicine can be obtained and the inventory of this medicine on each layer. The operator can select the layer where the medicine is located according to the need and confirm the quantity of medicine to be dispensed. Subsequently, the system will update the inventory of this medicine and complete the medicine dispensing operation. Specifically, after the user clicks on the layer with inventory, the system will enter the medicine storage cell page of that layer. At the same time, the intelligent cabinet will automatically open the drawer of the corresponding layer. The operator can take away the medicine in the storage cell showing "Can be taken", and the status of this storage cell will change to "Taken". After the medicine dispensing is completed, the operator can click the "Medicine Dispensing Completed" button. The system will close the drawer and complete the medicine dispensing operation. At this time, in addition to closing the drawer, the system will also synchronously refresh the inventory record to ensure information consistency.

[0029] Specifically, the drug return module 140 is used to perform drug return operations, which include selecting a storage cell where drugs can be returned and completing the drug return operation after determining to close the drawer of the storage cell. On the main menu page, after the user clicks the "Drug Return" button, they also need to go through the above-mentioned two-factor authentication steps of "swiping the card or using the username and password + fingerprint or face recognition". After successful authentication, the drug return page will be entered. The operator can scan or manually enter the drug information through the device. After entering the information of the returned drugs, the system will display the layer where the drugs can be returned and the remaining available positions on each layer. After the operator clicks on the layer with available positions, the system will enter the drug information entry interface and the drug storage cell page of that layer, and at the same time, the intelligent cabinet will automatically open the drawer of the corresponding layer. The operator can scan or manually enter the drug information through the device, and then click the "OK" button to put the drug into the storage cell marked as "Available". At this time, the status of this storage cell will change to "Placed", indicating that the drug return operation for a single drug has been completed. By repeating the above steps multiple times, the return of multiple drugs can be completed. After the drug return is completed, click the "Drug Return Completed" button, and the system will close the drawer and complete the drug return operation. At the same time, the system will also adjust the inventory data accordingly to reflect the latest changes.

[0030] Specifically, the recycling module 150 is used for drug recycling and drug write-off. The drug write-off includes entering the reason for write-off and confirming whether the entered reason for write-off is incorrect. On the main menu page, after the user clicks the "Recycling" button, the recycling layer and drug write-off selection page will be entered. On this page, the user needs to select the layer where the drug to be recycled is located and click the corresponding recycling layer button. At this time, the system will enter the drug recycling function page and automatically open the drawer of that layer. The user can perform drug recycling operations according to the page prompts. For example, after the operator scans or manually enters the drug information through the device, click the "Search" button to query the drug. Then, in the query result list, mark the drugs that need to be recycled and click the "Recycling" button to complete the recycling of a single drug. At the same time, the information of the drugs that have been recycled in this operation can be viewed in the recycling list. By repeating the above operations, the recycling of multiple drugs can be completed. After clicking the "Recycling Completed" button, the system will close the drawer and complete the drug recycling operation.

[0031] In addition, for the possible problem of damaged drugs, when performing the recycling task, the user needs to click the "Drug Damage" button on the drug recycling function page. The system will enter the drug damage function page. After the operator selects the layer where the drug to be damaged is located, scans the drug information through the device or manually enters it, and then clicks the "Search" button to query the drug. In the query result list, mark the drugs that need to be damaged, click the "Damage" button, fill in the reason for damage, and complete the damage of a single drug. At the same time, the user can also view the information of the drugs that have been damaged in this time in the damage list. By repeating the above operations, the damage of multiple drugs can be completed. After clicking the "Damage Completed" button, the system will close the drawer and complete the drug damage operation.

[0032] Specifically, the inventory module 160 is used to inventory all the drug information in the drug special cabinet. After the user clicks the "Inventory" button on the main menu page, it enters the inventory function page, and selects the layer that needs to be inventoried. The system will automatically open the drawer of this layer, and the page will default to display the list of all drug information on this layer. On this basis, the operator can modify the drug information or conduct inventory surplus or deficit operations on the drugs according to the actual situation. Click the "Inventory Record" button to view the drug records inventoried this time. After clicking the "Inventory Completed" button, the system will close the drawer of this layer and complete the inventory operation.

[0033] Specifically, the query module 170 is used to query drug information and drug storage information. After the user clicks the "Query" button on the main menu page, it enters the drug query page. On the drug query page, after entering the drug name and clicking the "Search Button", the corresponding drug information and storage information in the intelligent cabinet can be queried.

[0034] In addition, this system also supports querying operation records and inventory records. On the main menu page, after the user clicks the "Operation Record" button, it enters the operation record query page. On this page, the operation records can be filtered and viewed by entering the drug name, selecting the operation type, and selecting the operation time. The operation types include drug addition, drug retrieval, drug return, recycling, and damage. On the main menu page, after the user clicks the "Inventory Record" button, it enters the inventory record query page. On this page, the inventory records can be filtered and viewed by entering the drug name, selecting the inventory type, and selecting the inventory time. The inventory types include inventory surplus, inventory deficit, and modification.

[0035] Specifically, the account management module 180 is used for account management. On the main menu page, after the user clicks the "Account Management" button, the account list page is entered. On this page, the user can click the "Enter Information" button in the list to enter the personal biometric information and IC card information entry interface. Among them, the biometric information includes face information and fingerprint information. Clicking the "Enter Information" button can enter the information of a new user; clicking the "Re-enter Information" button can reset and update the information of an old user.

[0036] Specifically, the exit module 190 is used to exit the system. On the main menu page, when the user clicks the "Exit" button, the menu page is exited and the login page is returned.

[0037] In particular, aiming at the defects of the existing drug intelligent management system in the management of damaged drug recovery, the intelligent management system applied to the special drug cabinet in this application further optimizes the management process of damaged drugs. Specifically, when conducting damaged drug recovery, first, the damaged drug image is collected through a camera. Then, the image analysis of the damaged drug image is carried out by using the image processing technology based on deep learning to extract the state characteristics of the damaged drug, and cross-modal semantic interaction analysis is performed with the damaged reason input by the user, so as to intelligently identify whether the drug state conforms to its damaged reason and confirm whether the input of the damaged reason of the drug is incorrect. In this way, the accuracy and efficiency of damaged drug management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special drugs can be ensured.

[0038] Figure 2 It is a block diagram of the recovery module in the intelligent management system applied to the special drug cabinet according to the embodiment of the present application. Figure 3 It is a schematic diagram of the data flow of the recovery module in the intelligent management system applied to the special drug cabinet according to the embodiment of the present application. As Figure 1 and Figure 2 shown, the recovery module 150 includes: a damaged drug image acquisition unit 151 for acquiring the damaged drug image collected by the camera; a drug state feature extraction unit 152 for extracting the drug state features of the damaged drug image to obtain a damaged drug state image semantic coding feature map; a damaged reason semantic coding unit 153 for performing semantic coding on the input damaged reason to obtain a damaged reason semantic coding vector; a cross-modal interaction unit 154 for performing damaged information-damaged state cross-modal interaction on the damaged reason semantic coding vector and the damaged drug state image semantic coding feature map to obtain a damaged information-damaged state semantic alignment coding feature map; a damaged reason confirmation unit 155 for determining whether the input of the damaged reason is incorrect based on the damaged information-damaged state semantic alignment coding feature map.

[0039] In the recovery module 150, the damaged drug image acquisition unit 151 is configured to obtain the damaged drug images captured by a camera. It should be understood that images provide a direct and intuitive way to verify the status of drugs. By capturing the damaged drug images, the appearance of the drugs, the integrity of the packaging, the label information, etc. can be clearly seen, thus providing basic visual information for subsequent confirmation of the reasons for damage.

[0040] In the recovery module 150, the drug status feature extraction unit 152 is configured to extract the drug status features of the damaged drug images to obtain a semantic encoding feature map of the damaged drug status images. In a specific example of the present application, the drug status feature extraction unit 152 is configured to: input the damaged drug images into a drug status image feature extractor based on the DynamicViT model to obtain the semantic encoding feature map of the damaged drug status images. That is, in order to fully extract the drug status features from the damaged drug images, the present application uses the DynamicViT model, which has excellent performance in the field of computer vision, to construct a drug status image feature extractor. Those of ordinary skill in the art should be aware that the DynamicViT model is an efficient vision Transformer model based on dynamic Token sparsification, which can effectively extract discriminative features from images. Compared with traditional convolutional neural networks (CNNs), it performs particularly well in dealing with long-range dependencies in images. In addition, when traditional vision Transformer models perform image processing, they mainly divide the input image into multiple independent small blocks (called Patches or Tokens), and use the self-attention mechanism of the Transformer to capture the dependencies between multiple image blocks, so as to realize the feature encoding of the global visual information of the image. The DynamicViT model, by introducing the dynamic Token sparsification technology, can dynamically identify the complexity of the image content of each Token and cut off redundant Tokens, further improving the efficiency and accuracy of feature extraction. Based on this, in the technical solution of the present application, by using the DynamicViT model to process the damaged drug images, the self-attention mechanism of the Transformer can be used to simultaneously consider the global structure and local details of the damaged drugs, capture the overall shape of the packaging and subtle damage marks, thus providing a more accurate status description for subsequent confirmation of the reasons for damage, while effectively reducing the consumption of computing resources, reducing the computational amount of the model and improving the data processing efficiency.

[0041] In the recovery module 150, the loss cause semantic encoding unit 153 is used to semantically encode the input loss cause to obtain a loss cause semantic encoding vector. Here, in order to realize the semantic interaction analysis between the damaged drug status and the loss cause, the present application further integrates natural language processing (NLP) technology. After the user inputs the loss cause, the present application semantically encodes the loss cause to map it to a high-dimensional semantic space and convert it into a numerical vector representation that can be understood by a computer. In a specific example of the present application, the loss cause semantic encoding unit 153 is used to: semantically encode the loss cause using a semantic encoder based on the Bert model to obtain the loss cause semantic encoding vector. It should be understood that the Bert model is a pre-trained language representation model, which uses a bidirectional Transformer architecture to understand the content meaning of each word in the loss cause in the overall context, can effectively capture the complex semantic connection between words, and dig out the deep contextual semantic information of the loss cause, thereby achieving an in-depth understanding and accurate representation of the loss cause, and generating the loss cause semantic encoding vector.

[0042] In the recycling module 150, the cross-modal interaction unit 154 is used to perform cross-modal interaction of the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image to obtain a semantic alignment coding feature map of the cause of damage and the damage status. That is, by interactively analyzing the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image, the potential correlation between the cause of damage and the drug status is revealed, thereby verifying the rationality of the cause of damage.

[0043] Figure 4 FIG. 1 is a block diagram of a cross-modal interaction unit in an intelligent management system for special medicine cabinets according to an embodiment of the present application. Figure 4 As shown, the cross-modal interaction unit 154 includes: a local fine-grained semantic interaction sub-unit 1541, which is used to perform local-scale fine-grained semantic interaction on the loss cause semantic coding vector and the damaged drug status image semantic coding feature map to obtain a set of loss information-damage status cross-modal prompt information semantic coding matrices; a cross-modal interaction guidance optimization sub-unit 1542, which is used to guide the loss cause semantic coding vector and the damaged drug status image semantic coding feature map to perform cross-modal interaction optimization based on the set of loss information-damage status cross-modal prompt information semantic coding matrices to obtain the loss information-damage status semantic alignment coding feature map.

[0044] Specifically, the local fine-grained semantic interaction subunit 1541 includes: an autocorrelation encoding secondary subunit for performing autocorrelation encoding on the semantic encoding vector of the loss reporting reason to obtain an autocorrelation semantic encoding matrix of the loss reporting reason; a feature decoupling secondary subunit for performing feature decoupling on the semantic encoding feature map of the loss reporting drug status image along the channel dimension to obtain a set of local semantic feature matrices of the loss reporting drug status image; and a semantic interaction secondary subunit for performing cross-modal semantic interaction between the autocorrelation semantic encoding matrix of the loss reporting reason and each local semantic feature matrix of the loss reporting drug status image in the set of local semantic feature matrices of the loss reporting drug status image to obtain a set of cross-modal prompt information semantic encoding matrices of the loss reporting information - loss reporting status.

[0045] In a specific example of the present application, the calculation process of the autocorrelation encoding secondary subunit can be expressed by the formula:

[0046]

[0047] Wherein, represents the semantic encoding vector of the loss reporting reason, represents matrix multiplication operation, represents the transpose of the vector, represents the autocorrelation semantic encoding matrix of the loss reporting reason.

[0048] That is, first, by calculating the product between the semantic encoding vector of the loss reporting reason and its own transposed vector, the autocorrelation of the loss reporting reason is revealed by using the concept of outer product in matrix operation, and the autocorrelation semantic encoding matrix of the loss reporting reason is obtained.

[0049] In a specific example of the present application, the calculation process of the feature decoupling secondary subunit can be expressed by the formula:

[0050]

[0051] Wherein, represents the semantic encoding feature map of the loss reporting drug status image, represents feature decoupling, , , and respectively represent the first, second, th, and th local semantic feature matrices of the loss reporting drug status image along the channel dimension of the semantic encoding feature map of the loss reporting drug status image, is the number of channels of the semantic encoding feature map of the loss reporting drug status image.

[0052] That is, by performing feature decoupling on the semantic encoding feature map of the damaged drug status image along the channel dimension, the attention of the model to local details in the damaged drug status image is improved, the distinctiveness of features is enhanced, which helps to perform more fine-grained cross-modal interaction analysis.

[0053] In a specific example of the present application, the semantic interaction secondary subunit is used to: perform a linear transformation on the self-correlation semantic encoding matrix of the damage reason to obtain a damage reason query encoding matrix and a damage reason value encoding matrix; input each damaged drug status image local semantic feature matrix in the set of the damage reason query encoding matrix, the damage reason value encoding matrix, and the damaged drug status image local semantic feature matrix into a cross-modal prompt information encoder based on a transformer structure to obtain a set of damage information-damage status cross-modal prompt information semantic encoding matrices, which is expressed by the formula:

[0054]

[0055] Wherein, and respectively represent a query embedding matrix and a value embedding matrix, and respectively represent a damage reason query encoding matrix and a damage reason value encoding matrix, is the feature scale value of the damaged drug status image local semantic feature matrix, represents a normalization exponential function, represents the and the damage information-damage status cross-modal prompt information semantic encoding matrix between them.

[0056] That is, first, through a linear mapping method, the reported loss reason self-correlation semantic encoding matrix is converted into a reported loss reason query encoding matrix and a reported loss reason value encoding matrix to meet the input form requirements of the attention mechanism. Among them, the reported loss reason query encoding matrix is used to query the local semantic features related to the reported loss reason in the reported loss drug status image. For example, if the reported loss reason input by the user is "packaging damaged", the reported loss reason query encoding matrix can be used to focus on querying the image areas related to "packaging damaged", such as edge cracks, tear marks, etc. The reported loss reason value encoding matrix stores the original reported loss reason information to be fused for use in subsequent interactive analysis. Furthermore, based on the Transformer structure, cross-modal encoding is performed on the reported loss reason query encoding matrix, the reported loss reason value encoding matrix, and each local semantic feature matrix of the reported loss drug status image. Through the self-attention mechanism of the Transformer structure, feature fusion and interactive analysis of cross-modal data are realized, effectively capturing the deep semantic connection between the reported loss reason and the features of the reported loss drug status image, generating a set of semantic encoding matrices of the reported loss information-reported loss status cross-modal prompt information, and further guiding subsequent feature interaction optimization based on this.

[0057] Specifically, the cross-modal interaction guiding and optimizing subunit 1542 is used to: input the set of semantic encoding matrices of the reported loss information-reported loss status cross-modal prompt information into the information gating unit based on the decoder to obtain a set of cross-modal semantic interaction attention weights of the reported loss information-reported loss status; input the set of cross-modal semantic interaction attention weights of the reported loss information-reported loss status, the reported loss reason self-correlation semantic encoding matrix, and the set of local semantic feature matrices of the reported loss drug status image into the cross-modal interaction optimization unit to obtain the reported loss information-reported loss status semantic alignment encoding feature map, which is expressed by the formula:

[0058]

[0059] Among them, represents the decoder, represents the decoding weight matrix, 、 、 and represent the first, second, rd, and th cross-modal semantic interaction attention weights of the reported loss information-reported loss status, represents element-wise multiplication by position, represents feature concatenation, represents the reported loss information-reported loss status semantic alignment encoding feature map.

[0060] Here, the information gating unit learns the feature importance of the semantic encoding matrix of each loss report information - loss report status cross - modal prompt information through a decoder, and accordingly performs weight allocation to generate a set of cross - modal semantic interaction attention weights for the loss report information - loss report status, so as to effectively highlight the important semantic connection between the loss report drug status image and the loss report reason during the subsequent feature interaction process, while suppressing the interference of irrelevant information. Then, through element - wise multiplication by position, direct interaction is carried out between the self - correlation semantic encoding matrix of the loss report reason and the set of local semantic feature matrices of the loss report drug status image, so as to capture the correlation information at the same semantic feature space position of the two, and the interaction features between the loss report reason and the loss report drug status image are weighted and aggregated with the attention weights generated in the above process, thereby enhancing the expression ability of the features and generating a feature representation of the loss report drug status image incorporating the semantic information of the loss report reason, that is, the loss report information - loss report status semantic alignment encoding feature map.

[0061] In the recovery module 150, the loss report reason confirmation unit 155 is used to determine whether the input loss report reason is incorrect based on the loss report information - loss report status semantic alignment encoding feature map. In a specific example of the present application, the loss report reason confirmation unit 155 is configured to: input the loss report information - loss report status semantic alignment encoding feature map into a loss report reason auxiliary confirmation module based on a classifier to obtain a confirmation result, and the confirmation result is used to indicate whether the input loss report reason is incorrect. Specifically, during the training process of the classifier, it receives a large number of loss report information and corresponding drug status images as training samples, and is trained through supervised learning to learn the semantic correspondence between the loss report information and the drug loss report status, and continuously optimizes the model parameters through the backpropagation algorithm to minimize the classification error, thereby improving the accuracy of loss report reason confirmation. In practical applications, after receiving the loss report information - loss report status semantic alignment encoding feature map, the classifier performs feature learning and classification mapping on the loss report information - loss report status semantic alignment encoding feature map by combining the semantic correspondence between the loss report reason and the drug status learned during the training process, so as to output a final loss report reason confirmation result, which is used to indicate whether the loss report reason input by the user matches the drug status. If the confirmation result indicates that the loss report reason is not established, the user will be prompted to re - enter or further verification will be required, thereby ensuring the accuracy and compliance of loss report drug management. Through this intelligent verification mechanism, management loopholes caused by human errors can be effectively reduced, and the safety and efficiency of special drug management can be improved.

[0062] In a preferred example of the present application, inputting the loss report information - loss report status semantic alignment encoding feature map into a loss report reason auxiliary confirmation module based on a classifier to obtain a confirmation result includes:

[0063] First, expand the loss-reporting information-loss-reporting status semantic alignment encoding feature map into a loss-reporting information-loss-reporting status semantic alignment encoding feature vector;

[0064] Then, calculate the distance between each pair of eigenvalues of the loss-reporting information-loss-reporting status semantic alignment encoding feature vector, such as the L2 distance, and take the square root of the distance to obtain a loss-reporting information-loss-reporting status semantic alignment encoding distance representation matrix, that is:

[0065]

[0066] Among them, represents the loss-reporting information-loss-reporting status semantic alignment encoding feature vector, and respectively represent the eigenvalues at the th position and the th position in the loss-reporting information-loss-reporting status semantic alignment encoding feature vector, is a distance metric function, is the element value at the th position in the loss-reporting information-loss-reporting status semantic alignment encoding distance representation matrix;

[0067] Then, obtain the loss-reporting information-loss-reporting status semantic alignment encoding self-correlation matrix of the loss-reporting information-loss-reporting status semantic alignment encoding feature vector, that is:

[0068]

[0069] Among them, represents the loss-reporting information-loss-reporting status semantic alignment encoding self-correlation matrix, is a row vector;

[0070] Secondly, multiply the loss-reporting information-loss-reporting status semantic alignment encoding feature vector by the loss-reporting information-loss-reporting status semantic alignment encoding distance representation matrix to obtain a loss-reporting information-loss-reporting status semantic alignment encoding first-level mapping vector, that is:

[0071]

[0072] Among them, represents the loss-reporting information-loss-reporting status semantic alignment encoding first-level mapping vector, represents the loss-reporting information-loss-reporting status semantic alignment encoding distance representation matrix;

[0073] Furthermore, multiply the semantic alignment coding first-level mapping vector of the loss-reporting information-loss-reporting status by the matrix product of the semantic alignment coding distance representation matrix of the loss-reporting information-loss-reporting status and the self-correlation matrix of the semantic alignment coding of the loss-reporting information-loss-reporting status for matrix multiplication to obtain the semantic alignment coding multi-level mapping vector of the loss-reporting information-loss-reporting status, that is:

[0074]

[0075] where represents the semantic alignment coding multi-level mapping vector of the loss-reporting information-loss-reporting status;

[0076] Finally, dot-multiply the semantic alignment coding multi-level mapping vector of the loss-reporting information-loss-reporting status with the semantic alignment coding associated eigenvector (interpolate or fill with zeros if the eigenvalues are insufficient) composed of the eigenvalues of the self-correlation matrix of the semantic alignment coding of the loss-reporting information-loss-reporting status to obtain an optimized semantic alignment coding feature vector of the loss-reporting information-loss-reporting status; and, pass the optimized semantic alignment coding feature vector of the loss-reporting information-loss-reporting status through the loss-reporting reason auxiliary confirmation module based on a classifier to obtain a confirmation result.

[0077] That is, considering that the semantic coding vector of the loss-reporting reason and the semantic coding feature map of the loss-reporting drug status image respectively represent the text semantic coding features of the loss-reporting reason and the image semantic features of the loss-reporting drug image, during cross-modal interaction optimization coding, insufficient correspondence of the hint gating of cross-modal features will also lead to the lack of aggregation feature instance determination of the semantic alignment coding feature map of the loss-reporting information-loss-reporting status, thus affecting the accuracy of the confirmation result obtained through the loss-reporting reason auxiliary confirmation module based on a classifier.

[0078] Therefore, through the linear target mapping representation of the similarity distance representation matrix of the semantic alignment coding feature vector of the loss-reporting information-loss-reporting status expanded based on the semantic alignment coding feature map of the loss-reporting information-loss-reporting status, perform a quadratic target mapping representation of the complete similarity instantiation of the self-correlation of the semantic alignment coding feature map of the loss-reporting information-loss-reporting status at a multi-level distribution level, and compensate for the negative influence factor of association mismatch through an association fusion kernel bias to improve the eigenvalue instance determination degree of the semantic alignment coding feature map of the loss-reporting information-loss-reporting status under similarity constraints, that is, the significance degree of the eigenvalue as an instance for classification regression determination, and improve the accuracy of the confirmation result obtained by the semantic alignment coding feature map of the loss-reporting information-loss-reporting status through the loss-reporting reason auxiliary confirmation module based on a classifier.

[0079] In summary, an intelligent management system applied to a special medicine cabinet according to an embodiment of the present application is described, which includes a login module, a medicine adding module, a medicine taking module, a medicine returning module, a recycling module, an inventory module, a query module, an account management module, and an exit module. Among them, when the recycling module performs medicine recycling, it uses a camera to collect images of damaged medicines, and through cross-modal semantic interaction analysis of the damaged medicine images and the reported damage reasons input by the user, it intelligently identifies whether the medicine status conforms to the reported damage reasons and confirms whether there is an error in inputting the reported damage reasons of the medicines. In this way, the accuracy and efficiency of damaged medicine management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special medicines can be ensured.

[0080] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present invention to necessarily adopt the above specific details to be implemented.

Claims

1. An intelligent management system applied to a special medicine cabinet, characterized in that, Including: A login module for performing identity authentication; A medicine adding module for inputting medicine information, where the medicine information includes medicine name, packaging specification, medicine approval number, medicine expiration date, medicine traceability code, number of medicine adding layers, and storage grid position code; A medicine dispensing module for extracting medicines from the storage grids of the medicine special cabinet based on the medicine dispensing information and updating the medicine inventory; A medicine return module for performing a medicine return operation, where the medicine return operation includes selecting the storage grids where medicines can be returned and completing the medicine return operation after determining to close the drawers of the storage grids; A recycling module for performing medicine recycling and medicine write-off, where the medicine write-off includes inputting the reason for write-off and confirming whether the input reason for write-off is incorrect; An inventory module for taking inventory of all medicine information in the medicine special cabinet; A query module for querying medicine information and medicine storage information; An account management module for performing account management; An exit module for exiting the system; Among them, the recycling module includes: A damaged medicine image acquisition unit for acquiring damaged medicine images collected by a camera; A medicine status feature extraction unit for extracting the medicine status features of the damaged medicine images to obtain a damaged medicine status image semantic coding feature map; A damaged reason semantic coding unit for performing semantic coding on the input damaged reason to obtain a damaged reason semantic coding vector; A cross-modal interaction unit for performing damaged information-damaged status cross-modal interaction on the damaged reason semantic coding vector and the damaged medicine status image semantic coding feature map to obtain a damaged information-damaged status semantic alignment coding feature map; A damaged reason confirmation unit for determining whether the input damaged reason is incorrect based on the damaged information-damaged status semantic alignment coding feature map.

2. The intelligent management system applied to the special medicine cabinet according to claim 1, characterized in that, The medicine status feature extraction unit is used for: Inputting the damaged medicine images into a medicine status image feature extractor based on the DynamicViT model to obtain the damaged medicine status image semantic coding feature map.

3. The intelligent management system applied to the special medicine cabinet according to claim 2, wherein, The damaged reason semantic coding unit is used for: Using a semantic encoder based on the Bert model to perform semantic coding on the damaged reason to obtain the damaged reason semantic coding vector.

4. The intelligent management system applied to the special medicine cabinet according to claim 3, characterized in that, The cross-modal interaction unit includes: A local fine-grained semantic interaction subunit for performing local-scale fine-grained semantic interaction on the damaged reason semantic coding vector and the damaged medicine status image semantic coding feature map to obtain a set of damaged information-damaged status cross-modal prompt information semantic coding matrices; A cross-modal interaction guidance optimization subunit for guiding the damaged reason semantic coding vector and the damaged medicine status image semantic coding feature map to perform cross-modal interaction optimization based on the set of damaged information-damaged status cross-modal prompt information semantic coding matrices to obtain the damaged information-damaged status semantic alignment coding feature map.

5. The intelligent management system applied to the special medicine cabinet according to claim 4, characterized in that, The local fine-grained semantic interaction subunit includes: An autocorrelation coding secondary subunit for performing autocorrelation coding on the damaged reason semantic coding vector to obtain a damaged reason autocorrelation semantic coding matrix; A feature decoupling secondary subunit for performing feature decoupling on the semantic encoding feature map of the damaged drug status image along the channel dimension to obtain a set of local semantic feature matrices of the damaged drug status image; A semantic interaction secondary subunit for performing cross-modal semantic interaction between the self-correlated semantic encoding matrix of the damage cause and each local semantic feature matrix of the damaged drug status image in the set of local semantic feature matrices of the damaged drug status image to obtain a set of cross-modal prompt information semantic encoding matrices of the damage information-damage status; 6. The intelligent management system applied to the special medicine cabinet according to claim 5, characterized in that, The semantic interaction secondary subunit is used for: Performing a linear transformation on the self-correlated semantic encoding matrix of the damage cause to obtain a damage cause query encoding matrix and a damage cause value encoding matrix; Respectively inputting the damage cause query encoding matrix, the damage cause value encoding matrix, and each local semantic feature matrix of the damaged drug status image in the set of local semantic feature matrices of the damaged drug status image into a cross-modal prompt information encoder based on a transformer structure to obtain a set of cross-modal prompt information semantic encoding matrices of the damage information-damage status; 7. The intelligent management system applied to the special medicine cabinet according to claim 6, wherein The cross-modal interaction guidance optimization subunit is used for: Inputting the set of cross-modal prompt information semantic encoding matrices of the damage information-damage status into an information gating unit based on a decoder to obtain a set of cross-modal semantic interaction attention weights of the damage information-damage status; Inputting the set of cross-modal semantic interaction attention weights of the damage information-damage status, the self-correlated semantic encoding matrix of the damage cause, and the set of local semantic feature matrices of the damaged drug status image into a cross-modal interaction optimization unit to obtain a semantic alignment encoding feature map of the damage information-damage status; 8. The intelligent management system applied to the special medicine cabinet according to claim 7, characterized in that The damage cause confirmation unit is used for: Inputting the semantic alignment encoding feature map of the damage information-damage status into a damage cause auxiliary confirmation module based on a classifier to obtain a confirmation result, and the confirmation result is used to indicate whether the input damage cause is incorrect.

Images