Intelligent management system applied to special medicine cabinet

Abstract

The invention relates to the technical field of medicine management, and particularly discloses an intelligent management system applied to a special medicine cabinet, which comprises a login module, a medicine adding module, a medicine taking module, a medicine returning module, a recovery module, a checking module, a query module, an account management module and an exit module. Wherein when the recovery module is used for recovering drugs, a camera is used for collecting images of damaged drugs, and cross-modal semantic interaction analysis is carried out on the images of the damaged drugs and damage reasons input by a user, so that whether the states of the drugs conform to the damage reasons of the drugs or not is intelligently recognized, and whether the input of the damage reasons of the drugs is wrong or not is confirmed. Through the mode, the accuracy and the efficiency of management of the damaged medicines can be remarkably improved, human errors and management vulnerabilities are reduced, and the safety and the compliance of the special medicines are ensured.

Description

Technical Field

[0001] The present application relates to the technical field of drug management and processing, and more specifically, to an intelligent management system applied to special drug cabinets. Background Art

[0002] The pharmaceutical industry is an important field related to public health, in which the accuracy and safety of drug management are particularly important. In medical institutions such as hospitals, clinics and pharmacies, special drugs have higher requirements for management and tracking due to their special characteristics (such as high value, strict storage conditions and usage regulations). Traditional drug management systems rely on manual operations, including drug addition, drug withdrawal, drug return, recycling, inventory and other links. This method is not only time-consuming and labor-intensive, but also prone to errors and cannot ensure the accuracy and timeliness of information. Especially in the management of special drugs, since these drugs usually have high value, special storage conditions and strict safety supervision requirements, incorrect handling may lead to serious consequences, including economic losses, medical accidents and even legal risks.

[0003] In recent years, with the continuous advancement of information technology, intelligent management systems have gradually been applied to the field of drug management, aiming to improve the degree of automation and intelligence of drug management. However, most solutions on the market are limited to simple barcode or QR code scanning, and still remain at the level of simple information entry and query. They can only track and record basic operations such as storage and retrieval of drugs, and lack intelligent identification and verification mechanisms. For example, in the recovery management of damaged drugs, accurately recording the reasons for drug damage is crucial to ensuring drug safety, optimizing inventory management and controlling costs. However, existing intelligent management systems often lack intelligent verification mechanisms for the reasons for damage, and cannot automatically judge the rationality and accuracy of the input information, resulting in certain management loopholes and security risks.

[0004] Therefore, an optimized intelligent management system for special medicine cabinets is expected. Summary of the invention

[0005] In order to solve the above-mentioned technical problems, the present application is proposed. The embodiment of the present application provides an intelligent management system applied to special medicine cabinets, which includes a login module, a medicine adding module, a medicine taking module, a medicine return module, a recycling module, an inventory module, a query module, an account management module and an exit module. Among them, when the recycling module is recycling medicines, it uses a camera to collect images of damaged medicines, and through cross-modal semantic interactive analysis of the images of damaged medicines and the reasons for damage input by the user, it can intelligently identify whether the status of the medicine is consistent with its cause of damage, and confirm whether the reason for the damage of the medicine is entered incorrectly. In this way, the accuracy and efficiency of damaged medicine management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special medicines can be ensured.

[0006] Accordingly, according to one aspect of the present application, there is provided an intelligent management system for special medicine cabinets, comprising: Login module, used for identity authentication; The drug adding module is used to input drug information, including drug name, packaging specifications, drug approval number, drug expiration date, drug traceability code, number of drug adding layers and drug storage grid location code; The medicine fetching module is used to extract medicines from the medicine storage compartment of the medicine cabinet based on the medicine fetching information and update the medicine inventory; A medicine return module, used to execute a medicine return operation, wherein the medicine return operation includes selecting a medicine storage compartment for return and completing the medicine return operation after determining to close the drawer of the medicine storage compartment; A recycling module is used to recycle drugs and report drug damage, wherein the drug damage report includes inputting the cause of the damage report and confirming whether the cause of the damage report is input incorrectly; Inventory module, used to inventory all drug information in the drug cabinet; A query module, used to query drug information and drug storage information; Account management module, used for account management; Exit module, used to exit the system.

[0007] In the above-mentioned intelligent management system applied to special cabinets for special medicines, the recycling module includes: a damaged drug image acquisition unit, which is used to obtain damaged drug images collected by a camera; a drug status feature extraction unit, which is used to extract drug status features of the damaged drug image to obtain a semantic coding feature map of the damaged drug status image; a damage cause semantic coding unit, which is used to semantically encode the input damage cause to obtain a damage cause semantic coding vector; a cross-modal interaction unit, which is used to perform damage information-damage status cross-modal interaction on the damage cause semantic coding vector and the damaged drug status image semantic coding feature map to obtain a damage information-damage status semantic alignment coding feature map; a damage cause confirmation unit, which is used to determine whether the damage cause is input incorrectly based on the damage information-damage status semantic alignment coding feature map.

[0008] Compared with the prior art, the intelligent management system for special medicine cabinets provided by the present application includes a login module, a medicine adding module, a medicine taking module, a medicine return module, a recycling module, an inventory module, a query module, an account management module and an exit module. Among them, when recycling medicines, the recycling module uses a camera to collect images of damaged medicines, and through cross-modal semantic interactive analysis of the images of damaged medicines and the reasons for damage input by the user, it can intelligently identify whether the status of the medicine is consistent with its cause of damage, and confirm whether the reason for the damage of the medicine is entered incorrectly. In this way, the accuracy and efficiency of damaged medicine management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special medicines can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 The block diagram is a smart management system applied to special medicine cabinets according to an embodiment of the present application.

[0011] Figure 2 It is a block diagram of a recycling module in an intelligent management system applied to special medicine cabinets according to an embodiment of the present application.

[0012] Figure 3 This is a schematic diagram of the data flow of a recycling module in an intelligent management system applied to special medicine cabinets according to an embodiment of the present application.

[0013] Figure 4 The block diagram is a cross-modal interaction unit in an intelligent management system applied to special medicine cabinets according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0015] In response to the technical problems described in the above background technology, the present application proposes an intelligent management system for special medicine cabinets. Figure 1 FIG. 1 is a block diagram of an intelligent management system applied to a special medicine cabinet according to an embodiment of the present application. Figure 1As shown, the intelligent management system 100 applied to the special medicine cabinet includes: a login module 110 for identity authentication; a drug adding module 120 for entering drug information, the drug information including drug name, packaging specification, drug approval number, drug expiration date, drug traceability code, drug adding layer number and drug storage compartment position code; a drug taking module 130 for extracting drugs from the drug storage compartment of the drug cabinet based on the drug taking information and updating the drug inventory; a drug return module 140 for performing a drug return operation, the drug return operation including selecting a drug storage compartment to which the drug can be returned and completing the drug return operation after determining to close the drawer of the drug storage compartment; a recycling module 150 for drug recycling and drug loss reporting, the drug loss reporting including inputting the loss reporting reason and confirming whether the loss reporting reason is entered incorrectly; an inventory module 160 for inventorying all drug information of the drug cabinet; a query module 170 for querying drug information and drug storage information; an account management module 180 for account management; and an exit module 190 for exiting the system.

[0016] Specifically, the operating environment of the intelligent management system applied to special medicine cabinets described in this application is Android 11. The system integrates multiple functional modules and realizes the safe management, efficient distribution and precise tracking of special medicines through the synergy between various functional modules, while providing users with a convenient operation experience.

[0017] Specifically, the login module 110 is used for identity authentication. It should be understood that users need to authenticate their identities before using the system. The login module 110 includes a first level of verification - account password or IC card verification, and a second level of verification - the application of fingerprint or face recognition technology. Through this dual verification mechanism, it is ensured that only authorized personnel can access the system, greatly enhancing security. After the dual verification is successful, the user can enter the main menu page. The main menu page contains entrances to the functions of adding medicine, taking medicine, returning medicine, recycling, inventory, inquiry, account management, operation records, inventory records and exit, which meet the needs of different scenarios.

[0018] Specifically, the dosing module 120 is used to enter drug information, which includes drug name, packaging specifications, drug approval number, drug expiration date, drug traceability code, number of drug adding layers and drug storage compartment position code. On the main menu page, after the user clicks the "dosing" button, the secondary verification login page of the dosing module 120 will be entered, requiring the operator to log in again with "swipe card or username password + fingerprint or face recognition" double verification to confirm his identity. After completing the verification, the operator will enter the dosing interface, where the operator can scan or manually enter the drug information to be added to the smart cabinet through the device, and then click the "confirm" button. The system will display the number of layers that can be added to the drug and the number of remaining digits that can be added to each layer, so that the operator can intuitively select the appropriate storage compartment. After the operator clicks the number of layers with remaining digits, the system will enter the drug information entry interface and the drug storage compartment page of that layer. At the same time, the smart cabinet will automatically open the drawer of the corresponding number of layers. At this point, the operator can scan or manually enter the drug information through the device, and then click the "OK" button to put the drug into the storage compartment that displays "Available". The status of the storage compartment will change to "Put", indicating that the dosing operation of a single drug has been completed. Repeat the above steps multiple times to complete the dosing of multiple drugs. After the dosing is completed, click the "Drug Dosing Completed" button, the system will close the drawer and end the dosing process, and update the relevant data to maintain the accuracy of the inventory information.

[0019] Specifically, the medicine collection module 130 is used to extract medicines from the medicine storage compartment of the medicine cabinet based on the medicine collection information and update the medicine inventory. On the main menu page, after the user clicks the "Get Medicine" button, he also needs to go through the above-mentioned "swipe card or username and password + fingerprint or face recognition" secondary verification step. After the verification is passed, enter the medicine collection type selection page, and the user can choose two different ways of collecting medicines: "Prescription Collection" and "Allocation Collection": "Prescription Collection" is suitable for collecting specific medicines according to the prescription issued by the doctor. Click "Prescription Collection" to select the prescription and then enter the medicine collection page; click "Allocation Collection" to directly enter the medicine collection page. After entering the medicine collection page, the operator can scan or manually enter the medicine information through the device, and then click the "Confirm" button. The system will display the number of layers where the medicine can be taken and the inventory of the medicine on each layer. The operator can select the number of layers where the medicine is located as needed and confirm the number of medicines to be taken. Subsequently, the system will update the inventory of the medicine and complete the medicine collection operation. Specifically, after the user clicks on the layer with inventory, the system will enter the drug storage grid page on that layer, and the smart cabinet will automatically open the drawer of the corresponding layer. The operator can take the medicine in the storage grid that shows "available", and the status of the storage grid will change to "taken". After taking the medicine, the operator can click the "Drug Take Complete" button, and the system will close the drawer and complete the drug taking operation. At this time, in addition to closing the drawer, the system will also refresh the inventory record synchronously to ensure the consistency of information.

[0020] Specifically, the drug return module 140 is used to perform a drug return operation, which includes selecting a drug storage grid that can be returned and completing the drug return operation after determining to close the drawer of the drug storage grid. On the main menu page, after the user clicks the "Return Drug" button, it is also necessary to go through the above-mentioned "swipe card or username and password + fingerprint or face recognition" secondary verification step. After the verification is passed, the drug return page is entered, and the operator can scan or manually enter the drug information through the device. After entering the returned drug information, the system will display the number of layers where the drug can be returned and the number of remaining digits on each layer. After the operator clicks the number of layers with remaining digits, the system will enter the drug information entry interface and the drug storage grid page of the layer, and the smart cabinet will automatically open the drawer of the corresponding number of layers. The operator can scan or manually enter the drug information through the device, and then click the "OK" button to put the drug into the drug storage grid that displays "can be placed". At this time, the status of the drug storage grid will become "placed", indicating that the drug return operation of a single drug has been completed. Repeat the above steps multiple times to complete the return of multiple drugs. After the return is completed, click the "Return Completed" button, the system will close the drawer and complete the return operation. At the same time, the system will also adjust the inventory data accordingly to reflect the latest changes.

[0021] Specifically, the recycling module 150 is used for drug recycling and drug damage reporting, and the drug damage reporting includes inputting the cause of the damage and confirming whether the cause of the damage is entered incorrectly. On the main menu page, after the user clicks the "Recycle" button, the recycling layer number and drug damage selection page will be entered. On this page, the user needs to select the layer number of the drug to be recycled and click the corresponding recycling layer button. At this time, the system will enter the drug recycling function page and automatically open the drawer of this layer. The user can perform drug recycling operations according to the page prompts. For example, after the operator scans or manually enters the drug information through the device, clicks the "Search" button to query the drug. Then, in the query result list, mark the drug to be recycled, click the "Recycle" button, and complete the recycling of a single drug. At the same time, the information of the drug that has been recycled this time can be viewed in the recycling list. By repeating the above operations, the recycling of multiple drugs can be completed. After clicking the "Recycling Complete" button, the system will close the drawer and complete the drug recycling operation.

[0022] In addition, in response to possible drug damage, when performing the recycling task, the user needs to click the "Drug Damage Report" button on the drug recycling function page. The system will enter the drug damage function page. After the operator selects the layer number of the drug to be reported, scans the device or manually enters the drug information, and clicks the "Search" button to search for the drug. In the query result list, mark the drugs that need to be reported, click the "Report Damage" button, fill in the reason for the loss, and complete the loss of a single drug. At the same time, users can also view the information of the drugs that have been reported in the loss list. By repeating the above operations, you can complete the loss of multiple drugs. After clicking the "Damage Report Completed" button, the system will close the drawer and complete the drug loss operation.

[0023] Specifically, the inventory module 160 is used to inventory all drug information in the medicine cabinet. After the user clicks the "Inventory" button on the main menu page, he enters the inventory function page and selects the number of layers to be inventoried. The system will automatically open the drawer of that layer, and the page will display the list of all drug information on that layer by default. On this basis, the operator can modify the drug information or perform surplus or shortage operations on the drugs according to the actual situation. Click the "Inventory Record" button to view the drug records that have been inventoried this time. After clicking the "Inventory Completed" button, the system will close the drawer of that layer and complete the inventory operation.

[0024] Specifically, the query module 170 is used to query drug information and drug storage information. The user clicks the "Query" button on the main menu page to enter the drug query page. On the drug query page, after entering the drug name, click the "Search button" to query the corresponding drug information and storage information in the smart cabinet.

[0025] In addition, the system also supports querying operation records and inventory records. On the main menu page, the user clicks the "Operation Record" button to enter the operation record query page. On this page, you can filter and view operation records by entering the drug name, selecting the operation type, and selecting the operation time. Operation types include adding medicine, taking medicine, returning medicine, recycling, and reporting losses. On the main menu page, the user clicks the "Inventory Record" button to enter the inventory record query page. On this page, you can filter and view inventory records by entering the drug name, selecting the inventory type, and selecting the inventory time. Inventory types include inventory surplus, inventory shortage, and modification.

[0026] Specifically, the account management module 180 is used to manage accounts. On the main menu page, the user clicks the "Account Management" button to enter the account list page. On this page, the user can click the "Enter Information" button in the list to enter the personal biometric information and IC card information entry interface. The biometric information includes face information and fingerprint information. Click the "Enter Information" button to enter the information of a new user; click the "Re-enter Information" button to reset and update the information of an old user.

[0027] Specifically, the exit module 190 is used to exit the system. On the main menu page, the user clicks the "Exit" button to exit the menu page and return to the login page.

[0028] In particular, in response to the defects of existing intelligent drug management systems in the management of damaged and recovered drugs, the intelligent management system applied to special drug cabinets described in this application further optimizes the management process of damaged drugs. Specifically, when recovering damaged drugs, first, the image of the damaged drugs is collected by a camera, and then the image of the damaged drugs is analyzed using deep learning-based image processing technology to extract the status characteristics of the damaged drugs, and a cross-modal semantic interactive analysis is performed with the damage reasons input by the user, so as to intelligently identify whether the drug status is consistent with its damage reason and confirm whether the drug damage reason is entered incorrectly. In this way, the accuracy and efficiency of damaged drug management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special drugs can be ensured.

[0029] Figure 2 It is a block diagram of a recycling module in an intelligent management system applied to special medicine cabinets according to an embodiment of the present application. Figure 3 FIG. 1 is a schematic diagram of data flow of a recycling module in an intelligent management system for special medicine cabinets according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the recycling module 150 includes: a damaged drug image acquisition unit 151, which is used to obtain a damaged drug image collected by a camera; a drug status feature extraction unit 152, which is used to extract the drug status feature of the damaged drug image to obtain a semantic coding feature map of the damaged drug status image; a damage cause semantic coding unit 153, which is used to semantically encode the input damage cause to obtain a damage cause semantic coding vector; a cross-modal interaction unit 154, which is used to perform damage information-damage status cross-modal interaction on the damage cause semantic coding vector and the damaged drug status image semantic coding feature map to obtain a damage information-damage status semantic alignment coding feature map; a damage cause confirmation unit 155, which is used to determine whether the damage cause is input incorrectly based on the damage information-damage status semantic alignment coding feature map.

[0030] In the recycling module 150, the damaged drug image acquisition unit 151 is used to acquire images of damaged drugs acquired by a camera. It should be understood that images provide a direct and intuitive way to verify the status of drugs. By acquiring images of damaged drugs, the appearance, packaging integrity, label information, etc. of the drugs can be clearly seen, thereby providing basic visual information for the subsequent confirmation of the cause of the damage.

[0031] In the recycling module 150, the drug status feature extraction unit 152 is used to extract the drug status features of the damaged drug image to obtain a semantic coding feature map of the damaged drug status image. In a specific example of the present application, the drug status feature extraction unit 152 is used to: input the damaged drug image into a drug status image feature extractor based on the DynamicViT model to obtain the semantic coding feature map of the damaged drug status image. That is, in order to fully extract the status features of the drug from the damaged drug image, the present application uses the DynamicViT model with excellent performance in the field of computer vision to construct a drug status image feature extractor. Those of ordinary skill in the art should know that the DynamicViT model is an efficient visual Transformer model based on dynamic Token sparsification, which can effectively extract discriminative features from images. Compared with traditional convolutional neural networks (CNNs), it performs particularly well in processing long-distance dependencies in images. In addition, when performing image processing, the traditional visual Transformer model mainly divides the input image into multiple independent small blocks (called patches or tokens), and uses the Transformer's self-attention mechanism to capture the dependencies between multiple image blocks, thereby realizing feature encoding of the global visual information of the image. The DynamicViT model, by introducing dynamic Token sparsification technology, can dynamically identify the image content complexity of each Token and cut off redundant Tokens, further improving the efficiency and accuracy of feature extraction. Based on this, in the technical solution of the present application, by using the DynamicViT model to process the damaged drug image, the Transformer's self-attention mechanism can be used to simultaneously consider the global structure and local details of the damaged drug, capture the overall shape of the package and subtle signs of damage, thereby providing a more accurate state description for the subsequent confirmation of the cause of the damage, while effectively reducing the consumption of computing resources, reducing the model's computational complexity and improving data processing efficiency.

[0032] In the recovery module 150, the loss cause semantic encoding unit 153 is used to semantically encode the input loss cause to obtain a loss cause semantic encoding vector. Here, in order to realize the semantic interaction analysis between the damaged drug status and the loss cause, the present application further integrates natural language processing (NLP) technology. After the user inputs the loss cause, the present application semantically encodes the loss cause to map it to a high-dimensional semantic space and convert it into a numerical vector representation that can be understood by a computer. In a specific example of the present application, the loss cause semantic encoding unit 153 is used to: semantically encode the loss cause using a semantic encoder based on the Bert model to obtain the loss cause semantic encoding vector. It should be understood that the Bert model is a pre-trained language representation model, which uses a bidirectional Transformer architecture to understand the content meaning of each word in the loss cause in the overall context, can effectively capture the complex semantic connection between words, and dig out the deep contextual semantic information of the loss cause, thereby achieving an in-depth understanding and accurate representation of the loss cause, and generating the loss cause semantic encoding vector.

[0033] In the recycling module 150, the cross-modal interaction unit 154 is used to perform cross-modal interaction of the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image to obtain a semantic alignment coding feature map of the cause of damage and the damage status. That is, by interactively analyzing the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image, the potential correlation between the cause of damage and the drug status is revealed, thereby verifying the rationality of the cause of damage.

[0034] Figure 4 FIG. 1 is a block diagram of a cross-modal interaction unit in an intelligent management system for special medicine cabinets according to an embodiment of the present application. Figure 4 As shown, the cross-modal interaction unit 154 includes: a local fine-grained semantic interaction sub-unit 1541, which is used to perform local-scale fine-grained semantic interaction on the loss cause semantic coding vector and the damaged drug status image semantic coding feature map to obtain a set of loss information-damage status cross-modal prompt information semantic coding matrices; a cross-modal interaction guidance optimization sub-unit 1542, which is used to guide the loss cause semantic coding vector and the damaged drug status image semantic coding feature map to perform cross-modal interaction optimization based on the set of loss information-damage status cross-modal prompt information semantic coding matrices to obtain the loss information-damage status semantic alignment coding feature map.

[0035] Specifically, the local fine-grained semantic interaction sub-unit 1541 includes: an autocorrelation encoding secondary sub-unit, which is used to perform autocorrelation encoding on the semantic encoding vector of the cause of damage to obtain an autocorrelation semantic encoding matrix of the cause of damage; a feature decoupling secondary sub-unit, which is used to perform feature decoupling along the channel dimension on the semantic encoding feature map of the damaged drug status image to obtain a set of local semantic feature matrices of the damaged drug status image; a semantic interaction secondary sub-unit, which is used to perform cross-modal semantic interaction on the autocorrelation semantic encoding matrix of the cause of damage and each local semantic feature matrix of the damaged drug status image in the set of local semantic feature matrices of the damaged drug status image to obtain a set of semantic encoding matrices of the damage information-damage status cross-modal prompt information.

[0036] In a specific example of the present application, the calculation process of the autocorrelation coding secondary subunit can be expressed as follows:

[0037] in, represents the semantic encoding vector of the damage reporting reason, represents the matrix multiplication operation, represents the transpose of a vector, Represents the autocorrelation semantic encoding matrix of the damage reporting reason.

[0038] That is, firstly, by calculating the product between the semantic coding vector of the cause of damage and its own transposed vector, the concept of outer product in matrix operation is utilized to reveal the autocorrelation of the cause of damage, and the autocorrelation semantic coding matrix of the cause of damage is obtained.

[0039] In a specific example of the present application, the calculation process of the characteristic decoupling secondary subunit can be expressed by the formula:

[0040] in, represents the semantic coding feature map of the damaged drug status image, Indicates feature decoupling, , , and They respectively represent the first, second, and third semantic encoding feature maps of the damaged drug status image along the channel dimension. and The local semantic feature matrix of the damaged drug status image, is the number of channels of the semantic encoding feature map of the damaged drug status image.

[0041] That is, by performing feature decoupling along the channel dimension on the semantic encoding feature map of the damaged drug status image, the model's attention to local details in the damaged drug status image is improved, and the feature discrimination is enhanced, thereby facilitating a more fine-grained cross-modal interaction analysis.

[0042] In a specific example of the present application, the semantic interaction secondary subunit is used to: perform a linear transformation on the damage cause autocorrelation semantic coding matrix to obtain a damage cause query coding matrix and a damage cause value coding matrix; input each damaged drug status image local semantic feature matrix in the set of the damage cause query coding matrix, the damage cause value coding matrix and the damaged drug status image local semantic feature matrix into a cross-modal prompt information encoder based on a converter structure to obtain a set of the damage information-damage status cross-modal prompt information semantic coding matrices, which is expressed as follows:

[0043] in, and denote the query embedding matrix and the value embedding matrix respectively, and They represent the loss cause query coding matrix and the loss cause value coding matrix respectively. is the characteristic scale value of the local semantic feature matrix of the damaged drug status image, represents the normalized exponential function, Indicates the and stated The semantic encoding matrix of damage report information-damage report status cross-modal prompt information.

[0044] That is, firstly, by means of linear mapping, the damage cause autocorrelation semantic coding matrix is ​​converted into a damage cause query coding matrix and a damage cause value coding matrix, so that it meets the input form requirements of the attention mechanism. Among them, the damage cause query coding matrix is ​​used to query the local semantic features related to the damage cause in the damaged drug status image. For example, if the damage cause input by the user is "damaged packaging", the damage cause query coding matrix can be used to focus on querying image areas related to "damaged packaging", such as edge cracks, tear marks, etc. The damage cause value coding matrix saves the original damage cause information to be fused for use in subsequent interactive analysis. Furthermore, based on the converter structure, the damage cause query coding matrix, the damage cause value coding matrix and the local semantic feature matrix of each damaged drug status image are cross-modally encoded. The self-attention mechanism of the converter structure is used to realize feature fusion and interactive analysis of cross-modal data, effectively capture the deep semantic connection between the damage cause and the damaged drug status image features, generate a set of damage information-damage status cross-modal prompt information semantic coding matrices, and use this as a basis to further guide subsequent feature interaction optimization.

[0045] Specifically, the cross-modal interaction guidance optimization subunit 1542 is used to: input the set of the damage information-damage status cross-modal prompt information semantic coding matrix into the decoder-based information gating unit to obtain the set of damage information-damage status cross-modal semantic interaction attention weights; input the set of damage information-damage status cross-modal semantic interaction attention weights, the damage cause autocorrelation semantic coding matrix, and the set of the damaged drug status image local semantic feature matrix into the cross-modal interaction optimization unit to obtain the damage information-damage status semantic alignment coding feature map, which is expressed as follows:

[0046] in, Represents a decoder, represents the decoding weight matrix, , , and Indicates the first, second, and Damage report information-damage report status cross-modal semantic interaction attention weight, It means point multiplication by position. represents feature concatenation, Represents the damage information-damage status semantic alignment encoding feature map.

[0047] Here, the information gating unit learns the feature importance of each damage information-damage status cross-modal prompt information semantic encoding matrix through a decoder, and performs weight allocation accordingly to generate a set of damage information-damage status cross-modal semantic interaction attention weights, so that in the subsequent feature interaction process, it can effectively highlight the important semantic connection between the damaged drug status image and the damage cause, while suppressing the interference of irrelevant information. Afterwards, the direct interaction between the damage cause autocorrelation semantic encoding matrix and the damaged drug status image local semantic feature matrix is ​​performed by position point multiplication to capture the association information of the two at the same semantic feature space position, and the interaction features between the damage cause and the damaged drug status image are weighted and aggregated with the attention weights generated in the above process, thereby enhancing the feature expression ability and generating a damaged drug status image feature representation that incorporates the semantic information of the damage cause, that is, the damage information-damage status semantic alignment coding feature map.

[0048] In the recovery module 150, the damage cause confirmation unit 155 is used to determine whether the damage cause is input incorrectly based on the damage information-damage status semantic alignment coding feature map. In a specific example of the present application, the damage cause confirmation unit 155 is used to: input the damage information-damage status semantic alignment coding feature map into the classifier-based damage cause auxiliary confirmation module to obtain a confirmation result, and the confirmation result is used to indicate whether the damage cause is input incorrectly. Specifically, during the training process of the classifier, it receives a large amount of damage information and corresponding drug status images as training samples, and is trained by supervised learning to learn the semantic correspondence between the damage information and the drug damage status, and continuously optimizes the model parameters through the back-propagation algorithm to minimize the classification error, thereby improving the accuracy of the confirmation of the damage cause. In actual applications, after the classifier receives the damage information-damage status semantic alignment coding feature map, it performs feature learning and classification mapping on the damage information-damage status semantic alignment coding feature map by combining the semantic correspondence between the damage cause and the drug status learned during the training process to output the final damage cause confirmation result, which is used to indicate whether the damage cause entered by the user matches the drug status. If the confirmation result shows that the damage cause is not valid, the user will be prompted to re-enter or require further verification, thereby ensuring the accuracy and compliance of damaged drug management. Through this intelligent verification mechanism, management loopholes caused by human errors can be effectively reduced, and the safety and efficiency of special drug management can be improved.

[0049] In a preferred example of the present application, the damage information-damage status semantic alignment encoding feature map is passed through a classifier-based damage cause auxiliary confirmation module to obtain a confirmation result, including: First, the damage information-damage status semantic alignment coding feature graph is expanded into a damage information-damage status semantic alignment coding feature vector; Then, the distance between each pair of eigenvalues ​​of the damage information-damage status semantic alignment coding feature vector is calculated, such as the L2 distance, and the square root of the distance is taken to obtain the damage information-damage status semantic alignment coding distance representation matrix, that is:

[0050] in, represents the damage information-damage status semantic alignment encoding feature vector, and Respectively represent the first Location and The eigenvalues ​​of the positions, is the distance metric function, The damage information-damage status semantic alignment encoding distance representation matrix is The element value at position; Then, the damage information-damage status semantic alignment coding auto-association matrix of the damage information-damage status semantic alignment coding feature vector is obtained, that is:

[0051] in, Represents the damage information-damage status semantic alignment encoding self-association matrix, is a row vector; Secondly, the damage information-damage status semantic alignment coding feature vector is matrix-multiplied with the damage information-damage status semantic alignment coding distance representation matrix to obtain the damage information-damage status semantic alignment coding primary mapping vector, that is:

[0052] in, Indicates the first-level mapping vector of damage information-damage status semantic alignment encoding, A damage information-damage status semantic alignment encoding distance representation matrix is ​​represented; Furthermore, the first-level mapping vector of the damage information-damage status semantic alignment coding is matrix-multiplied with the matrix product of the damage information-damage status semantic alignment coding distance representation matrix and the damage information-damage status semantic alignment coding self-association matrix to obtain the damage information-damage status semantic alignment coding multi-level mapping vector, that is:

[0053] in, A multi-level mapping vector representing the damage information-damage status semantic alignment encoding; Finally, the multi-level mapping vector of the damage information-damage status semantic alignment coding is interpolated with the damage information-damage status semantic alignment coding associated eigenvector composed of the eigenvalues ​​of the damage information-damage status semantic alignment coding auto-association matrix (if the eigenvalue is insufficient, interpolation or zero padding is performed) to obtain the optimized damage information-damage status semantic alignment coding feature vector; and the optimized damage information-damage status semantic alignment coding feature vector is passed through a classifier-based damage cause auxiliary confirmation module to obtain a confirmation result.

[0054] That is, considering that the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image represent the text semantic coding features of the cause of damage and the image semantic features of the damaged drug image respectively, when performing cross-modal interactive optimization encoding, insufficient correspondence of the prompt gating of cross-modal features will also lead to the lack of aggregated feature instance determination of the damage information-damage status semantic alignment coding feature map, thereby affecting the accuracy of the confirmation result obtained by the classifier-based damage cause auxiliary confirmation module.

[0055] Therefore, through the linear target mapping representation of the similarity distance representation matrix of the damage information-damage status semantic alignment coding feature vector after the damage information-damage status semantic alignment coding feature map is expanded, the self-correlation complete similarity instantiation of the damage information-damage status semantic alignment coding feature map is represented by a secondary target mapping based on a multi-level distribution hierarchy, and the association mismatch negative influence factor is compensated by the association fusion kernel bias to improve the degree of instance determination of the eigenvalue of the damage information-damage status semantic alignment coding feature map under similarity constraints, that is, the significance of the eigenvalue as an instance for classification regression judgment, and improve the accuracy of the confirmation result obtained by the damage information-damage status semantic alignment coding feature map through the classifier-based damage cause auxiliary confirmation module.

[0056] In summary, according to the embodiment of the present application, an intelligent management system applied to a special medicine cabinet is explained, which includes a login module, a medicine adding module, a medicine taking module, a medicine return module, a recycling module, an inventory module, a query module, an account management module and an exit module. Among them, when the recycling module is recycling medicines, it uses a camera to collect images of damaged medicines, and through cross-modal semantic interactive analysis of the images of damaged medicines and the reasons for damage input by the user, it can intelligently identify whether the status of the medicine is consistent with its cause of damage, and confirm whether the reason for the damage of the medicine is entered incorrectly. In this way, the accuracy and efficiency of damaged medicine management can be significantly improved, human errors and management loopholes can be reduced, and the safety and compliance of special medicines can be ensured.

[0057] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to be implemented by adopting the above specific details.

Claims

1. An intelligent management system applied to special medicine cabinets, characterized in that: include: Login module, used for identity authentication; The drug adding module is used to input drug information, including drug name, packaging specifications, drug approval number, drug expiration date, drug traceability code, number of drug adding layers and drug storage grid location code; The medicine fetching module is used to extract medicines from the medicine storage compartment of the medicine cabinet based on the medicine fetching information and update the medicine inventory; A medicine return module, used to execute a medicine return operation, wherein the medicine return operation includes selecting a medicine storage compartment for return and completing the medicine return operation after determining to close the drawer of the medicine storage compartment; A recycling module is used to recycle drugs and report drug damage, wherein the drug damage report includes inputting the cause of the damage report and confirming whether the cause of the damage report is input incorrectly; Inventory module, used to inventory all drug information in the drug cabinet; A query module, used to query drug information and drug storage information; Account management module, used for account management; Exit module, used to exit the system.

2. The intelligent management system for special medicine cabinets according to claim 1 is characterized in that: The recycling module comprises: A damaged medicine image acquisition unit, used to acquire images of damaged medicines acquired by a camera; A drug status feature extraction unit, used to extract drug status features of the damaged drug image to obtain a semantic coding feature map of the damaged drug status image; A damage cause semantic coding unit, used for semantically coding the input damage cause to obtain a damage cause semantic coding vector; A cross-modal interaction unit, configured to perform a cross-modal interaction of the damage information-damage status on the damage cause semantic coding vector and the damaged drug status image semantic coding feature map to obtain a damage information-damage status semantic alignment coding feature map; The damage report reason confirmation unit is used to determine whether the damage report reason is input incorrectly based on the damage report information-damage report status semantic alignment coding feature map.

3. The intelligent management system for special medicine cabinets according to claim 2 is characterized in that: The drug status feature extraction unit is used to: The damaged drug image is input into a drug status image feature extractor based on the DynamicViT model to obtain a semantic encoding feature map of the damaged drug status image.

4. The intelligent management system for special medicine cabinets according to claim 3 is characterized in that: The damage reporting cause semantic encoding unit is used to: The cause of the damage report is semantically encoded using a semantic encoder based on the Bert model to obtain a semantic encoding vector of the cause of the damage report.

5. The intelligent management system for special medicine cabinets according to claim 4 is characterized in that: The cross-modal interaction unit includes: A local fine-grained semantic interaction subunit, used for performing fine-grained semantic interaction based on a local scale on the semantic coding vector of the damage cause and the semantic coding feature map of the damaged drug status image to obtain a set of semantic coding matrices of damage information-damage status cross-modal prompt information; The cross-modal interaction guidance optimization subunit is used to guide the cross-modal interaction optimization of the semantic coding vector of the cause of damage and the semantic coding feature map of the damaged drug status image based on the set of the semantic coding matrix of the cross-modal prompt information of the damage information-damage status to obtain the semantic alignment coding feature map of the damage information-damage status.

6. The intelligent management system for special medicine cabinets according to claim 5 is characterized in that: The local fine-grained semantic interaction sub-unit includes: An autocorrelation coding secondary subunit is used to perform autocorrelation coding on the damage cause semantic coding vector to obtain a damage cause autocorrelation semantic coding matrix; A feature decoupling secondary subunit is used to perform feature decoupling along the channel dimension on the semantic encoding feature map of the damaged drug status image to obtain a set of local semantic feature matrices of the damaged drug status image; The semantic interaction secondary sub-unit is used to perform cross-modal semantic interaction on the damage cause autocorrelation semantic coding matrix and each local semantic feature matrix of the damaged drug status image in the set of local semantic feature matrices of the damaged drug status image to obtain a set of damage information-damage status cross-modal prompt information semantic coding matrices.

7. The intelligent management system for special medicine cabinets according to claim 6 is characterized in that: The semantic interaction secondary sub-unit is used to: Performing a linear transformation on the damage cause autocorrelation semantic coding matrix to obtain a damage cause query coding matrix and a damage cause value coding matrix; Each of the local semantic feature matrices of the damaged drug status image in the set of the damage cause query coding matrix, the damage cause value coding matrix and the local semantic feature matrix of the damaged drug status image is respectively input into the cross-modal prompt information encoder based on the converter structure to obtain the set of the damage information-damage status cross-modal prompt information semantic coding matrices.

8. The intelligent management system for special medicine cabinets according to claim 7 is characterized in that: The cross-modal interaction guidance optimization subunit is used to: Inputting the set of the damage information-damage status cross-modal prompt information semantic encoding matrices into the decoder-based information gating unit to obtain a set of damage information-damage status cross-modal semantic interaction attention weights; The set of cross-modal semantic interaction attention weights of the damage information-damage status, the autocorrelation semantic coding matrix of the damage cause, and the set of local semantic feature matrix of the damaged drug status image are input into the cross-modal interaction optimization unit to obtain the damage information-damage status semantic alignment coding feature map.

9. The intelligent management system for special medicine cabinets according to claim 8 is characterized in that: The damage reporting cause confirmation unit is used to: The damage information-damage status semantic alignment encoding feature map is input into the classifier-based damage cause auxiliary confirmation module to obtain a confirmation result, and the confirmation result is used to indicate whether the damage cause is input incorrectly.

Images