Medical intelligent warehousing system for code scanning matrix recognition based on stereoscopic scanning

Abstract

The invention relates to a medical intelligent warehousing system for code scanning matrix identification based on stereo scanning, and based on the implementation of the invention, a stereo scanning system is utilized to carry out stereo scanning on warehouse-in and warehouse-out medical instruments or medicine packages, and package pictures at multiple angles can be obtained at the same time and uploaded to a background server. The background traverses each package image, identifies the bar code information of the current medical instrument or medicine package from a certain package image by using an image identification technology, and carries out warehouse-in and warehouse-out recording; and meanwhile, a damage identification detection model pre-deployed on the background can be utilized to traverse and identify whether appearance damage characteristics exist on each package image, if so, the background generates corresponding records and sends the records to the storage terminal, and inventory management is carried out on the storage terminal. Barcode information identification and extraction and appearance damage state detection can be carried out only through one-time picture collection, the warehouse-in and warehouse-out detection efficiency is improved, the manual inspection workload is reduced, and the operation difficulty is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent warehousing technology, and in particular to a medical intelligent warehousing system for performing code matrix recognition based on stereoscopic scanning, a medical intelligent warehousing method for performing code matrix recognition based on stereoscopic scanning, and electronic equipment. Background Art

[0002] The storage operation steps for medical devices in and out of the warehouse mainly include the following key links:

[0003] First, the warehousing operation:

[0004] Acceptance preparation: Before the consumables arrive, the purchasing department notifies the warehouse manager, who prepares the acceptance tools and equipment.

[0005] Acceptance process: After the consumables arrive, relevant personnel are organized to inspect them according to the process, including quantity, quality, etc.

[0006] Warehouse management: Determine the placement according to the nature of the goods and inventory status, fill out the material transfer form, enter the material management system, and make quality inspection records.

[0007] Secondly, the outbound operation

[0008] Shipping preparation: Receive and confirm customer shipping orders, check shipping list information to ensure accuracy.

[0009] Pick up and ship out: Pick up goods according to the outbound delivery note and outbound delivery note, record the outbound goods, and issue outbound behavior warnings.

[0010] However, when medical devices are in and out of the warehouse, the warehouse manager needs to scan the barcodes on the packaging of the medical devices in and out of the warehouse and collect information to generate the corresponding medical device in and out records for the warehouse inventory; in addition, it is necessary to check whether the medical device packaging is damaged, etc., so as to facilitate timely inventory. However, the scanning and damage detection of medical devices in and out of the warehouse are currently mainly manual operations. This method not only requires continuous on-site inspections by humans, which is time-consuming, but also hospitals usually operate 24 hours a day, which makes people very tired and inefficient. In addition, in the damage inspection of medical device packaging, it is also necessary to manually inspect multiple sides of the packaging, which may require shutdown or use of an assembly line for inspection. Therefore, the efficiency is low and multiple people are required to conduct external damage inspections, which will reduce the speed of medical device in and out of the warehouse. Summary of the invention

[0012] In order to solve the above problems, the present application proposes a medical intelligent warehousing system for code matrix recognition based on stereoscopic scanning, a medical intelligent warehousing method and electronic equipment for code matrix recognition based on stereoscopic scanning.

[0013] On the one hand, the present application proposes a medical intelligent warehousing system for code matrix recognition based on stereoscopic scanning, comprising:

[0014] The three-dimensional scanning system is used to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway;

[0015] An industrial gateway, used for forwarding the packaging image to a backend server;

[0016] The backend server is used to receive and input the packaging image into a pre-deployed damage recognition and detection model, recognize the packaging image through the damage recognition and detection model and output a corresponding recognition result, and forward the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level;

[0017] A storage terminal, used to display the recognition result;

[0018] The three-dimensional scanning system is connected to the backend server through the industrial gateway;

[0019] The storage terminal is communicatively connected with the backend server.

[0020] Preferably, the stereoscopic scanning system includes a plurality of stereoscopically arranged cameras, which communicate data with the industrial gateway via an Internet of Things network.

[0021] Preferably, the backend server is also used for:

[0022] Storing a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and storing them separately in a preset storage space in a background database;

[0023] Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images;

[0024] The barcode information is parsed to obtain basic information of the medical device or drug packaging and save it in the background database.

[0025] Preferably, the backend server is also used for:

[0026] Read the barcode number of the medical device or drug in the basic information;

[0027] Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system:

[0028] If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database;

[0029] Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

[0030] Preferably, the method for generating the damage identification detection model comprises:

[0031] Collect pictures of damaged packaging of several medical devices or medicines;

[0032] Preprocessing the damaged packaging picture;

[0033] Using a convolutional neural network (CNN) to extract features of the damaged packaging image, extracting appearance damage features in the damaged packaging image, including: damage morphology;

[0034] Performing feature annotation on the appearance damage feature, wherein the annotation information includes: a damage level corresponding to the appearance damage feature;

[0035] Collecting a feature set consisting of a number of annotated appearance damage features;

[0036] Dividing the feature set into a training set and a validation set according to a preset ratio;

[0037] Inputting the training set into a preset CNN model to perform feature training and learning to generate the damage recognition and detection model;

[0038] The prediction performance of the damage identification and detection model is verified using the verification set:

[0039] If the prediction is successful, the damage identification and detection model is deployed on the background server and put into use;

[0040] Otherwise, repeat the above steps and retrain the model.

[0041] Preferably, the backend server is also used for:

[0042] Determine whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions:

[0043] If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal;

[0044] Otherwise, give up.

[0045] On the other hand, the present application proposes a medical intelligent warehousing method based on stereoscopic scanning for code matrix recognition, which is implemented based on the above-mentioned medical intelligent warehousing system based on stereoscopic scanning for code matrix recognition, including the following steps:

[0046] Use the 3D scanning system to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway;

[0047] The industrial gateway forwards the packaging image to the backend server;

[0048] The backend server receives and inputs the packaging image into a pre-deployed damage recognition and detection model, recognizes the packaging image through the damage recognition and detection model, outputs a corresponding recognition result, and forwards the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level;

[0049] The warehouse terminal receives and displays the recognition result.

[0050] Preferably, the method further comprises the following steps:

[0051] The backend server stores a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and stores them separately in a preset storage space in the backend database;

[0052] Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images;

[0053] Parse the barcode information, obtain basic information of the medical device or drug packaging and save it in the background database;

[0054] Read the barcode number of the medical device or drug in the basic information;

[0055] Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system:

[0056] If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database;

[0057] Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

[0058] Preferably, it also includes:

[0059] The backend server determines whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions:

[0060] If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal;

[0061] Otherwise, give up.

[0062] In another aspect, the present application further provides an electronic device, comprising:

[0063] processor;

[0064] a memory for storing processor-executable instructions;

[0065] Among them, the processor is configured to implement the medical intelligent warehousing method based on stereoscopic scanning for code matrix recognition when executing the executable instructions.

[0066] Technical effects of the present invention:

[0067] Based on the implementation of the present invention, the present invention uses a stereoscopic scanning system to perform stereoscopic scanning on the medical devices or drug packaging entering and leaving the warehouse, and can simultaneously obtain packaging pictures of multiple angles and upload them to the background server. On the background server, each packaging image can be traversed, and the barcode information of the current medical device or drug packaging can be identified from a certain packaging image using image recognition technology to record the entry and exit of the warehouse; at the same time, the damage recognition detection model pre-deployed on the background can be used to traverse and identify whether there are appearance damage features on each packaging image. If so, the background generates corresponding records and sends them to the storage terminal, and inventory management can be performed on the storage terminal. At the same time, the use of this model can intelligently identify the level of damage to the packaging, and the background can perform replenishment notification based on the damage level of the current packaging. Therefore, this solution does not require multiple people to perform on-site operations, and only requires the terminal to track the identification results of the medical devices or drugs entering and leaving the warehouse on site. Only one picture acquisition is required to perform barcode information recognition and extraction and appearance damage state detection, thereby improving the efficiency of entry and exit detection. The present invention can reduce the workload of manual inspection, reduce the difficulty of operation, and greatly improve the speed of entry and exit of medical devices and improve the efficiency of warehouse operation.

[0068] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0070] Figure 1 Shown is a schematic diagram of the system composition structure of the present invention;

[0071] Figure 2 It is shown as a schematic diagram of the model training process of the present invention;

[0072] Figure 3 It is a schematic diagram showing the application of the electronic device of the present invention. DETAILED DESCRIPTION

[0073] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0074] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0075] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0076] Example 1

[0077] like Figure 1 As shown, on the one hand, the present application proposes a medical intelligent warehousing system for scanning code matrix recognition based on stereoscopic scanning, including:

[0078] The three-dimensional scanning system is used to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway;

[0079] An industrial gateway, used for forwarding the packaging image to a backend server;

[0080] The backend server is used to receive and input the packaging image into a pre-deployed damage recognition and detection model, recognize the packaging image through the damage recognition and detection model and output a corresponding recognition result, and forward the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level;

[0081] A storage terminal, used to display the recognition result;

[0082] The three-dimensional scanning system is connected to the backend server through the industrial gateway;

[0083] The storage terminal is communicatively connected with the backend server.

[0084] The present invention uses a stereoscopic scanning system to perform stereoscopic scanning on the medical devices or drug packaging entering and leaving the warehouse. It can simultaneously obtain packaging images from multiple angles and upload them to the background server. On the background server, each packaging image can be traversed, and the barcode information of the current medical device or drug packaging can be identified from a certain packaging image using image recognition technology to record entry and exit. At the same time, the damage recognition detection model pre-deployed on the background can be used to traverse and identify whether there are appearance damage features on each packaging image. If so, the background generates corresponding records and sends them to the warehouse terminal, and inventory management can be performed on the warehouse terminal. At the same time, the use of this model can intelligently identify the damage level of the package, and the background can issue a replenishment notification based on the damage level of the current package.

[0085] Capturing an image once can achieve multiple functions:

[0086] The barcode information in the image is extracted and recognized, and the product information in the system database and the barcode of the extracted information are analyzed to obtain the production date, expiration date, batch number, serial number, product name, specification model, packaging specifications, quantity of a single package and other information of the incoming products. This can be used to determine whether the goods are ordered and store them in the database.

[0087] Scan the outer packaging image to identify whether the outer packaging of medical devices or drugs is damaged. The analysis results are bound to the identified product information and stored in the system. The system will promptly issue an unqualified sound and light warning if the packaging is damaged or does not belong to the ordered product.

[0088] The system can deploy in advance a damage recognition detection model trained and constructed based on big data of images of medical devices or medicines, so as to intelligently detect whether there are damage features in the packaging images of medical devices or medicines.

[0089] Specific:

[0090] The stereo scanning system includes several stereo cameras, which communicate with the industrial gateway through the Internet of Things network. Each camera can be an industrial IoT camera, which can collect image data and upload the collected image data to the industrial gateway based on its own Internet of Things, and then the industrial gateway reports it to the backend server.

[0091] For the industrial gateway, the IoT communication links with the backend server and each IoT camera can be configured according to the hardware facilities purchased by the user. For example, the existing cameras on the market can be activated to establish data access communication of the industrial gateway by addressing. The communication between the industrial gateway and the backend server can be carried out by wired or wireless data communication link. The configuration of the corresponding data communication link and the construction of the communication link (such as data communication based on the TCP protocol) are not described in detail in this embodiment.

[0092] The cameras in the stereo scanning system are set at different positions and angles in space, and can collect images of medical devices or drug packages entering and leaving the warehouse from multiple angles. In this aspect, the package is set as a hexahedron, and corresponding 6 cameras can be deployed at 6 spatial angles of the detection point.

[0093] For medical devices or drug packaging entering and leaving the warehouse, an assembly line can be used for in-and-out transportation. The bottom of the assembly line needs to be hollowed out to facilitate scanning the bottom of the package to collect the bottom camera, and the image of the bottom of the package can be collected through the hollow space.

[0094] The storage terminal can be a PDA terminal or an EDA terminal, etc., which can communicate data with the background server, log in to the background and view the recognition results of the corresponding packaging.

[0095] An AI model is deployed on the background server. The AI ​​model is generated through several packaging images with damage features and based on CNN model training. The model can traverse and identify several packaging images collected by three-dimensional scanning of each package, so as to identify whether there are external damage features on each packaging image of the package and the damage level marked on the corresponding features. The background can identify and detect damage to the medical device or drug packaging currently in and out of the warehouse based on the recognition results output by the model, and can also count its damage level.

[0096] When calculating the damage level (see the subsequent description), it can be determined that as long as one surface has external damage features, its damage level does not need to be considered, and the current packaging is directly determined to be unqualified and needs to be replenished. The damage level of the packaging image of each surface can also be determined. If the damage level of at least one surface among the damaged images of several surfaces is greater than the damage level threshold set by the administrator, it is determined that the packaging needs to be returned to the factory for replenishment. The damage level of the packaging image of each surface can also be weighted. If the damage level obtained after the weighted calculation does not exceed the preset value, the product does not need to be returned to the factory for replenishment, and the weight of each surface can be set by the administrator.

[0097] There is barcode information on the packaging image of a certain surface, so the background can perform image recognition on each packaging image of the package by traversing and extract the barcode information in the corresponding image. It is only necessary to scan and recognize the image, without determining which surface has the barcode information. By traversing and recognizing, the packaging images of each surface of the package can be recognized to obtain its barcode information (including the production date of the product entering / exiting the warehouse, expiration date, batch number, serial number, product name, specification model, packaging specifications, quantity of a single package, etc.).

[0098] Preferably, the backend server is also used for:

[0099] Storing a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and storing them separately in a preset storage space in a background database;

[0100] Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images;

[0101] The barcode information is parsed to obtain basic information of the medical device or drug packaging and save it in the background database.

[0102] For the scanned image of the current medical device or drug package, the background stores several package images of the package. After storage, the traversal algorithm is used to traverse and identify each package image of the stored package, so as to identify and extract the corresponding barcode information. After the background analyzes the barcode information, the basic information of the package can be obtained. At the same time, the in-and-out warehouse file of the package can be established in the background and saved in the background database, which is convenient for the background to carry out inventory management.

[0103] The steps of traversing each package image of the recognized package to obtain the barcode information therein can refer to the following steps:

[0104] 1. Install and import the library

[0105] Install OpenCV and necessary extension libraries.

[0106] 2.Import the cv2 library.

[0107] Reading image list

[0108] Get the file paths of all images to be processed.

[0109] 3. Image Preprocessing

[0110] Loop processing: Perform the following operations on each image.

[0111] 4. Read the image.

[0112] Convert to grayscale image to improve barcode recognition efficiency.

[0113] Optional: Apply Gaussian Blur to reduce noise.

[0114] 5. Barcode detection and decoding

[0115] Use OpenCV's BarcodeDetector or QRCodeDetector class.

[0116] Detection is performed on the preprocessed image.

[0117] Get and decode barcode information.

[0118] 6. Storage and Output

[0119] Store or output the decoded barcode information to a specified location.

[0120] Through the above steps, each image can be traversed and identified, and the barcode information on a certain image can be effectively obtained.

[0121] Preferably, the backend server is also used for:

[0122] Read the barcode number of the medical device or drug in the basic information;

[0123] Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system:

[0124] If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database;

[0125] Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

[0126] The ordering system on the backend can be used to manage ordering information. The barcode numbers of medical devices or drugs entering / exiting the warehouse will be saved in the ordering system. Therefore, after parsing the basic information of the medical device or drug packaging, the barcode number of the medical device or drug in the basic information can be read, and further search in the ordering system to see if there is the barcode number of the medical device or drug currently entering / exiting the warehouse.

[0127] The background records the search results in time. If the ordering system has the barcode number of the medical device or drug currently in / out of the warehouse, a medical device or drug in / out record file containing the basic information is generated and saved in the background database. The subsequent storage terminal can log in to the background to view the in / out record file of the medical device or drug in / out of the current batch.

[0128] If the ordering system does not have the barcode number of the medical device or drug currently entering / exiting the warehouse, the background generates an early warning message containing the barcode number of the medical device or drug and sends it to the warehouse terminal, so that the administrator can timely check and manage the inventory of the medical devices or drugs currently entering / exiting the warehouse with early warning information, and promptly remove unqualified medical devices or drugs (which can be removed by a manipulator. After the background finds that the medical devices or drugs currently entering / exiting the warehouse trigger an early warning, the background (on which the robot control software is deployed) can control the robot's movements to grab the medical devices or drugs currently entering / exiting the warehouse and move them aside. The robot's grabbing movements and motion trajectories can be constructed in the background software in advance to achieve intelligent in and out of warehouse management and fully automatic in and out of warehouse operations. There is no need for manual removal management of unqualified packaging products that trigger early warnings, saving labor), and the system will eventually generate an unqualified list, and automatically feedback to the manufacturer's background according to the manufacturer's address, requesting replenishment).

[0129] For the communication between the backend and the manufacturer's server, the backend can generate a replenishment report for the medical devices or drugs with warning information in this batch, and send it to the third party after encryption and decryption. The third party decrypts the data based on the encryption and decryption consistency to obtain the data and ensure the security of data transmission.

[0130] like Figure 2 As shown, preferably, the method for generating the damage identification detection model includes:

[0131] Collect pictures of damaged packaging of several medical devices or medicines;

[0132] Preprocessing the damaged packaging picture;

[0133] Using a convolutional neural network (CNN) to extract features of the damaged packaging image, extracting appearance damage features in the damaged packaging image, including: damage morphology;

[0134] Performing feature annotation on the appearance damage feature, wherein the annotation information includes: a damage level corresponding to the appearance damage feature;

[0135] Collecting a feature set consisting of a number of annotated appearance damage features;

[0136] Dividing the feature set into a training set and a validation set according to a preset ratio;

[0137] Inputting the training set into a preset CNN model to perform feature training and learning to generate the damage recognition and detection model;

[0138] The prediction performance of the damage identification and detection model is verified using the verification set:

[0139] If the prediction is successful, the damage identification and detection model is deployed on the background server and put into use;

[0140] Otherwise, repeat the above steps and retrain the model.

[0141] Pictures of damaged packaging of several medical devices or drugs can be obtained from the packaging inspection pictures stored historically in the background database. They can be collected by the administrator.

[0142] For the images of damaged packaging of several medical devices or drugs, feature extraction is performed to extract the appearance damage features, mainly the damaged morphological feature images. A convolutional neural network can be used to extract the damaged morphological feature images and construct a feature set, and divide the training set and verification set, as well as Xining CNN model training and verification, so as to build a damage recognition and detection model and deploy it in the background for application.

[0143] The structure and principle of CNN are not described in detail in this embodiment.

[0144] The convolutional neural network (CNN) is used to learn the appearance damage features and generate a damage recognition detection model. The following steps are usually followed:

[0145] 1. Data preparation

[0146] Data collection: Collect image data of damaged samples of various medical devices or pharmaceutical packages. Make sure the data is representative and covers different types of damage and different objects.

[0147] Data annotation: Annotate the image, mark the damaged area or provide a label whether it is damaged or not. This can be done manually or using an existing annotation dataset. The damage level corresponding to the appearance damage characteristics can also be annotated here. The administrator can annotate the damage level of the current package based on the damage shape and size.

[0148] Data preprocessing: Preprocess images, such as scaling, cropping, and normalization, to unify image size and format and improve model training efficiency.

[0149] 2. Build a CNN model

[0150] Select architecture: Choose a suitable CNN architecture based on task complexity and data volume, such as LeNet, AlexNet, VGG, ResNet, etc.

[0151] Define the model: Use a deep learning framework (such as TensorFlow, PyTorch, etc.) to define the CNN model, including convolutional layers, pooling layers, fully connected layers, etc.

[0152] Set hyperparameters: Determine hyperparameters such as learning rate, batch size, and number of iterations.

[0153] 3. Model training

[0154] Data segmentation: The dataset is divided into a training set and a validation set (8:2). The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the model performance.

[0155] Loss function: Choose an appropriate loss function, such as the cross-entropy loss function, to measure the difference between the model prediction and the true label.

[0156] Optimizer: Select an optimization algorithm, such as SGD, Adam, etc., to update model parameters.

[0157] Training process: input training data into the CNN model, calculate the loss through forward propagation, and update the model parameters through back propagation. Repeat this process until the predetermined number of iterations is reached or the loss converges.

[0158] 4. Model evaluation and tuning

[0159] Performance evaluation: Use the validation set to evaluate the performance of the model, such as accuracy, recall, F1 score, etc.

[0160] Model tuning: Adjust the model architecture, hyperparameters, or data preprocessing methods based on the evaluation results to improve model performance.

[0161] 5. Model deployment and application

[0162] Model export: Export the trained model to a deployable format, such as TensorFlow's SavedModel, PyTorch's model file, etc.

[0163] Integration into applications: Integrate the model into damage identification and detection applications, such as mobile applications, web applications, or industrial inspection systems.

[0164] Real-time detection: Perform real-time damage detection on new image data and output information such as the location of the damaged area and the type of damage.

[0165] 6. Continuous maintenance and updates

[0166] Data collection and annotation: Continuously collect new damaged sample data, annotate and preprocess them to update and expand the dataset.

[0167] Model update: Regularly retrain or fine-tune the model with new datasets to maintain the accuracy and adaptability of the model.

[0168] Performance monitoring: Monitor the performance of the model in actual applications to identify and resolve problems in a timely manner.

[0169] Through the above steps, CNN can be used to learn the appearance damage features and generate an effective damage recognition and detection model.

[0170] Please understand and implement the above model training in conjunction with the CNN structure.

[0171] Preferably, the backend server is also used for:

[0172] Determine whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions:

[0173] If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal;

[0174] Otherwise, give up.

[0175] The conditions for returning to the factory for replenishment can be found in the above-mentioned methods for judging the damage level, which will not be described here. The backend can record the packaging whose damage level meets the conditions for returning to the factory for replenishment, generate corresponding replenishment notifications, and send them to the warehouse terminal for the administrator to review and determine the packaging for returning to the factory for replenishment. The administrator can then log in to the backend through the terminal, send the replenishment list to the third-party server, and request replenishment.

[0176] Therefore, this solution does not require multiple people to perform on-site operations, and only requires the terminal to track the identification results of medical devices or drugs entering and leaving the warehouse on site. Only one picture acquisition is required to perform barcode information identification and extraction and appearance damage status detection, thereby improving the efficiency of warehouse entry and exit detection. The present invention can reduce the workload of manual inspection, reduce the difficulty of operation, and greatly improve the speed of medical device entry and exit and improve the efficiency of warehouse operation.

[0177] Obviously, those skilled in the art should understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned control embodiments. Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned control embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (Solid-State Drive, SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0178] Example 2

[0179] Based on the implementation principle of Example 1, on the other hand, the present application proposes a medical intelligent warehousing method based on three-dimensional scanning for code matrix recognition, which is implemented based on the above-mentioned medical intelligent warehousing system based on three-dimensional scanning for code matrix recognition, including the following steps:

[0180] Use the 3D scanning system to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway;

[0181] The industrial gateway forwards the packaging image to the backend server;

[0182] The backend server receives and inputs the packaging image into a pre-deployed damage recognition and detection model, recognizes the packaging image through the damage recognition and detection model, outputs a corresponding recognition result, and forwards the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level;

[0183] The warehouse terminal receives and displays the recognition result.

[0184] Preferably, the method further comprises the following steps:

[0185] The backend server stores a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and stores them separately in a preset storage space in the backend database;

[0186] Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images;

[0187] Parse the barcode information, obtain basic information of the medical device or drug packaging and save it in the background database;

[0188] Read the barcode number of the medical device or drug in the basic information;

[0189] Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system:

[0190] If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database;

[0191] Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

[0192] Preferably, it also includes:

[0193] The backend server determines whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions:

[0194] If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal;

[0195] Otherwise, give up.

[0196] Please understand the above steps in conjunction with the method description principles of Example 1, and will not be repeated in this example.

[0197] The modules or steps of the present invention described above can be implemented by a general-purpose computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0198] Example 3

[0199] like Figure 3 As shown, further, in another aspect, the present application also proposes an electronic device, including:

[0200] processor;

[0201] a memory for storing processor-executable instructions;

[0202] Among them, the processor is configured to implement a medical intelligent warehousing method based on stereoscopic scanning for code matrix recognition as described in Example 2 when executing the executable instructions.

[0203] The electronic device of the embodiment of the present disclosure includes a processor and a memory for storing processor executable instructions. The processor is configured to implement the medical intelligent warehousing method for scanning code matrix recognition based on stereoscopic scanning as described in the previous embodiment 2 when executing the executable instructions.

[0204] Here, it should be noted that the number of processors can be one or more. At the same time, the electronic device of the embodiment of the present disclosure may also include an input system and an output system. Among them, the processor, memory, input system and output system may be connected through a bus or in other ways, which are not specifically limited here.

[0205] As a computer-readable storage medium, the memory can be used to store software programs, computer executable programs and various modules, such as the program or module corresponding to the medical intelligent warehousing method for scanning code matrix recognition based on stereoscopic scanning in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0206] The input system can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output system can include display devices such as display screens.

[0207] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A medical intelligent storage system based on three-dimensional scanning for code matrix recognition, characterized in that: include: The three-dimensional scanning system is used to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway; An industrial gateway, used for forwarding the packaging image to a backend server; The backend server is used to receive and input the packaging image into a pre-deployed damage recognition and detection model, recognize the packaging image through the damage recognition and detection model and output a corresponding recognition result, and forward the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level; A storage terminal, used to display the recognition result; The three-dimensional scanning system is connected to the backend server through the industrial gateway; The storage terminal is communicatively connected with the backend server.

2. According to claim 1, the medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning is characterized in that: The stereoscopic scanning system includes a plurality of stereoscopically arranged cameras, which communicate data with the industrial gateway via the Internet of Things network.

3. According to claim 2, the medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning is characterized in that: The backend server is also used for: Storing a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and storing them separately in a preset storage space in a background database; Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images; The barcode information is parsed to obtain basic information of the medical device or drug packaging and save it in the background database.

4. According to claim 3, the medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning is characterized in that: The backend server is also used for: Read the barcode number of the medical device or drug in the basic information; Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system: If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database; Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

5. According to claim 1, the medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning is characterized in that: The method for generating the damage identification and detection model comprises: Collect pictures of damaged packaging of several medical devices or medicines; Preprocessing the damaged packaging picture; Using a convolutional neural network (CNN) to extract features of the damaged packaging image, extracting appearance damage features in the damaged packaging image, including: damage morphology; Performing feature annotation on the appearance damage feature, wherein the annotation information includes: a damage level corresponding to the appearance damage feature; Collecting a feature set consisting of a number of annotated appearance damage features; Dividing the feature set into a training set and a validation set according to a preset ratio; Inputting the training set into a preset CNN model to perform feature training and learning to generate the damage recognition and detection model; The prediction performance of the damage identification and detection model is verified using the verification set: If the prediction is successful, the damage identification and detection model is deployed on the background server and put into use; Otherwise, repeat the above steps and retrain the model.

6. The medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning according to claim 5 is characterized in that: The backend server is also used for: Determine whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions: If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal; Otherwise, give up.

7. A medical intelligent warehousing method based on stereoscopic scanning for code matrix recognition, implemented based on the medical intelligent warehousing system based on stereoscopic scanning for code matrix recognition according to any one of claims 1 to 6, characterized in that: The steps include: Use the 3D scanning system to scan the medical device or drug packaging entering and leaving the warehouse, obtain the corresponding packaging image and upload it to the industrial gateway; The industrial gateway forwards the packaging image to the backend server; The backend server receives and inputs the packaging image into a pre-deployed damage recognition and detection model, recognizes the packaging image through the damage recognition and detection model, outputs a corresponding recognition result, and forwards the recognition result to the warehouse terminal; wherein the recognition result includes the following information: whether there are appearance damage features and damage level; The warehouse terminal receives and displays the recognition result.

8. The medical intelligent warehousing method based on three-dimensional scanning for code matrix recognition according to claim 7 is characterized in that: The following steps are also included: The backend server stores a plurality of packaging images of medical devices or drug packaging captured by various cameras from different angles, and stores them separately in a preset storage space in the backend database; Traversing and identifying each of the packaging images, and extracting barcode information of the medical device or drug packaging from the packaging images; Parse the barcode information, obtain basic information of the medical device or drug packaging and save it in the background database; Read the barcode number of the medical device or drug in the basic information; Retrieve and determine whether the medical device or drug barcode number belongs to the number pre-determined in the ordering system: If yes, a medical device or drug in and out storage record file containing the basic information is generated and saved in the backend database; Otherwise, an early warning message including the barcode number of the medical device or drug is generated and sent to the storage terminal.

9. The medical intelligent storage system for scanning code matrix recognition based on stereoscopic scanning according to claim 8 is characterized in that: Also includes: The backend server determines whether the damage level in the current identification result of the medical device or drug exceeds the preset return and replenishment conditions: If yes, a replenishment notice for the current medical device or drug is generated and sent to the storage terminal; Otherwise, give up.

10. An electronic device, characterized in that include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the medical intelligent warehousing method for code matrix recognition based on stereoscopic scanning as described in any one of claims 7-8 when executing the executable instructions.

Images