Medical instrument AI screening management system and method based on image recognition

Abstract

According to the medical instrument AI screening management system and method based on image recognition, image collection of medical instruments in and out of a warehouse is carried out through an independently-deployed image collection unit, the integration difficulty of a delivery robot can be reduced, and the purchase cost of the delivery robot can be reduced. According to the method, manual addressing is not needed, the medical instrument AI recognition model is adopted to recognize the collected image, so that the type of the medical instrument which is currently put in and out of a warehouse and the storage address marked on the image feature of the medical instrument are recognized, and the system can quickly find the storage address of the medical instrument according to the image recognition result; the robot carries out corresponding warehouse-in and warehouse-out distribution according to the storage address, manpower can be saved, the storage address to which the medical instrument belongs can be quickly found, the robot is notified to go to execute warehouse-in and warehouse-out operation, the application efficiency of the medical instrument can be improved, the distribution requirement sent by a front-end nurse through a PDA can be responded, the corresponding medical instrument can be quickly prepared, and the medical instrument delivery efficiency is improved. The operation progress is prevented from being delayed.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent medical technologies, and particularly to a medical device AI screening management system based on image recognition, a medical device AI screening management method based on image recognition, and an electronic device. Background Art

[0002] With the development and popularization of artificial intelligence, the medical industry, especially hospitals, has gradually utilized artificial intelligence to assist medical operations.

[0003] In the medical device warehousing system, the inbound and outbound management of medical devices is related to the subsequent supply and application of medical devices in hospital surgeries or other departments. If the inbound and outbound management of medical devices in the early stage is not properly carried out, serious medical device accidents may occur.

[0004] Traditional inbound and outbound operations of medical devices are mainly completed manually. However, in the face of a large number of inbound and outbound operations of medical devices, it brings relatively complex workloads to warehouse management personnel, with low efficiency and easy errors.

[0005] Although robots have been widely used, the existing robots for the inbound and outbound operations of medical devices generally have the following steps:

[0006] Preparation stage: Ensure that the robot system is installed and debugged, and connected to the medical device warehouse management system;

[0007] System login: The operator logs in to the robot control system and enters the username and password;

[0008] Inbound / outbound instruction: Enter the inbound / outbound instruction in the system, including information such as the name, model, and quantity of the medical device;

[0009] Robot navigation: The robot automatically plans a path according to the system instruction and goes to the specified shelf or storage area;

[0010] Recognition and confirmation: After the robot reaches the destination, it recognizes the medical device through the vision system and proceeds to the next step after confirmation;

[0011] Picking / placement: The robot accurately grabs the medical device using the robotic arm and then places it at the specified position or takes out the required item;

[0012] Data synchronization: After the robot completes the operation, the system automatically updates the inventory information to ensure real-time data synchronization;

[0013] Completion confirmation: The operator checks the operation result of the robot and ends the current task after confirmation;

[0014] Exception handling: In case of a failure or an abnormal situation, the robot promptly reports to the system, and the operator conducts troubleshooting and handling;

[0015] Operation completed: The robot returns to the charging area and automatically charges for the next use.

[0016] However, the above-mentioned operations of the robot for the warehousing and outbound of medical devices still face the following technical problems in practical applications:

[0017] When the existing robot executes the navigation path sent by the background, its system needs to integrate a vision system, which increases the hardware and software costs of the robot and the procurement cost of the hospital;

[0018] For special medical devices, in order to facilitate classification management and use, they need to be stored in the warehouse by category; while the existing storage of medical devices is only placed in the storage cabinets with corresponding preservation labels, and manual search for the required medical devices is needed, so it takes more time and delays the application efficiency of medical devices;

[0019] The existing robot lacks pre-management of medical devices in the warehousing and distribution of medical devices. Usually, nurses go to the warehouse to request the corresponding medical devices and then out of the warehouse, and it is impossible to prepare the medical devices to be used in advance, which will delay the application efficiency of medical devices;

[0020] In addition, medical devices should have a dedicated storage location and cannot be placed randomly or casually in the cabinet, which is likely to cause management chaos or cross-influence. Therefore, a dedicated adapted storage location needs to be provided. The traditional robot distribution scheme does not take this into account and cannot perform intelligent addressing and distribution according to the adapted addresses of medical devices. Summary of the Invention

[0021] To solve the above problems, the present application proposes a medical device AI screening and management system based on image recognition, a medical device AI screening and management method based on image recognition, and an electronic device.

[0022] On the one hand, the present application proposes a medical device AI screening and management system based on image recognition, including:

[0023] An Internet of Things intelligent warehouse for storing medical devices;

[0024] An image acquisition unit for acquiring images of outbound / inbound medical devices and uploading them to the medical device management background;

[0025] An IOT gateway for the IoT communication between the image acquisition unit and the medical device management background;

[0026] A PDA nursing terminal for logging in to the medical device management background to implement corresponding outbound / inbound operations;

[0027] Medical device management backend, used for the outbound / inbound management of medical devices, including: responding to outbound / inbound operations, identifying the current outbound / inbound medical device pictures through a preset medical device AI recognition model, obtaining the storage address ID of the current outbound / inbound medical device, and controlling the delivery robot to perform outbound / inbound operations on the current medical device according to the storage address ID; and, managing the IoT intelligent warehouse and the delivery robot according to the outbound / inbound operation information and recording their status data; wherein, the method for generating the medical device AI recognition model includes:

[0028] (1) Obtain a number of medical device pictures;

[0029] (2) Perform feature engineering on the medical device pictures to extract the medical device picture features therein;

[0030] (3) Add address annotations to the medical device picture features, label the storage address ID of the storage unit adapted by the medical device in the IoT intelligent warehouse on the medical device picture features, and generate annotation features;

[0031] (4) Based on the annotation features, construct a feature set and divide it into a training set and a validation set according to a preset ratio;

[0032] (5) Input the training set into a preset CNN model for feature training and learning to generate the medical device AI recognition model;

[0033] (6) Use the validation set to verify / optimize the medical device AI recognition model. If the verification is qualified, proceed to the next step; otherwise, re-enter the feature engineering step;

[0034] (7) Deploy and apply the medical device AI recognition model to the warehouse management system;

[0035] Delivery robot, used to respond to the control operations of the medical device management backend;

[0036] The IoT intelligent warehouse and the image acquisition unit are respectively communicatively connected to the medical device management backend through the IOT gateway;

[0037] The PDA nursing terminal and the delivery robot are respectively communicatively connected to the medical device management backend.

[0038] As an optional implementation of the present application, optionally, the medical device management backend includes a warehouse management system and a robot management system, wherein:

[0039] The warehouse management system is used to manage the IoT intelligent warehouse, and the medical device AI recognition model is deployed thereon;

[0040] The robot management system is used to manage the delivery robots and realizes communication control with the delivery robots through a control terminal.

[0041] The warehouse management system collects the medical device pictures uploaded by the image acquisition unit, imports the medical device pictures into the preset medical device AI recognition model. The medical device AI recognition model recognizes the medical device picture features in the medical device pictures, analyzes the medical device picture features, obtains the marked medical device storage address ID therein and sends it to the robot management system.

[0042] The robot management system generates corresponding outbound / inbound instructions according to the medical device storage address ID and issues them to the control terminal. The control terminal issues the outbound / inbound instructions to the idle delivery robots according to the working status of each delivery robot.

[0043] As an optional implementation scheme of the present application, optionally, the Internet of Things intelligent warehouse includes:

[0044] A controller for logical control;

[0045] A memory for storing the medical device storage address ID;

[0046] A communication unit for communicating with the medical device management background through the IOT gateway;

[0047] Several storage units for storing medical devices;

[0048] The memory, the communication unit and the storage unit are respectively communicatively connected to the controller.

[0049] On the other hand, the present application proposes a method for AI screening management of medical devices based on image recognition, which is implemented based on a system for AI screening management of medical devices based on image recognition, and includes the following steps:

[0050] (1) Inbound operation

[0051] The warehouse management system generates a bin retrieval instruction containing the medical device storage address ID and issues it to the Internet of Things intelligent warehouse through the IOT gateway;

[0052] The Internet of Things intelligent warehouse responds to the bin retrieval instruction and checks whether the storage unit adapted to the current medical device is in an idle bin:

[0053] If it is idle, feedback is sent to the warehouse management system through the IOT gateway, and the medical device storage address ID of the current medical device is sent to the robot management system; the robot management system generates a corresponding warehousing instruction based on the medical device storage address ID and sends it to the control terminal, and the control terminal controls the idle delivery robot to deliver the current medical device to the storage unit adapted by the current medical device;

[0054] If the storage unit adapted by the current medical device is not in an idle position, the warehouse management system generates an idle position search instruction and sends it to the IoT smart warehouse through the IoT gateway;

[0055] The IoT smart warehouse responds to the free storage location search instruction and checks whether there is a free storage location in the storage unit:

[0056] If so, the storage address ID of the storage unit with free space is fed back to the warehouse management system through the IOT gateway, and the warehouse management system updates the medical device storage address ID of the current medical device and sends the storage address ID of the storage unit with free space to the robot management system; the robot management system generates a corresponding warehousing instruction based on the medical device storage address ID and sends it to the control terminal, and the control terminal controls the idle delivery robot to deliver the current medical device to the storage unit with free space;

[0057] If not, the warehouse management system generates a warehousing alert containing the medical device image features of the current medical device and notifies the administrator;

[0058] (2) Outbound Operation

[0059] Sending a delivery notice to the warehouse management system in advance through the PDA nursing terminal, wherein the delivery notice includes a list of medical devices to be used and a pickup point address;

[0060] The warehouse management system parses the list of medical devices to be used, obtains medical device images of each of the medical devices to be used in the list, identifies medical device image features in the medical device images using the preset medical device AI recognition model, parses the medical device image features, obtains the medical device storage address ID marked therein, and sends it to the robot management system;

[0061] The robot management system generates corresponding outbound instructions based on the medical device storage address ID and sends them to the control terminal. The control terminal controls the idle delivery robot to go to the storage unit corresponding to the medical device storage address ID to deliver the medical devices to be used to the pickup point.

[0062] On the other hand, this application also proposes an electronic device, including:

[0063] A processor;

[0064] A memory for storing instructions executable by the processor;

[0065] Wherein, when the processor is configured to execute the executable instructions, it implements the above-mentioned method for AI screening management of medical devices based on image recognition.

[0066] Technical effects of the present invention:

[0067] In this application, the image acquisition unit deployed separately is used to acquire images of medical devices for incoming and outgoing storage, which can reduce the integration difficulty of the delivery robot and its procurement cost. And in the present invention, manual addressing is not required. Instead, an AI recognition model for medical devices is used to recognize the acquired images, so as to identify the type of the current incoming and outgoing medical devices and the storage address marked on their image features, which can enable the system to quickly find the storage address of the medical devices according to the image recognition result, and let the robot perform corresponding incoming and outgoing storage distribution according to the storage address.

[0068] By adopting the present invention, using the AI recognition model for medical devices to perform image recognition on the incoming and outgoing medical devices, obtaining the marked storage address, and then letting the robot perform corresponding incoming and outgoing operations according to the storage address, it can save labor, quickly find the storage address where the medical devices belong, and notify the robot to go to perform the incoming and outgoing operations, which can improve the application efficiency of medical devices, and can respond to the distribution requirements sent by the front-end nurse through the PDA, quickly prepare the corresponding medical devices, and avoid delaying the surgical progress.

[0069] According to the following detailed description of the exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings, which are included in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the present disclosure together with the specification, and are used to explain the principles of the present disclosure.

[0071] Figure 1 It is shown as a schematic diagram of the system composition of the present invention;

[0072] Figure 2 It is shown as a schematic diagram of the training process of the AI model of the present invention;

[0073] Figure 3 It is shown as an application schematic diagram of the electronic device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote functionally identical or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0075] As used herein, the term "exemplary" means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0076] In addition, for a better description of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, well-known means, elements, and circuits have not been described in detail so as to highlight the gist of the present disclosure.

[0077] Embodiment 1

[0078] As Figure 1 shown, on the one hand, the present application provides a medical device AI screening management system based on image recognition, including:

[0079] An Internet of Things intelligent warehouse for storing medical devices;

[0080] An image acquisition unit for acquiring images of the medical devices during outbound / inbound and uploading them to the medical device management background;

[0081] An IOT gateway for the Internet of Things communication between the image acquisition unit and the medical device management background;

[0082] A PDA nursing terminal for logging in to the medical device management background to implement corresponding outbound / inbound operations;

[0083] A medical device management background for the outbound / inbound management of medical devices, including: in response to an outbound / inbound operation, identifying the image of the current outbound / inbound medical device through a preset medical device AI recognition model to obtain the storage address ID of the current outbound / inbound medical device, and controlling the delivery robot to implement outbound / inbound of the current medical device according to the storage address ID; and, managing the Internet of Things intelligent warehouse and the delivery robot according to the outbound / inbound operation information and recording their status data;

[0084] A delivery robot for responding to the control operation of the medical device management background;

[0085] The Internet of Things intelligent warehouse and the image acquisition unit are respectively communicatively connected to the medical device management background through the IOT gateway;

[0086] The PDA nursing terminal and the delivery robot are respectively communicatively connected to the medical device management background.

[0087] In this embodiment, the Internet of Things intelligent warehouse can refer to the Internet of Things storage bins or lockers in existing hospitals, such as honeycomb cabinets, etc., and this solution is not limited. For the image acquisition unit, a CCD camera or other industrial cameras, etc. can be used, which can collect the image information of the medical devices during outbound or inbound, and report it to the background through the Internet of Things gateway (IOT gateway). The IOT gateway, that is, the Internet of Things gateway, can be purchased and configured by the hospital itself. The image acquisition unit can use wireless cameras, etc., and can be communicatively connected to the gateway. The delivery robots are all deployed in the robot waiting area, and the outbound and inbound can be controlled through the control terminal in the waiting area, such as a PC terminal, etc. The control terminal communicates with the medical device management background, executes the instructions issued by the background, and reports the working information of each delivery robot.

[0088] In the present invention, the type of the delivery robot can be purchased by the hospital, but it does not include a vision system. In the present invention, for image acquisition, the image acquisition unit is used to uniformly acquire images.

[0089] The Internet of Things intelligent warehouse can place the medical devices to be outbound or inbound on its loading platform or window, facilitating the delivery robot to directly pick up or place the goods (an administrator can also be configured for handover management, and the robot can reach the specific location). The automatic goods placement principle of existing lockers can be referred to.

[0090] When the delivery robot is inbound or outbound, the administrator can place the medical devices to be inbound on its manipulator. When outbound, the medical devices to be outbound can be sent to the pick-up point location required by the nurse (the pick-up point / destination address can be obtained through the information sent by the PDA).

[0091] The control of the robot is to generate the corresponding delivery route and perform delivery management by the robot management system on the background. The storage management of each medical device in the Internet of Things intelligent warehouse is managed by the warehouse management system software on the background. The background management software can be provided by the corresponding equipment party.

[0092] Therefore, by using a separately deployed image acquisition unit to acquire images of medical devices for inbound and outbound operations, the integration difficulty of the delivery robot can be reduced, and its procurement cost can be lowered. In addition, the present invention does not require manual addressing. Instead, a medical device AI recognition model is used to recognize the acquired images, so as to identify the type of the current inbound and outbound medical device and the storage address marked on its image features, enabling the system to quickly find the storage address of the medical device according to the image recognition result and allowing the robot to perform corresponding inbound and outbound deliveries according to the storage address.

[0093] By adopting the present invention, a medical device AI recognition model is used to recognize images of inbound and outbound medical devices, obtain the marked storage address, and then let the robot perform corresponding inbound and outbound operations according to the storage address. This can save labor, quickly find the storage address where the medical device belongs, notify the robot to go and perform inbound and outbound operations, improve the application efficiency of medical devices, and respond to the delivery requirements sent by the front-end nurse through the PDA, quickly prepare the corresponding medical devices, and avoid delaying the surgical progress.

[0094] Next, the training and construction method of the medical device AI recognition model of the present invention will be described.

[0095] As Figure 2 shown, the generation method of the medical device AI recognition model includes:

[0096] (1) Obtain a number of medical device pictures;

[0097] (2) Perform feature engineering on the medical device pictures to extract the medical device picture features therein;

[0098] (3) Add address annotations to the medical device picture features, and mark the storage address ID of the storage unit adapted to the medical device in the Internet of Things intelligent warehouse on the medical device picture features to generate marked features;

[0099] (4) Based on the marked features, construct a feature set and divide it into a training set and a validation set according to a preset ratio;

[0100] (5) Input the training set into a preset CNN model for feature training and learning to generate the medical device AI recognition model;

[0101] (6) Use the validation set to verify / optimize the medical device AI recognition model. If the verification is qualified, proceed to the next step; otherwise, re-enter the feature engineering step;

[0102] (7) Deploy and apply the medical device AI recognition model to the warehouse management system.

[0103] Pictures of different types of medical devices can be prepared by the administrator. For example, scalpels, surgical sponges of different models, etc. Or some consumables.

[0104] The present invention constructs an AI recognition model for medical devices using a CNN model, which can refer to the following steps:

[0105] 1. Data collection and preprocessing:

[0106] Collect a picture dataset of medical devices to ensure data diversity, including pictures under different angles and different lighting conditions.

[0107] Preprocess the pictures, including resizing the pictures, normalizing pixel values, and data augmentation (rotation, scaling, cropping, etc.).

[0108] 2. Feature engineering:

[0109] Convolutional neural networks can be used to extract picture features from medical device pictures, and the picture features are labeled. Each picture is assigned a category label of the medical device and the storage address ID of the storage unit where the medical device should be stored (the storage address ID of the storage unit adapted to the surgical instrument in the Internet of Things intelligent warehouse is assigned according to the storage requirements of the surgical instrument. For example, if scalpel A should be stored in storage unit XXX, then the storage address ID of this storage unit is labeled on the picture features of scalpel A). In this way, a feature set composed of several labeled features is obtained.

[0110] 3. Divide the dataset:

[0111] Divide the dataset into a training set, a validation set, and a test set.

[0112] 4. Design the CNN model architecture:

[0113] Select a suitable convolutional neural network architecture, such as AlexNet, VGGNet, ResNet, etc., or design a custom CNN structure.

[0114] Design the network layers, including convolutional layers, activation layers (such as ReLU), pooling layers, fully connected layers, etc.

[0115] Compile the model:

[0116] Select a suitable loss function, such as the cross-entropy loss function.

[0117] Select an optimizer, such as Adam or SGD.

[0118] Define evaluation metrics, such as accuracy.

[0119] 5. Train the model:

[0120] Train the CNN model using the training set data.

[0121] Monitor the model performance on the validation set and adjust the hyperparameters to prevent overfitting.

[0122] 6. Model evaluation:

[0123] Evaluate the model performance on the test set to ensure that the model has good generalization ability. The test method is to calculate metrics such as accuracy, recall, and F1 score to ensure that the model has good generalization ability. The specific test method can be operated by the administrator.

[0124] 7. Model optimization and adjustment:

[0125] Fine-tune the model according to the evaluation results, which may include adjusting the network structure, optimizer parameters, learning rate, etc.

[0126] 8. Deploy the model:

[0127] Deploy the trained model to practical applications, such as a medical device management system or an auxiliary diagnosis system.

[0128] Model monitoring and maintenance:

[0129] Regularly monitor the model performance, collect new data to retrain and update the model to adapt to new types of medical devices or improve the recognition accuracy.

[0130] The above model training can be operated in combination with the principle of the CNN model, which will not be elaborated in this embodiment.

[0131] After model training, deploy it to the background and use it for applications.

[0132] As an optional implementation of this application, optionally, the medical device management background includes a warehouse management system and a robot management system, where:

[0133] The warehouse management system is used to manage the IoT intelligent warehouse, and the medical device AI recognition model is deployed on it;

[0134] The robot management system is used to manage the delivery robots and achieve communication control with the delivery robots through a control terminal;

[0135] The warehouse management system collects the medical device pictures uploaded by the image acquisition unit, imports the medical device pictures into the preset medical device AI recognition model, and the medical device AI recognition model identifies the medical device picture features in the medical device pictures, analyzes the medical device picture features, obtains the medical device storage address ID marked therein, and sends it to the robot management system;

[0136] The robot management system generates corresponding outbound / inbound instructions according to the medical device storage address ID and sends them to the control terminal. The control terminal issues the outbound / inbound instructions to the idle delivery robots according to the working status of each delivery robot.

[0137] In the present invention, multiple IoT intelligent warehouses can be set up. Each intelligent warehouse can communicate with the background through an IoT gateway based on IoT communication. A warehouse management system is deployed on the background for intelligent management of each intelligent warehouse. For the storage location, storage quantity, storage information, etc. of medical devices in each intelligent warehouse, the intelligent warehouse can report the corresponding storage information to the warehouse management system. Therefore, the warehouse management system can manage the storage situation of medical devices stored in each intelligent warehouse. An AI model constructed through the above training is deployed in the warehouse management system. After the image acquisition unit uploads the pictures of inbound and outbound medical devices, the warehouse management system can call the medical device AI recognition model to intelligently recognize the current pictures of inbound and outbound medical devices, identify the picture features in the pictures, and parse the medical device storage addresses marked on the recognition features. Therefore, the storage address ID of the current inbound and outbound medical devices can be obtained and sent to the robot management system.

[0138] An image acquisition unit can be respectively set at the outbound exit and the inbound entrance.

[0139] The robot management system calculates the corresponding delivery path according to the storage address ID identified and issued by the warehouse management system. The robot management system on the background generates corresponding outbound or inbound instructions and sends them to the control terminal. The control terminal assigns corresponding delivery tasks to the idle delivery robots in the robot waiting area. The control terminal can perform work scheduling and status management on each delivery robot, view the idle delivery robots, and execute the current outbound or inbound instructions. The control terminal communicates with the background for data communication and performs interactive control of the robots.

[0140] The inbound and outbound processes of the present invention will be further described in Embodiment 2 later.

[0141] As an optional implementation scheme of the present application, optionally, the IoT intelligent warehouse includes:

[0142] A controller for logical control;

[0143] A memory for storing the storage address ID of medical devices;

[0144] A communication unit for communicating with the medical device management background through the IOT gateway;

[0145] Several storage units for storing medical devices;

[0146] The memory, communication unit, and storage unit are respectively communicatively connected to the controller.

[0147] The hardware system of the Internet of Things intelligent warehouse can refer to the hardware system and control method of existing intelligent lockers such as Fengchao lockers. Each storage unit has its own control unit and control chip and can be independently controlled and operate.

[0148] Obviously, those skilled in the art should understand that to implement all or part of the processes in the above embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above control embodiments. Those skilled in the art can understand that to implement all or part of the processes in the above embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above control embodiments. Among them, the storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (abbreviation: HDD), or solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0149] Embodiment 2

[0150] Based on the implementation principle of Embodiment 1, on the other hand, this application proposes an AI screening management method for medical devices based on image recognition, which is implemented based on an AI screening management system for medical devices based on image recognition in Embodiment 1, and includes the following steps:

[0151] (1) Warehousing operation

[0152] The warehouse management system generates a bin retrieval instruction containing the storage address ID of the medical device and sends it to the Internet of Things intelligent warehouse through the IOT gateway;

[0153] The Internet of Things intelligent warehouse responds to the bin retrieval instruction and checks whether the storage unit adapted to the current medical device is in an idle bin:

[0154] If it is idle, send feedback to the warehouse management system through the IOT gateway, and send the medical device storage address ID of the current medical device to the robot management system; the robot management system generates a corresponding warehousing instruction according to the medical device storage address ID and sends it to the control terminal, and the control terminal controls the idle delivery robot to deliver the current medical device to the storage unit adapted to the current medical device;

[0155] If the storage unit adapted to the current medical device is not in an idle position, the warehouse management system generates an idle position retrieval instruction and sends it to the Internet of Things intelligent warehouse through the IOT gateway;

[0156] The Internet of Things intelligent warehouse responds to the idle position retrieval instruction and checks whether there is a storage unit with an idle position:

[0157] If there is, the storage address ID of the storage unit with an idle position is fed back to the warehouse management system through the IOT gateway. The warehouse management system updates the medical device storage address ID of the current medical device and sends the storage address ID of the storage unit with an idle position to the robot management system; the robot management system generates a corresponding warehousing instruction according to the medical device storage address ID and sends it to the control terminal, and the control terminal controls the idle delivery robot to deliver the current medical device to the storage unit with an idle position;

[0158] If not, the warehouse management system generates a warehousing alarm including the medical device picture features of the current medical device and notifies the administrator;

[0159] At the time of warehousing, first, the image acquisition unit collects pictures of the medical devices to be warehoused and uploads them to the background system. The warehouse management system calls the AI model to identify the images, so as to obtain the types of the current medical devices to be warehoused and the storage address IDs of their corresponding storage locations. Subsequently, the warehouse management system generates a warehouse location retrieval instruction containing the storage address ID of the medical device to be warehoused to determine whether the storage unit where the medical device to be warehoused should be stored is in an idle state. When specifically executed, the warehouse management system needs to generate the corresponding warehouse location retrieval instruction and send it to the Internet of Things intelligent warehouse through the Internet of Things gateway. The intelligent warehouse responds to the instruction and checks whether the storage unit suitable for the current medical device is in an idle warehouse location, so as to avoid the risk of storage failure caused by the inability of the storage unit suitable for the current medical device to accommodate the medical device to be warehoused. Therefore, the warehouse management system can notify the robot to perform warehousing storage according to the inspection results of the idle warehouse locations fed back by the intelligent warehouse. If there is an idle warehouse location, the warehouse management system interacts with the robot management system and sends the address to the robot management system to generate the corresponding warehousing instruction. The warehousing instruction contains the corresponding warehousing operation information and path, and sends it to the control terminal, and the control terminal controls the idle delivery robot to perform the warehousing operation.

[0160] If there is no idle warehouse location, the system will perform an idle warehouse location search. The warehouse management system checks which storage units still have idle warehouse locations. According to the retrieved idle warehouse locations, it instructs the robot to cooperate with the robot management system to control the delivery robot for distribution. At this time, the warehouse management system will update the warehouse location address of the current medical device for warehousing.

[0161] If it cannot be effectively stored, the picture features of the medical device that cannot be effectively stored currently will be sent to the administrator. For example, an inbound alarm can be sent to the administrator through the PDA management (nursing) terminal to let the administrator intervene in the inbound management.

[0162] Therefore, the warehousing operation can be intelligently identified by the system for the address of the medical device to be warehoused and perform an effective warehouse location search for adaptability to avoid ineffective storage.

[0163] (2) Outbound operation

[0164] An outbound notice is sent to the warehouse management system in advance through the PDA nursing terminal. The outbound notice contains a list of the medical devices to be used and the pick-up point address;

[0165] The warehouse management system analyzes the list of the medical devices to be used, obtains the medical device pictures of each of the medical devices to be used in the list, uses the preset medical device AI recognition model to recognize the medical device picture features in the medical device pictures, and analyzes the medical device picture features to obtain the marked medical device storage address ID therein and sends it to the robot management system;

[0166] The robot management system generates a corresponding outbound instruction according to the medical device storage address ID and issues it to the control terminal. The control terminal controls the idle delivery robot to go to the storage unit corresponding to the medical device storage address ID and deliver the medical device to be used to the pick-up point.

[0167] For each medical device that needs to be used for outbound operation during the outbound operation, the nurse can send an outbound notice to the warehouse management system in advance through the PDA nursing terminal (prepare in advance to improve efficiency). The outbound notice includes the list of medical devices to be outbound or used and the pick-up point address, and the robot needs to deliver the outbound medical devices to the pick-up point where the nurse is located. In the list, the nurse can place the pictures of the medical devices to be used in the list to facilitate system recognition. After receiving the outbound notice, the warehouse management system can parse the list, obtain each medical device picture therein, extract the picture features therein in the same way as recognized by the above AI model, and read the storage address ID of the medical device to be outbound marked in the features, and send it to the robot management system. The robot management system can go to the corresponding storage unit according to the recognized and read storage address ID, receive the corresponding medical device and deliver it to the pick-up point.

[0168] In the present invention, a corresponding manipulator can be configured on the robot to grab the corresponding medical device (the storage address can include corresponding coordinates such as height, angle, and three axes to control the robot to automatically grab the medical device). Or when specifically grabbing, the administrator can take down the medical device and place it on the robot and let the robot deliver it. In this aspect, the action design of the robot for automatically identifying and grabbing the medical device is not limited, and only the robot needs to be controlled to reach the storage unit where the corresponding storage address is located.

[0169] Therefore, it can combine the AI recognition technology to further improve the intelligent screening management and inbound and outbound operations of medical devices and improve the use efficiency.

[0170] Each module or step of the present invention described above can be implemented by a general-purpose computing system. They can be concentrated on a single computing system or distributed over a network composed of multiple computing systems. Optionally, they can be implemented by program code executable by the computing system. Thus, they can be stored in a storage system and executed by the computing system, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0171] Embodiment 3

[0172] Furthermore, on the other hand, the present application also proposes an electronic device, including:

[0173] A processor;

[0174] A memory for storing instructions executable by the processor;

[0175] Wherein, when the processor is configured to execute the executable instructions, it implements a method for AI screening management of medical devices based on image recognition described in Embodiment 2.

[0176] The electronic device according to an embodiment of the present disclosure includes a processor and a memory for storing instructions executable by the processor. Wherein, when the processor is configured to execute the executable instructions, it implements a method for AI screening management of medical devices based on image recognition described in Embodiment 2 above.

[0177] Here, it should be noted that the number of processors can be one or more. At the same time, in the electronic device according to an embodiment of the present disclosure, an input system and an output system can also be included. Wherein, the processor, the memory, the input system, and the output system can be connected through a bus or in other ways, which is not specifically limited herein.

[0178] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and various modules, such as: programs or modules corresponding to a method for AI screening management of medical devices based on image recognition according to an embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0179] The input system can be used to receive input numbers or signals. Wherein, the signal can be a key signal related to user settings and function control of the device / terminal / server. The output system can include a display device such as a display screen.

[0180] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of technologies in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. An AI screening management system for medical devices based on image recognition, characterized in that, Comprising: An Internet of Things intelligent warehouse for storing medical devices; An image acquisition unit for acquiring images of the medical devices during outbound / inbound and uploading them to the medical device management background; An IOT gateway for IoT communication between the image acquisition unit and the medical device management background; A PDA nursing terminal for logging in to the medical device management background to implement corresponding outbound / inbound operations; A medical device management background for the outbound / inbound management of medical devices, including: responding to outbound / inbound operations, identifying the current medical device pictures during outbound / inbound through a preset medical device AI recognition model to obtain the storage address ID of the current medical device during outbound / inbound, and controlling the delivery robot to perform outbound / inbound on the current medical device according to the storage address ID; and, managing the Internet of Things intelligent warehouse and the delivery robot according to the outbound / inbound operation information and recording their status data; wherein, the method for generating the medical device AI recognition model includes: (1) Obtain a number of medical device pictures; (2) Perform feature engineering on the medical device pictures to extract the medical device picture features therein; (3) Add address annotations to the medical device picture features, and label the storage address ID of the storage unit adapted to the medical device in the Internet of Things intelligent warehouse on the medical device picture features to generate annotated features; (4) Based on the annotated features, construct a feature set and divide it into a training set and a validation set according to a preset ratio; (5) Input the training set into a preset CNN model for feature training and learning to generate the medical device AI recognition model; (6) Use the validation set to verify / optimize the medical device AI recognition model. If the verification is qualified, proceed to the next step; otherwise, re-enter the feature engineering step; (7) Deploy and apply the medical device AI recognition model to the warehouse management system; A delivery robot for responding to the control operations of the medical device management background; The Internet of Things intelligent warehouse and the image acquisition unit are respectively communicatively connected to the medical device management background through the IOT gateway; The PDA nursing terminal and the delivery robot are respectively communicatively connected to the medical device management background.

2. The AI screening management system for medical devices based on image recognition according to claim 1, wherein The medical device management background includes a warehouse management system and a robot management system, wherein: The warehouse management system is used to manage the Internet of Things intelligent warehouse, on which the medical device AI recognition model is deployed; The robot management system is used to manage the delivery robot and communicate and control with the delivery robot through a control terminal; The warehouse management system collects the medical device pictures uploaded by the image acquisition unit, imports the medical device pictures into the preset medical device AI recognition model, the medical device AI recognition model identifies the medical device picture features in the medical device pictures, and analyzes the medical device picture features to obtain the labeled medical device storage address ID and send it to the robot management system; The robot management system generates corresponding outbound / inbound instructions according to the medical device storage address ID and sends them to the control terminal. The control terminal issues the outbound / inbound instructions to the idle delivery robots according to the working status of each delivery robot.

3. The AI screening management system for medical devices based on image recognition according to claim 1, characterized in that, The IoT intelligent warehouse includes: A controller for logical control; A memory for storing the storage address ID of medical devices; A communication unit for communicating with the medical device management background through the IOT gateway; A plurality of storage units for storing medical devices; The memory, the communication unit and the storage unit are respectively communicatively connected to the controller.

4. A medical device AI screening management method based on image recognition, which is implemented based on the medical device AI screening management system according to any one of claims 1-3, and is characterized in that, Including the following steps: (1) Inbound operation The warehouse management system generates a warehouse location retrieval instruction containing the medical device storage address ID and sends it to the IoT intelligent warehouse through the IOT gateway; The IoT intelligent warehouse responds to the warehouse location retrieval instruction and checks whether the storage unit suitable for the current medical device is in an idle warehouse location: If it is idle, it sends a feedback to the warehouse management system through the IOT gateway and sends the medical device storage address ID of the current medical device to the robot management system. The robot management system generates a corresponding inbound instruction according to the medical device storage address ID and sends it to the control terminal. The control terminal controls the idle delivery robot to deliver the current medical device to the storage unit suitable for the current medical device; If the storage unit suitable for the current medical device is not in an idle warehouse location, the warehouse management system generates an idle warehouse location retrieval instruction and sends it to the IoT intelligent warehouse through the IOT gateway; The IoT intelligent warehouse responds to the idle warehouse location retrieval instruction and checks whether there is a storage unit with an idle warehouse location: If there is, it feeds back the storage address ID of the storage unit with an idle warehouse location to the warehouse management system through the IOT gateway. The warehouse management system updates the medical device storage address ID of the current medical device and sends the storage address ID of the storage unit with an idle warehouse location to the robot management system; The robot management system generates a corresponding inbound instruction according to the medical device storage address ID and sends it to the control terminal. The control terminal controls the idle delivery robot to deliver the current medical device to the storage unit with an idle warehouse location; If not, the warehouse management system generates an inbound alarm containing the medical device picture features of the current medical device and notifies the administrator; (2) Outbound operation An outbound notice is sent to the warehouse management system in advance through the PDA nursing terminal. The outbound notice includes a list of medical devices to be used and the pick-up point address; The warehouse management system analyzes the list of medical devices to be used, obtains the medical device pictures of each medical device in the list, uses the preset medical device AI recognition model to recognize the medical device picture features in the medical device pictures, analyzes the medical device picture features, obtains the medical device storage address ID marked therein and sends it to the robot management system; The robot management system generates corresponding outbound instructions based on the medical device storage address ID and sends them to the control terminal. The control terminal controls the idle delivery robot to go to the storage unit corresponding to the medical device storage address ID to deliver the medical devices to be used to the pickup point.

5. An electronic device, characterized in that, include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement an image recognition-based medical device AI screening management method as described in claim 4 when executing the executable instructions.

Images