Intelligent logistics distribution management system and method for operating room consumables

Abstract

The present application discloses an intelligent logistics distribution management system and method for operating room consumables, which relates to the field of intelligent logistics distribution technology. It uses cameras and lidar sensors to obtain real-time images and location information of transportation, and uses deep learning algorithms to process and analyze these data, thereby mining the semantic interaction features of transportation status and location information containing full-time domain information, characterizing the obstacle feature information around the intelligent transportation vehicle for operating room consumables, so as to achieve rapid identification of obstacles, thereby achieving rapid identification of obstacles and intelligent obstacle avoidance warning, greatly improving the safety and efficiency of the intelligent transportation vehicle for operating room consumables, and providing reliable support for the distribution of consumables in medical facilities.

Description

Technical Field

[0001] The present application relates to the field of intelligent logistics and distribution technology, and in particular to an intelligent logistics and distribution management system and method for operating room consumables. Background Art

[0002] In the operating room, the distribution of consumables and medicines is the responsibility of circulating nurses, who are on call at all times and are not allowed to leave the operating room without authorization except in special circumstances. On the one hand, the current shortage of human resources in the operating room causes a large workload for nurses; on the other hand, leaving the operating room causes interruptions in the surgical process. The cumulative impact of small interruptions may include delays in surgery and increased ineffective patient stay time in the operating room, thus affecting the safety of the operating room; on the other hand, there is a risk of human error in the manual distribution of surgical consumables or items, and circulating nurses are more likely to make mistakes based on handwritten requisition forms or verbal notifications; at the same time, there is confusion in the collection of consumables, incomplete, irregular, inaccurate information records, and consumables are easily lost.

[0003] Therefore, an intelligent logistics distribution management system and method for operating room consumables is desired. Summary of the invention

[0004] The present application provides an intelligent logistics distribution management system and method for operating room consumables, which uses cameras and lidar sensors to obtain real-time images and location information of transportation, and uses deep learning algorithms to process and analyze these data, thereby mining the semantic interaction features of transportation status and location information containing full-time domain information, and characterizing the obstacle feature information around the intelligent transportation vehicle for operating room consumables to achieve rapid identification of obstacles, thereby achieving rapid identification of obstacles and intelligent obstacle avoidance warning, greatly improving the safety and efficiency of the intelligent transportation vehicle for operating room consumables, and providing reliable support for the distribution of consumables in medical facilities.

[0005] The present application also provides an intelligent logistics distribution management system for operating room consumables, which includes:

[0006] Intelligent storage cabinet for operating room consumables, used to store operating room consumables;

[0007] An intelligent sorting robot for operating room consumables, used to take out corresponding operating room consumables from the intelligent storage cabinet for operating room consumables according to sorting instructions, and to load the corresponding operating room consumables into a logistics box;

[0008] An intelligent transport vehicle for operating room consumables, used to transport the logistics box to a designated operating room according to a transport instruction, the intelligent transport vehicle for operating room consumables including an obstacle avoidance subsystem;

[0009] A backend management system, used for generating the sorting instructions and the transportation instructions;

[0010] Wherein, the obstacle avoidance subsystem comprises:

[0011] The transport real-time picture acquisition module is used to acquire the time series of the transport real-time pictures collected by the camera;

[0012] A position information acquisition module, used to acquire a time series of position information collected by a laser radar sensor;

[0013] A local time series analysis module, used for performing local time series analysis on the time series of the real-time transport picture and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors;

[0014] A fine-grained cross-modal feature interaction module, used to pass the sequence of the transport state feature vectors and the sequence of the position time series feature vectors through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors;

[0015] The obstacle avoidance warning generation module is used to determine whether to issue an obstacle avoidance warning based on the full time domain correlation features of the sequence of the transportation status-position information semantic interaction feature vector.

[0016] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the local timing analysis module includes: a data preprocessing subunit, which is used to perform data preprocessing on the time series of the real-time transport image and the time series of the position information to obtain a sequence of local time series of the real-time transport image and a sequence of local time series of the position information; a timing feature extraction subunit, which is used to perform timing feature extraction on the sequence of local time series of the real-time transport image and the sequence of local time series of the position information to obtain a sequence of the transport state feature vector and a sequence of the position timing feature vector.

[0017] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the data preprocessing subunit is used to: split the time series of the real-time transportation image and the time series of the location information based on the same time scale to obtain a sequence of local time series of the real-time transportation image and a sequence of local time series of the location information.

[0018] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the timing feature extraction subunit is used to: pass the sequence of local time series of the real-time transportation picture through a transportation picture feature extractor based on a convolutional neural network model to obtain a sequence of transportation state feature vectors; pass the sequence of local time series of the position information through a sequence timing feature extractor based on a one-dimensional convolutional layer to obtain a sequence of position timing feature vectors.

[0019] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the transport image feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

[0020] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the fine-grained cross-modal feature interaction module is used to: process the sequence of the transport state feature vectors and the sequence of the position time series feature vectors using the following fine-grained cross-modal feature interaction formula to obtain the sequence of the transport state-position information semantic interaction feature vectors; wherein the fine-grained cross-modal feature interaction formula is:

[0021] ;

[0022] ;

[0023] in, is the first in the sequence of the transport state feature vector transport state feature vector, is the first in the sequence of the transport state feature vector The transposed vector of the transport state feature vector, is the first position in the sequence of the position time series feature vector Position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, is the first in the sequence of the transport status-location information semantic interaction feature vector transport status-location information semantic interaction feature vector, represents the exponential function operation, is the number of the position time series feature vectors.

[0024] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the obstacle avoidance warning generation module includes: a full-time domain correlation feature extraction unit, which is used to extract the full-time domain correlation features of the sequence of the transport status-position information semantic interaction feature vector to obtain the full-time domain transport status-position information semantic interaction feature vector; a feature distribution optimization unit, which is used to optimize the full-time domain transport status-position information semantic interaction feature vector to obtain an optimized full-time domain transport status-position information semantic interaction feature vector; a classification unit, which is used to pass the optimized full-time domain transport status-position information semantic interaction feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether to issue an obstacle avoidance warning.

[0025] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the full-time domain correlation feature extraction unit is used to: pass the sequence of the transportation status-location information semantic interaction feature vectors through a converter-based full-time domain context association encoding module to obtain the full-time domain transportation status-location information semantic interaction feature vector.

[0026] In the above-mentioned intelligent logistics distribution management system for operating room consumables, the classification unit includes: a fully connected encoding subunit, which is used to use multiple fully connected layers of the classifier to fully connect the full-time domain transportation status-position information semantic interaction feature vector to obtain a coded classification feature vector; a classification subunit, which is used to pass the coded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0027] The present application also provides an intelligent logistics distribution management method for operating room consumables, which includes:

[0028] Store operating room consumables in the operating room consumables smart storage cabinet;

[0029] The operating room consumables intelligent sorting robot takes out corresponding operating room consumables from the operating room consumables intelligent storage cabinet according to the sorting instructions, and puts the corresponding operating room consumables into the logistics box;

[0030] The operating room consumables intelligent transport vehicle transports the logistics box to the designated operating room according to the transport instruction, and the operating room consumables intelligent transport vehicle includes an obstacle avoidance subsystem;

[0031] The backend management system generates the sorting instruction and the transportation instruction;

[0032] Wherein, the obstacle avoidance subsystem comprises:

[0033] Obtain the time series of real-time transportation images captured by the camera;

[0034] Obtaining a time series of position information collected by a lidar sensor;

[0035] Performing local time series analysis on the time series of the real-time transport images and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors;

[0036] The sequence of the transport state feature vectors and the sequence of the position time series feature vectors are passed through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors;

[0037] Based on the full time domain correlation features of the sequence of the transportation status-position information semantic interaction feature vectors, it is determined whether to issue an obstacle avoidance warning.

[0038] Compared with the prior art, the intelligent logistics distribution management system and method for operating room consumables provided by this application can realize rapid identification of obstacles and intelligent obstacle avoidance warning, greatly improving the safety and efficiency of intelligent transport vehicles for operating room consumables, and providing reliable support for the distribution of consumables in medical facilities. Furthermore, it can also realize all-round monitoring and management of the transportation process to ensure that intelligent transport vehicles can avoid collisions and other safety hazards during transportation, and improve the reliability and safety of the entire distribution system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0040] Figure 1 This is a block diagram of an intelligent logistics distribution management system for operating room consumables provided in an embodiment of the present application.

[0041] Figure 2 This is a flow chart of an intelligent logistics distribution management method for operating room consumables provided in an embodiment of the present application.

[0042] Figure 3 This is a data flow diagram of an intelligent logistics distribution management method for operating room consumables provided in an embodiment of the present application.

[0043] Figure 4 This is an application scenario diagram of an intelligent logistics distribution management system for operating room consumables provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present application more clear, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Here, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not intended to limit the present application.

[0045] Unless otherwise specified, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the present application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application.

[0046] In the embodiments of the present application, it should be noted that, unless otherwise specified and limited, the term "connection" should be understood in a broad sense. For example, it can be an electrical connection or a connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0047] It should be noted that the terms "first\second\third" involved in the embodiments of the present application are merely used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\third" can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\third" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0048] The distribution of consumables and medicines in the operating room is a vital part of the operation of the operating room. Usually, this task is the responsibility of circulating nurses. Circulating nurses need to be on call in the operating room at all times and are not allowed to leave the operating room without permission unless there are special circumstances. However, the current distribution system has some defects and challenges:

[0049] Shortage of human resources: Nurses in operating rooms have a heavy workload. The shortage of human resources causes circulating nurses to face greater work pressure, which may affect their work efficiency and concentration.

[0050] Interruptions in surgical procedures: The absence of the circulating nurse from the operating room may cause interruptions in the surgical procedure, which may delay the surgery and increase the patient's ineffective stay time in the operating room, thus affecting the safety and efficiency of the operating room.

[0051] There is a risk of errors in manual delivery: There is a possibility of errors when relying on handwritten requisition forms or verbal notifications, which may lead to incorrect delivery of consumables and thus affect the smooth progress of the operation.

[0052] Confusion in the collection of consumables and problems with information recording: There are problems such as confusion in the collection of consumables, incomplete, irregular, inaccurate information recording, and loss of consumables, which may lead to confusion and delays during surgery.

[0053] To solve these problems, consider introducing automated and digital solutions, such as using an electronic claim system and an automated consumables distribution system, to reduce the workload of circulating nurses, improve the accuracy and efficiency of distribution, reduce the possibility of human error, and improve the management efficiency of consumables and drugs. At the same time, it is also very important to establish a sound consumables management system and standard operating procedures to ensure the accuracy and safety of distribution.

[0054] In one embodiment of the present application, Figure 1 FIG. 1 is a block diagram of an intelligent logistics distribution management system for operating room consumables provided in an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the intelligent logistics distribution management system 100 for operating room consumables includes: an intelligent storage cabinet 110 for operating room consumables, which is used to store operating room consumables; an intelligent sorting robot 120 for operating room consumables, which is used to take out corresponding operating room consumables from the intelligent storage cabinet for operating room consumables according to sorting instructions, and put the corresponding operating room consumables into a logistics box; an intelligent transport vehicle 130 for operating room consumables, which is used to transport the logistics box to a designated operating room according to the transport instruction, and the intelligent transport vehicle for operating room consumables includes an obstacle avoidance subsystem; a background management system 140 is used to generate the sorting instruction and the transport instruction; wherein the obstacle avoidance subsystem includes: a real-time transport picture acquisition module 150, which is used to acquire a time series of real-time transport pictures collected by a camera; The location information acquisition module 151 is used to obtain the time series of location information collected by the lidar sensor; the local time series analysis module 152 is used to perform local time series analysis on the time series of the real-time transportation picture and the time series of the location information to obtain a sequence of transportation state feature vectors and a sequence of location time series feature vectors; the fine-grained cross-modal feature interaction module 153 is used to pass the sequence of transportation state feature vectors and the sequence of location time series feature vectors through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transportation state-location information semantic interaction feature vectors; the obstacle avoidance warning generation module 154 is used to determine whether to issue an obstacle avoidance warning based on the full time domain correlation features of the sequence of transportation state-location information semantic interaction feature vectors.

[0055] Among them, the intelligent storage cabinet 110 for operating room consumables is used to store operating room consumables and ensure the safe storage of consumables, including real-time monitoring and management functions of consumable inventory, so as to replenish and update inventory information in time. The intelligent sorting robot 120 for operating room consumables takes out the corresponding operating room consumables from the intelligent storage cabinet according to the sorting instructions, and loads the required operating room consumables into the logistics box for subsequent transportation. The intelligent transport vehicle 130 for operating room consumables transports the logistics box to the designated operating room according to the transportation instructions, including an obstacle avoidance subsystem for avoiding collisions and ensuring safe transportation. The background management system 140 generates sorting instructions and transportation instructions to coordinate the sorting and transportation processes, including monitoring and management functions for inventory, distribution process and equipment status.

[0056] For each module in the obstacle avoidance subsystem, the transport real-time picture acquisition module 150 acquires the time series of the transport real-time picture collected by the camera to monitor the environment around the transport vehicle in real time, including obstacles, other vehicles and pedestrians, etc., and provide visual information for obstacle avoidance decisions. The position information acquisition module 151 acquires the time series of the position information collected by the laser radar sensor to acquire the position information of the transport vehicle in real time, including coordinates, speed and direction, etc., to provide spatial information for obstacle avoidance decisions. The local time series analysis module 152 performs local time series analysis on the time series of the transport real-time picture and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors. By analyzing the real-time picture and position information, key transport state features and position features are extracted to provide basic data for subsequent cross-modal feature interaction. The fine-grained cross-modal feature interaction module 153 passes the sequence of transport state feature vectors and the sequence of position time series feature vectors through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors. Through cross-modal feature interaction, visual information and spatial information are effectively integrated, which improves the ability to understand and represent transportation status and location information. The obstacle avoidance warning generation module 154 determines whether to issue an obstacle avoidance warning based on the full-time domain correlation characteristics of the sequence of transportation status-location information semantic interaction feature vectors. According to the comprehensive transportation status and location information characteristics, it is timely judged whether there is a potential collision risk and a corresponding obstacle avoidance warning is generated, thereby ensuring the safety of the intelligent transportation vehicle during transportation.

[0057] Among them, in the process of transporting the logistics box to the operating room, due to the complex environment of the operating room, there are many obstacles, such as operating tables, other surgical equipment, personnel, etc. The existence of the obstacle avoidance subsystem can help the operating room consumables intelligent transport vehicle detect and avoid these obstacles during movement, ensuring the safe and smooth delivery of the logistics box to the designated operating room. However, in the prior art, there are some problems with the obstacle avoidance function. For example, there are problems such as slow response speed and system failure. Specifically, if the obstacle avoidance subsystem has a slow response speed, it may not be able to respond to sudden obstacles in time, increasing the risk of collision with obstacles. And if the obstacle avoidance subsystem itself may have a fault, it will cause it to fail to work properly, or even cause some unexpected situations. Therefore, a solution is expected.

[0058] In response to the above-mentioned technical problems, the technical concept of the present application is to use cameras and lidar sensors to obtain real-time images and location information of transportation, and use deep learning algorithms to process and analyze these data, thereby mining the semantic interaction characteristics of the transportation status and location information containing full time domain information, and characterizing the obstacle feature information around the operating room consumables intelligent transport vehicle, thereby achieving rapid identification of obstacles, and performing intelligent obstacle avoidance warnings, thereby improving the safety and efficiency of the transport vehicle.

[0059] Among them, cameras and lidar sensors can provide rich full-time information, including visual information and spatial information, covering the environmental status and position characteristics around the transport vehicle. Deep learning algorithms can efficiently process and analyze these data and mine key features, including obstacles, road conditions, and the status of the transport vehicle itself. Through deep learning algorithms, the semantic interaction features of the transport status and position information containing full-time information can be mined from the real-time transport images and position information, which can effectively characterize the obstacle feature information around the intelligent transport vehicle for operating room consumables, including semantic information in terms of position, shape, and motion status. Based on the analysis of semantic interaction features by deep learning algorithms, obstacles around the transport vehicle can be quickly identified and intelligent obstacle avoidance warnings can be performed. This intelligent obstacle avoidance warning can help the transport vehicle avoid obstacles in time, improving the safety and efficiency of the transportation process.

[0060] Based on this, in the technical solution of the present application, the encoding process of the obstacle avoidance subsystem includes: first, obtaining the time series of the real-time transport images collected by the camera; at the same time, obtaining the time series of the position information collected by the laser radar sensor. Among them, the time series of the real-time transport images collected by the camera is to obtain visual information about the environment inside the operating room. The camera can capture images of obstacles, personnel and other objects around the transport vehicle. Through the continuous acquisition of the camera, the dynamic changes on the path of the transport vehicle can be monitored in real time, including the position and distance changes of obstacles, providing the necessary visual input for the obstacle avoidance subsystem. The time series of position information collected by the laser radar sensor is obtained to obtain accurate data about the position of the transport vehicle itself and the position of surrounding objects. Specifically, the laser radar sensor uses a laser beam to scan the surrounding environment and calculates the distance and position of the object by measuring the reflection time of the laser beam. Through the continuous acquisition of the laser radar sensor, the position and motion state of the surrounding obstacles can be detected, providing accurate position data for the obstacle avoidance subsystem.

[0061] In one embodiment of the present application, the local timing analysis module 152 includes: a data preprocessing subunit, which is used to perform data preprocessing on the time series of the real-time transport picture and the time series of the position information to obtain a sequence of local time series of the real-time transport picture and a sequence of local time series of the position information; a timing feature extraction subunit, which is used to perform timing feature extraction on the sequence of local time series of the real-time transport picture and the sequence of local time series of the position information to obtain a sequence of the transport state feature vector and a sequence of the position timing feature vector.

[0062] Then, the time series of the real-time transport picture and the time series of the location information are segmented based on the same time scale to obtain a sequence of local time series of the real-time transport picture and a sequence of local time series of the location information; and the sequence of local time series of the real-time transport picture is passed through a transport picture feature extractor based on a convolutional neural network model to obtain a sequence of transport state feature vectors; at the same time, the sequence of local time series of the location information is passed through a sequence time series feature extractor based on a one-dimensional convolutional layer to obtain a sequence of location time series feature vectors. Among them, segmenting the time series of the real-time transport picture and the time series of the location information based on the same time scale can ensure that they have the same time span, so as to perform subsequent feature extraction and modeling. In addition, by segmenting them into a sequence of local time series of the real-time transport picture and a sequence of local time series of the location information, the dynamic changes and time series correlations in different time periods can be better captured. At the same time, in the technical solution of the present application, a convolutional neural network model and a one-dimensional convolutional layer are respectively used to construct a transportation picture feature extractor and a sequence time series feature extractor to capture the visual features related to obstacles in the sequence of the local time series of the real-time transportation picture, and to capture the time series patterns and associated change characteristics of the position information in the sequence of the local time series of the position information.

[0063] In a specific embodiment of the present application, the data preprocessing subunit is used to: split the time series of the real-time transportation picture and the time series of the position information based on the same time scale to obtain a sequence of local time series of the real-time transportation picture and a sequence of local time series of the position information.

[0064] Furthermore, in a specific embodiment of the present application, the timing feature extraction subunit is used to: pass the sequence of local time series of the real-time transportation picture through a transportation picture feature extractor based on a convolutional neural network model to obtain a sequence of transportation state feature vectors; pass the sequence of local time series of the position information through a sequence timing feature extractor based on a one-dimensional convolutional layer to obtain a sequence of position timing feature vectors.

[0065] Among them, the transport picture feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

[0066] Next, the sequence of the transport state feature vectors and the sequence of the position time series feature vectors are passed through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors. That is, the visual feature information and position feature information in the transport process are fused through the cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a richer feature representation. Among them, the cross-modal feature interaction module based on a fine-grained attention mechanism can allow the visual features and position features in the transport process to complement each other, thereby improving the expressive power and semantic representation of the features. Specifically, the cross-modal feature interaction module based on a fine-grained attention mechanism constructs a correlation relationship between each transport state feature vector and each position time series feature vector, and fuses and interacts based on this correlation relationship. In this way, the interaction between visual feature information and position feature information is realized, so that the transport state feature can pay attention to the detail information related to the position change, and the position time series feature can also be affected by the transport state, and is reflected in the sequence of the transport state-position information semantic interaction feature vector.

[0067] In a specific embodiment of the present application, the fine-grained cross-modal feature interaction module is used to: process the sequence of the transport state feature vectors and the sequence of the location time series feature vectors using the following fine-grained cross-modal feature interaction formula to obtain the sequence of the transport state-location information semantic interaction feature vectors; wherein the fine-grained cross-modal feature interaction formula is:

[0068] ;

[0069] ;

[0070] in, is the first in the sequence of the transport state feature vector transport state feature vector, is the first in the sequence of the transport state feature vector The transposed vector of the transport state feature vector, is the first position in the sequence of the position time series feature vector Position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, is the first in the sequence of the transport status-location information semantic interaction feature vector transport status-location information semantic interaction feature vector, represents the exponential function operation, is the number of the position time series feature vectors.

[0071] In one embodiment of the present application, the obstacle avoidance warning generation module 154 includes: a full-time domain correlation feature extraction unit, which is used to extract the full-time domain correlation features of the sequence of the transport status-position information semantic interaction feature vector to obtain the full-time domain transport status-position information semantic interaction feature vector; a feature distribution optimization unit, which is used to optimize the full-time domain transport status-position information semantic interaction feature vector to obtain an optimized full-time domain transport status-position information semantic interaction feature vector; a classification unit, which is used to pass the optimized full-time domain transport status-position information semantic interaction feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether to issue an obstacle avoidance warning.

[0072] Subsequently, the sequence of the transport state-position information semantic interaction feature vectors is passed through a converter-based full-time domain context association coding module to obtain a full-time domain transport state-position information semantic interaction feature vector. In this way, global context information is introduced through the converter-based full-time domain context association coding module to promote feature communication between each transport state-position information semantic interaction feature vector, which helps to better understand the dynamic changes of transport state and position information in the sequence of transport state-position information semantic interaction feature vectors, thereby improving the expressiveness and robustness of the overall feature.

[0073] In a specific embodiment of the present application, the full-time domain association feature extraction unit is used to: pass the sequence of the transportation status-location information semantic interaction feature vectors through a converter-based full-time domain context association encoding module to obtain the full-time domain transportation status-location information semantic interaction feature vector.

[0074] In the technical solution of the present application, the sequence of the transport state feature vectors and the sequence of the position time series feature vectors respectively express the local image semantic features of the real-time transport picture in the local time domain determined by sequence segmentation in the global time domain and the local time series association features of the position information in the local time domain determined by sequence segmentation in the global time domain. Therefore, the sequence of the transport state feature vectors and the sequence of the position time series feature vectors have differences in the representation of cross-modal semantic feature distribution information based on the local time domain due to differences in the cross-modal time series distribution of the source data.

[0075] Therefore, when the sequence of the transport state feature vectors and the sequence of the position time series feature vectors pass through the cross-modal feature interaction module based on the fine-grained attention mechanism, the inconsistency in the distribution information representation of the sequence of the transport state feature vectors and the sequence of the position time series feature vectors during the cross-modal feature interaction based on the fine-grained attention mechanism will be taken into account. This will lead to poor iterative consistency in the convergence mapping of the full-time domain transport state-position information semantic interaction feature vector to the predetermined class probability category representation in the probability space, thereby reducing the accuracy of the classification results.

[0076] Based on this, the present application optimizes the full-time-domain transport state-location information semantic interaction feature vector to obtain an optimized full-time-domain transport state-location information semantic interaction feature vector, including: for the first eigenvalue of the full-time-domain transport state-location information semantic interaction feature vector and the second eigenvalue , calculate the absolute value of the difference divided by the sum of the absolute values ​​and then correct the relative probability value, wherein the correction makes the relative probability value greater than zero and less than one; calculate the first product of the first eigenvalue multiplied by the relative probability value, the second product of the second eigenvalue multiplied by the absolute value of the difference between the relative probability value and one, the first quotient of the first eigenvalue divided by the absolute value of the difference between the relative probability value and one, and the second quotient of the second eigenvalue divided by the relative probability value; after adding the first product and the second product, further subtract the difference between the first quotient and the second quotient to obtain the value equal to the first eigenvalue and the second eigenvalue The corresponding eigenvalues ​​of the corresponding optimization matrix ; Multiply the optimization matrix with the full-time-domain transportation status-position information semantic interaction feature vector to obtain an optimized full-time-domain transportation status-position information semantic interaction feature vector.

[0077] Specifically, it is expressed as:

[0078] ;

[0079] ;

[0080] ;

[0081] in, represents the full-time transport status-location information semantic interaction feature vector, and respectively represent the first eigenvalue and the second eigenvalue of the full-time-domain transport status-position information semantic interaction feature vector, represents the relative probability value, Represents the first eigenvalue and the second eigenvalue The corresponding eigenvalues ​​of the corresponding optimization matrix, Represents the first eigenvalue and the second eigenvalue The corresponding optimization matrix is, represents matrix multiplication, Represents the optimized full-time domain transportation status-location information semantic interaction feature vector.

[0082] Among them, the first eigenvalue and the second eigenvalue The corresponding relative probability value It is expressed as:

[0083] ;

[0084] in, and respectively represent the first eigenvalue and the second eigenvalue of the full-time-domain transport status-position information semantic interaction feature vector, represents the predetermined hyperparameters, Represents the relative probability value.

[0085] Here, with respect to the region-boundary integral relationship of the high-dimensional feature manifold of the full-time-domain transport status-position information semantic interaction feature vector, by defining the overall probability distribution representation of the feature set of the full-time-domain transport status-position information semantic interaction feature vector as a manifold constraint boundary, the high-dimensional manifold representation of the feature set of the full-time-domain transport status-position information semantic interaction feature vector, which is composed of eigenvalue pairs, is approximated to avoid the ambiguity in the mapping of the diversified feature representation of the full-time-domain transport status-position information semantic interaction feature vector to the local manifold representation in the high-dimensional feature classification regression convergence space, and to improve the execution iteration consistency of each local feature distribution in the mapping task, so as to improve the feature classification regression convergence effect and the accuracy of the classification results.

[0086] Then, the optimized full-time-domain transportation status-position information semantic interaction feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether to issue an obstacle avoidance warning.

[0087] In a specific embodiment of the present application, the classification unit includes: a fully connected encoding subunit, which is used to use multiple fully connected layers of the classifier to fully connect the full time domain transportation status-position information semantic interaction feature vector to obtain an encoded classification feature vector; a classification subunit, which is used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0088] The classifier can distinguish and identify possible obstacles based on the features extracted from the semantic interaction feature vector of the full-time transport status-position information. This classification result can accurately indicate whether there is a potential collision risk, thereby achieving accurate obstacle avoidance warning. The classification result can provide real-time decision support for the intelligent transport vehicle. When the system determines that there is a collision risk, it can immediately issue an obstacle avoidance warning, prompting the transport vehicle to take corresponding actions, such as slowing down, changing lanes or stopping, so as to avoid potential dangerous situations. Through accurate obstacle avoidance warnings, the safety of intelligent transport vehicles inside medical facilities can be improved, and collisions and accidents can be avoided. At the same time, timely obstacle avoidance decisions can also help improve the efficiency of the transportation process and ensure the timely delivery of consumables.

[0089] In summary, an intelligent logistics distribution management system 100 for operating room consumables based on an embodiment of the present application is explained, which uses cameras and lidar sensors to obtain real-time images and location information of transportation, and uses deep learning algorithms to process and analyze these data, thereby mining the semantic interaction features of the transportation status and location information containing full time domain information, and characterizing the obstacle feature information around the intelligent transport vehicle for operating room consumables, thereby achieving rapid identification of obstacles, and performing intelligent obstacle avoidance warnings, thereby improving the safety and efficiency of the transport vehicle.

[0090] As described above, the intelligent logistics distribution management system 100 for operating room consumables according to the embodiment of the present application can be implemented in various terminal devices, such as a server for intelligent logistics distribution management of operating room consumables. In one example, the intelligent logistics distribution management system 100 for operating room consumables according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent logistics distribution management system 100 for operating room consumables can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the intelligent logistics distribution management system 100 for operating room consumables can also be one of the many hardware modules of the terminal device.

[0091] Alternatively, in another example, the operating room consumables intelligent logistics distribution management system 100 and the terminal device may also be separate devices, and the operating room consumables intelligent logistics distribution management system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0092] Figure 2 This is a flow chart of an intelligent logistics distribution management method for operating room consumables provided in an embodiment of the present application. Figure 3 Schematic diagram of the system architecture of an intelligent logistics distribution management method for operating room consumables provided in an embodiment of the present application. Figure 2 and Figure 3As shown, a method for intelligent logistics distribution management of operating room consumables includes: 210, storing operating room consumables in an intelligent storage cabinet for operating room consumables; 220, an intelligent sorting robot for operating room consumables takes out corresponding operating room consumables from the intelligent storage cabinet for operating room consumables according to a sorting instruction, and puts the corresponding operating room consumables into a logistics box; 230, an intelligent transport vehicle for operating room consumables transports the logistics box to a designated operating room according to a transport instruction, and the intelligent transport vehicle for operating room consumables includes an obstacle avoidance subsystem; 240, a background management system generates the sorting instruction and the transport instruction; wherein the obstacle avoidance subsystem includes: 250, obtaining real-time transport information collected by a camera time series of images; 251, obtaining the time series of position information collected by the lidar sensor; 252, performing local time series analysis on the time series of the real-time transport images and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors; 253, passing the sequence of transport state feature vectors and the sequence of position time series feature vectors through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors; 254, determining whether to issue an obstacle avoidance warning based on the full time domain correlation features of the sequence of transport state-position information semantic interaction feature vectors.

[0093] Those skilled in the art will appreciate that the specific operations of each step in the above-mentioned operating room consumables intelligent logistics distribution management method have been referenced above. Figure 1 The invention has been introduced in detail in the description of the intelligent logistics distribution management system for operating room consumables, and therefore, its repeated description will be omitted.

[0094] Figure 4 This is an application scenario diagram of an intelligent logistics distribution management system for operating room consumables provided in an embodiment of the present application. Figure 4 As shown, in this application scenario, first, a time series of real-time transport images captured by a camera is obtained (e.g. Figure 4 C1 shown in FIG); obtaining a time series of position information collected by a lidar sensor (e.g. Figure 4 Then, the time series of the real-time transport images and the time series of the location information are input into a server (e.g., a server) that is deployed with an intelligent logistics distribution management algorithm for operating room consumables. Figure 4 In S) shown in FIG. 1 , the server is capable of processing the time series of the real-time transport image and the time series of the location information based on the intelligent logistics distribution management algorithm for operating room consumables to determine whether to issue an obstacle avoidance warning.

[0095] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An intelligent logistics distribution management system for operating room consumables, characterized in that: include: Intelligent storage cabinet for operating room consumables, used to store operating room consumables; An intelligent sorting robot for operating room consumables, used to take out corresponding operating room consumables from the intelligent storage cabinet for operating room consumables according to sorting instructions, and to load the corresponding operating room consumables into a logistics box; An intelligent transport vehicle for operating room consumables, used to transport the logistics box to a designated operating room according to a transport instruction, the intelligent transport vehicle for operating room consumables including an obstacle avoidance subsystem; A backend management system, used for generating the sorting instructions and the transportation instructions; Wherein, the obstacle avoidance subsystem comprises: The transport real-time picture acquisition module is used to acquire the time series of the transport real-time pictures collected by the camera; A position information acquisition module, used to acquire a time series of position information collected by a laser radar sensor; A local time series analysis module, used for performing local time series analysis on the time series of the real-time transport picture and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors; A fine-grained cross-modal feature interaction module, used to pass the sequence of the transport state feature vectors and the sequence of the position time series feature vectors through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors; An obstacle avoidance warning generation module, used to determine whether to issue an obstacle avoidance warning based on the full time domain correlation features of the sequence of the transport state-position information semantic interaction feature vector; The fine-grained cross-modal feature interaction module is used to: The sequence of the transport state feature vectors and the sequence of the location time series feature vectors are processed using the following fine-grained cross-modal feature interaction formula to obtain a sequence of the transport state-location information semantic interaction feature vectors; wherein the fine-grained cross-modal feature interaction formula is: ； ； in, is the first in the sequence of the transport state feature vector transport state feature vector, is the first in the sequence of the transport state feature vector The transposed vector of the transport state feature vector, is the first position in the sequence of the position time series feature vector Position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, For the The transport state feature vector and the The similarity matrix between the position time series feature vectors, is the first in the sequence of the transport status-location information semantic interaction feature vector transport status-location information semantic interaction feature vector, represents the exponential function operation, is the number of the position time series feature vectors; The obstacle avoidance warning generation module includes: A full-time domain correlation feature extraction unit, used to extract the full-time domain correlation features of the sequence of the transport state-position information semantic interaction feature vectors to obtain a full-time domain transport state-position information semantic interaction feature vector; A feature distribution optimization unit, used for optimizing the full-time-domain transport state-position information semantic interaction feature vector to obtain an optimized full-time-domain transport state-position information semantic interaction feature vector; A classification unit is used to pass the optimized full-time-domain transportation status-position information semantic interaction feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether to issue an obstacle avoidance warning.

2. The intelligent logistics distribution management system for operating room consumables according to claim 1 is characterized in that: The local timing analysis module comprises: A data preprocessing subunit, used for performing data preprocessing on the time series of the real-time transport picture and the time series of the position information to obtain a sequence of local time series of the real-time transport picture and a sequence of local time series of the position information; The time series feature extraction subunit is used to extract time series features from the sequence of local time series of the real-time transport images and the sequence of local time series of the position information to obtain the sequence of the transport state feature vectors and the sequence of the position time series feature vectors.

3. The intelligent logistics distribution management system for operating room consumables according to claim 2 is characterized in that: The data preprocessing subunit is used to: The time series of the real-time transport picture and the time series of the position information are segmented based on the same time scale to obtain a sequence of local time series of the real-time transport picture and a sequence of local time series of the position information.

4. The intelligent logistics distribution management system for operating room consumables according to claim 3 is characterized in that: The temporal feature extraction subunit is used to: The sequence of local time series of the real-time transport picture is passed through a transport picture feature extractor based on a convolutional neural network model to obtain a sequence of transport state feature vectors; The sequence of the local time series of the position information is passed through a sequence time series feature extractor based on a one-dimensional convolutional layer to obtain a sequence of the position time series feature vectors.

5. The intelligent logistics distribution management system for operating room consumables according to claim 4 is characterized in that: The transport picture feature extractor based on the convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

6. The intelligent logistics distribution management system for operating room consumables according to claim 5 is characterized in that: The full time domain correlation feature extraction unit is used to: The sequence of the transport status-position information semantic interaction feature vectors is passed through a converter-based full-time domain context association coding module to obtain the full-time domain transport status-position information semantic interaction feature vector.

7. The intelligent logistics distribution management system for operating room consumables according to claim 6 is characterized in that: The classification unit includes: A fully connected encoding subunit, used for performing fully connected encoding on the full-time domain transportation state-position information semantic interaction feature vector using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; The classification subunit is used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

8. An intelligent logistics distribution management method for operating room consumables, using the intelligent logistics distribution management system for operating room consumables according to claim 1, characterized in that: include: Store operating room consumables in the operating room consumables smart storage cabinet; The operating room consumables intelligent sorting robot takes out corresponding operating room consumables from the operating room consumables intelligent storage cabinet according to the sorting instructions, and puts the corresponding operating room consumables into the logistics box; The operating room consumables intelligent transport vehicle transports the logistics box to the designated operating room according to the transport instruction, and the operating room consumables intelligent transport vehicle includes an obstacle avoidance subsystem; The backend management system generates the sorting instruction and the transportation instruction; Wherein, the obstacle avoidance subsystem comprises: Obtain the time series of real-time transportation images captured by the camera; Obtaining a time series of position information collected by a lidar sensor; Performing local time series analysis on the time series of the real-time transport images and the time series of the position information to obtain a sequence of transport state feature vectors and a sequence of position time series feature vectors; The sequence of the transport state feature vectors and the sequence of the position time series feature vectors are passed through a cross-modal feature interaction module based on a fine-grained attention mechanism to obtain a sequence of transport state-position information semantic interaction feature vectors; Based on the full time domain correlation features of the sequence of the transportation status-position information semantic interaction feature vectors, it is determined whether to issue an obstacle avoidance warning.

Images